Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Programmatically expose the output of CDK list #8436

Closed
benwainwright opened this issue Jun 8, 2020 · 3 comments
Closed

Programmatically expose the output of CDK list #8436

benwainwright opened this issue Jun 8, 2020 · 3 comments
Assignees
Labels
effort/large Large work item – several weeks of effort feature-request A feature should be added or improved. p2 package/tools Related to AWS CDK Tools or CLI

Comments

@benwainwright
Copy link

benwainwright commented Jun 8, 2020

Currently, I can get a list of stacks that will be generated by my project in the order they should be deployed (this is important; see below) by running the command cdk list

It would be useful if the output from this command could be exposed as an api in a way that doesn't require full synthesis of the templates under the hood.

Use Case

On my team in the BBC, we are currently using the CDK to generate templates for 17 CodeBuild projects associated with various different Github repositories, along with three other templates that the others all depend on. Typically in the BBC, we deploy CloudFormation stacks via an internal service that acts as an intermediary between the developer or CI service and the AWS API.

When we first started developing the project that generates the CodeBuild templates, none of them had any dependencies on each other, and so a deployment solution was written that first runs cdk synth to generate the templates, then deploys the templates in filesystem order.

As our use case became more complicated, we started introducing dependencies between templates and we learned that, under the hood, this resulted in imports/exports between templates being implicitly generated

This obviously left us with the problem that we had no guarantee that we would deploy the templates in the correct order, and we quickly ran into deployment failures. We reached out to our AWS account team about this and after a quick call, we got a response identifying that we should use the output of cdk list, since this not only lists the stacks to be generated, but does so in the order they should be deployed.

While this solves our immediate problem (and so this is not a high priority issue for us), it would be really useful to be able to do this programmatically without having to shell out. This would make our tests much cleaner/simpler, and it is also worth noting that cdk list took approx 4 seconds when I ran it which seems surprisingly slow given what it does.

This is a 🚀 Feature Request

@benwainwright benwainwright added feature-request A feature should be added or improved. needs-triage This issue or PR still needs to be triaged. labels Jun 8, 2020
@SomayaB SomayaB added the package/tools Related to AWS CDK Tools or CLI label Jun 8, 2020
@shivlaks
Copy link
Contributor

shivlaks commented Jun 9, 2020

@benwainwright - I believe we shoudl track the use case in #601
I'd like to collect and consolidate the use cases in a single point. This is the most desired feature in the tools space and will likely need an RFC before we get to implementing a solution.

That's an interesting point about cdk list taking 4 seconds. Can you help me understand the layout and organization of your infrastructure a bit better? I think that's worth following up on to see if there are optimizations we can make.

mergify bot pushed a commit that referenced this issue Jun 16, 2020
…8515)


Add a test to verify stacks are serialized to the manifest file in a topological order, based on their dependencies. By adding this test we are committing to maintain the topological order of the stacks in the manifest file, allowing users to consume it without needing to sort the stacks. 

Note that all of the CDK toolkit commands such as `cdk list` and `cdk deploy` already assume this and do not sort stacks upon execution.
> see [`cdk list`](https://github.com/aws/aws-cdk/blob/master/packages/aws-cdk/lib/cdk-toolkit.ts#L264) and [`cdk deploy`](https://github.com/aws/aws-cdk/blob/master/packages/aws-cdk/lib/cdk-toolkit.ts#L111) for reference  

I have initially added the test in `cx-api` module but after more consideration I think it is better suited in the `core` module since we want to verify this behavior in the context of a CDK application. I have left both tests in this PR for the sake of discussion in the event the reviewer thinks this actually fits better in `cx-api`.

helps #8436

----

*By submitting this pull request, I confirm that my contribution is made under the terms of the Apache-2.0 license*
@SomayaB SomayaB removed the needs-triage This issue or PR still needs to be triaged. label Jun 18, 2020
@shivlaks shivlaks added the effort/large Large work item – several weeks of effort label Jun 22, 2020
@shivlaks shivlaks added the p2 label Aug 31, 2020
@benwainwright
Copy link
Author

Closing in favour of #601

@github-actions
Copy link

⚠️COMMENT VISIBILITY WARNING⚠️

Comments on closed issues are hard for our team to see.
If you need more assistance, please either tag a team member or open a new issue that references this one.
If you wish to keep having a conversation with other community members under this issue feel free to do so.

@mrgrain mrgrain changed the title Programatically expose the output of CDK list Programmatically expose the output of CDK list Jan 10, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
effort/large Large work item – several weeks of effort feature-request A feature should be added or improved. p2 package/tools Related to AWS CDK Tools or CLI
Projects
None yet
Development

No branches or pull requests

3 participants