Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

cli: automatic multi environment integration tests #32083

Open
1 of 2 tasks
Tracked by #32171
iliapolo opened this issue Nov 11, 2024 · 1 comment
Open
1 of 2 tasks
Tracked by #32171

cli: automatic multi environment integration tests #32083

iliapolo opened this issue Nov 11, 2024 · 1 comment
Labels
effort/medium Medium work item – several days of effort feature-request A feature should be added or improved. p1 package/tools Related to AWS CDK Tools or CLI

Comments

@iliapolo
Copy link
Contributor

iliapolo commented Nov 11, 2024

Describe the feature

When the CLI deploys a stack as part of an integration test, automatically deploy the stack to multiple environments within a single CLI execution

Use Case

A single CLI execution can deploy multiple stacks to multiple environments. This opens the door for bugs that arise from code that doesn't take into account that each environment may contain information that affects the deployment of the stack.

For example, code that uses any bootstrap stack resource must not assume that the resource is the same in all environments.

Automatically deploying stacks to multiple environments will catch these bugs without needing to explicitly write a test for it.

Proposed Solution

There are several technical approaches we can take:

  • Provide a base stack to be used in tests which duplicates itself across multiple environments.
  • Mutate the assembly at runtime.
  • ...

For simplicity from an auth perspective, we can narrow this down to just multi-region tests and not deal with multi account.

Regardless though, since these tests require multiple environments, we'll need to make a change to our region allocator. We might be able to get away with just acquiring an additional region lease here:

public async using<B>(block: (x: A) => B | Promise<B>): Promise<B> {
const lease = await this.take();
try {
return await block(lease.value);
} finally {
await lease.dispose();
}
}

But since we are planning a revamp to our environment allocation service, its likely better to just hold off until we implement it - taking into account requirements surfaced by this use-case.

Other Information

Some peripheral considerations:

  • We need to provide a way for tests to opt out of this behavior (there may be good reasons for it).
  • Do we add this as a permutation of the test? or instead?

Acknowledgements

  • I may be able to implement this feature request
  • This feature might incur a breaking change

CDK version used

ALL

Environment details (OS name and version, etc.)

ALL

@iliapolo iliapolo added feature-request A feature should be added or improved. needs-triage This issue or PR still needs to be triaged. labels Nov 11, 2024
@github-actions github-actions bot added the package/tools Related to AWS CDK Tools or CLI label Nov 11, 2024
@iliapolo iliapolo added p1 and removed needs-triage This issue or PR still needs to be triaged. labels Nov 11, 2024
@mrgrain
Copy link
Contributor

mrgrain commented Nov 11, 2024

I like this idea!

@pahud pahud added the effort/medium Medium work item – several days of effort label Nov 11, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
effort/medium Medium work item – several days of effort feature-request A feature should be added or improved. p1 package/tools Related to AWS CDK Tools or CLI
Projects
None yet
Development

No branches or pull requests

3 participants