Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Tracking: [Integration Tests] #20384

Open
4 of 18 tasks
corymhall opened this issue May 17, 2022 · 1 comment
Open
4 of 18 tasks

Tracking: [Integration Tests] #20384

corymhall opened this issue May 17, 2022 · 1 comment
Labels
management/tracking Issues that track a subject or multiple issues p2

Comments

@corymhall
Copy link
Contributor

corymhall commented May 17, 2022

Overview

Tracking issue for integ-tests and integ-runner

Link to the service’s CDK Construct Library API reference page.

https://docs.aws.amazon.com/cdk/api/v2/docs/integ-tests-alpha-readme.html

Maturity: CloudFormation Resources Only

Experimental

Implementation

Integ Tests

Features

  • Add HttpApiCall similar to AwsApiCall (in progress)
  • Add the ability to have the assertions "wait" for something to occur (in progress)
    • Move assertion into the *ApiCall resources (in progress)
  • Access to addPolicyStatementFromSdkCall on AssertionsProvider (turns out there are a lot of permissions it can't figure out)

Integ Runner

Features

  • Automatically remove assets above a certain size (e.g. 20KB) - [high]
  • Add a tests remaining log (maybe only if # of tests is > something) - [low]
  • Allow filter by package name (or glob filter, etc) e.g. yarn integ 'aws-lambda/*' - [low]
  • Add flag to integ manifest to skip during automation - [low]
  • Add mass bootstrap of accounts/regions that are used - [low]
  • Add flag to disable snapshot creation
  • Add a flag to disable snapshots altogether (don't create or diff snapshots)
  • Report failed status on individual test cases/stacks

Bugs/Refactor

Issue list

  • [ ]
  • [ ]
@corymhall corymhall added the management/tracking Issues that track a subject or multiple issues label May 17, 2022
@corymhall corymhall self-assigned this May 17, 2022
mergify bot pushed a commit that referenced this issue May 27, 2022
If you run `integ-runner --update-on-failed` and the test succeeds, then
the cli should not return an exit code.

re #20384


----

### All Submissions:

* [ ] Have you followed the guidelines in our [Contributing guide?](https://github.com/aws/aws-cdk/blob/master/CONTRIBUTING.md)

### Adding new Unconventional Dependencies:

* [ ] This PR adds new unconventional dependencies following the process described [here](https://github.com/aws/aws-cdk/blob/master/CONTRIBUTING.md/#adding-new-unconventional-dependencies)

### New Features

* [ ] Have you added the new feature to an [integration test](https://github.com/aws/aws-cdk/blob/master/INTEGRATION_TESTS.md)?
	* [ ] Did you use `yarn integ` to deploy the infrastructure and generate the snapshot (i.e. `yarn integ` without `--dry-run`)?

*By submitting this pull request, I confirm that my contribution is made under the terms of the Apache-2.0 license*
@mrgrain
Copy link
Contributor

mrgrain commented Aug 17, 2022

TypeScript support
Out template for new CDK apps is configured to run jest via ts-node. Integ-runner needs files to be js.

mergify bot pushed a commit that referenced this issue Aug 25, 2022
#21763)

I don't know why I was specifying `nargs: 1` for array arguments. This
tells yargs to only accept a single argument. You can no provide
multiple options, i.e.

```
yarn integ --regions us-east-1 us-east-2
```

re #20384


----

### All Submissions:

* [ ] Have you followed the guidelines in our [Contributing guide?](https://github.com/aws/aws-cdk/blob/main/CONTRIBUTING.md)

### Adding new Unconventional Dependencies:

* [ ] This PR adds new unconventional dependencies following the process described [here](https://github.com/aws/aws-cdk/blob/main/CONTRIBUTING.md/#adding-new-unconventional-dependencies)

### New Features

* [ ] Have you added the new feature to an [integration test](https://github.com/aws/aws-cdk/blob/main/INTEGRATION_TESTS.md)?
	* [ ] Did you use `yarn integ` to deploy the infrastructure and generate the snapshot (i.e. `yarn integ` without `--dry-run`)?

*By submitting this pull request, I confirm that my contribution is made under the terms of the Apache-2.0 license*
josephedward pushed a commit to josephedward/aws-cdk that referenced this issue Aug 30, 2022
aws#21763)

I don't know why I was specifying `nargs: 1` for array arguments. This
tells yargs to only accept a single argument. You can no provide
multiple options, i.e.

```
yarn integ --regions us-east-1 us-east-2
```

re aws#20384


----

### All Submissions:

* [ ] Have you followed the guidelines in our [Contributing guide?](https://github.com/aws/aws-cdk/blob/main/CONTRIBUTING.md)

### Adding new Unconventional Dependencies:

* [ ] This PR adds new unconventional dependencies following the process described [here](https://github.com/aws/aws-cdk/blob/main/CONTRIBUTING.md/#adding-new-unconventional-dependencies)

### New Features

* [ ] Have you added the new feature to an [integration test](https://github.com/aws/aws-cdk/blob/main/INTEGRATION_TESTS.md)?
	* [ ] Did you use `yarn integ` to deploy the infrastructure and generate the snapshot (i.e. `yarn integ` without `--dry-run`)?

*By submitting this pull request, I confirm that my contribution is made under the terms of the Apache-2.0 license*
@corymhall corymhall removed their assignment Apr 15, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
management/tracking Issues that track a subject or multiple issues p2
Projects
None yet
Development

No branches or pull requests

3 participants