Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(stepfunctions-tasks): support dynamic values for Glue Job Worker Type #32453

Merged
merged 4 commits into from
Dec 11, 2024

Conversation

GavinZZ
Copy link
Contributor

@GavinZZ GavinZZ commented Dec 10, 2024

Issue # (if applicable)

Closes #32359

Reason for this change

Now, I can't put the workers type dynamic from payload input in the state machine because it only accept a enum type workerType that its a enum and only accepts the define workers. If i pass a string that references the payload input it shows an error.

Description of changes

Turn the ENUM type WorkerType to a class-based implementation. This should be backward compatible as there's no change on user side.

Description of how you validated changes

New integ tests that use dynamic value for Worker type. When invoking StateMachine with the JSON payload, it will fetch the data correctly and invoke the Glue job with the current worker type and number of workers.

Checklist


By submitting this pull request, I confirm that my contribution is made under the terms of the Apache-2.0 license

@aws-cdk-automation aws-cdk-automation requested a review from a team December 10, 2024 00:55
@github-actions github-actions bot added bug This issue is a bug. effort/medium Medium work item – several days of effort p1 labels Dec 10, 2024
@mergify mergify bot added the contribution/core This is a PR that came from AWS. label Dec 10, 2024
@GavinZZ GavinZZ marked this pull request as ready for review December 10, 2024 01:00
Copy link

codecov bot commented Dec 10, 2024

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 78.67%. Comparing base (9d5ad93) to head (88dc47a).
Report is 1 commits behind head on main.

Additional details and impacted files
@@           Coverage Diff           @@
##             main   #32453   +/-   ##
=======================================
  Coverage   78.67%   78.67%           
=======================================
  Files         107      107           
  Lines        7237     7237           
  Branches     1329     1329           
=======================================
  Hits         5694     5694           
  Misses       1357     1357           
  Partials      186      186           
Flag Coverage Δ
suite.unit 78.67% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

Components Coverage Δ
packages/aws-cdk 78.67% <ø> (ø)

@GavinZZ GavinZZ force-pushed the yuanhaoz/dynamic_worker branch from 4ce6094 to b7916fd Compare December 10, 2024 21:49
@aws-cdk-automation aws-cdk-automation added the pr/needs-maintainer-review This PR needs a review from a Core Team Member label Dec 10, 2024
*
* @default - must choose one of `workerType` or `workerTypeV2`
*/
readonly workerTypeV2?: WorkerTypeV2;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I wonder do we have a naming pattern for this kind of change? e.g. workerTypeV2 or workerTypeEnum

Copy link
Contributor

mergify bot commented Dec 11, 2024

Thank you for contributing! Your pull request will be updated from main and then merged automatically (do not update manually, and be sure to allow changes to be pushed to your fork).

@aws-cdk-automation aws-cdk-automation removed the pr/needs-maintainer-review This PR needs a review from a Core Team Member label Dec 11, 2024
Copy link
Contributor

mergify bot commented Dec 11, 2024

Thank you for contributing! Your pull request will be updated from main and then merged automatically (do not update manually, and be sure to allow changes to be pushed to your fork).

Copy link
Contributor

mergify bot commented Dec 11, 2024

This pull request has been removed from the queue for the following reason: checks failed.

The merge conditions cannot be satisfied due to failing checks:

You should look at the reason for the failure and decide if the pull request needs to be fixed or if you want to requeue it.

If you want to requeue this pull request, you need to post a comment with the text: @mergifyio requeue

Copy link
Contributor

mergify bot commented Dec 11, 2024

Thank you for contributing! Your pull request will be updated from main and then merged automatically (do not update manually, and be sure to allow changes to be pushed to your fork).

@GavinZZ
Copy link
Contributor Author

GavinZZ commented Dec 11, 2024

@mergify update

Copy link
Contributor

mergify bot commented Dec 11, 2024

update

✅ Branch has been successfully updated

@aws-cdk-automation
Copy link
Collaborator

AWS CodeBuild CI Report

  • CodeBuild project: AutoBuildv2Project1C6BFA3F-wQm2hXv2jqQv
  • Commit ID: 88dc47a
  • Result: SUCCEEDED
  • Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

Copy link
Contributor

mergify bot commented Dec 11, 2024

Thank you for contributing! Your pull request will be updated from main and then merged automatically (do not update manually, and be sure to allow changes to be pushed to your fork).

@mergify mergify bot merged commit 7df954c into main Dec 11, 2024
17 checks passed
@mergify mergify bot deleted the yuanhaoz/dynamic_worker branch December 11, 2024 18:51
Copy link

Comments on closed issues and PRs are hard for our team to see.
If you need help, please open a new issue that references this one.

@github-actions github-actions bot locked as resolved and limited conversation to collaborators Dec 11, 2024
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
bug This issue is a bug. contribution/core This is a PR that came from AWS. effort/medium Medium work item – several days of effort p1
Projects
None yet
Development

Successfully merging this pull request may close these issues.

aws-stepfunctions-tasks: workerType not dynamic from payload
3 participants