-
Notifications
You must be signed in to change notification settings - Fork 3.9k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
fix(glue): support Ray jobs with Runtime parameter #25867
Conversation
/** | ||
* Ray 2.4 | ||
*/ | ||
RAY_TWO_FOUR = 'Ray2.4', |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This would be a repetitive operation, wondering if we want to move away from enums
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
should be an enum-like class:
export class Runtime {
public static RAY_TWO_FOUR = new Runtime('Ray2.4');
public constructor(public readonly runtime: string) {}
}
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks for sharing the example. Reflected with minor change.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks for getting started on this @moomindani! Some minor comments
/** | ||
* Ray 2.4 | ||
*/ | ||
RAY_TWO_FOUR = 'Ray2.4', |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
should be an enum-like class:
export class Runtime {
public static RAY_TWO_FOUR = new Runtime('Ray2.4');
public constructor(public readonly runtime: string) {}
}
@@ -150,6 +160,12 @@ interface PythonExecutableProps { | |||
} | |||
|
|||
interface SharedJobExecutableProps { | |||
/** | |||
* Runtime. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
stronger docs needed. need to know when is runtime required, etc
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Added bit more information.
Our public doc does not have much information about this property, but I tried to add as much as possible.
@@ -388,6 +407,13 @@ export interface JobExecutableConfig { | |||
*/ | |||
readonly pythonVersion?: PythonVersion; | |||
|
|||
/** | |||
* The Runtime to use. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
better docs
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Our public doc does not have much information about this property, but I tried to add as much as possible.
AWS CodeBuild CI Report
Powered by github-codebuild-logs, available on the AWS Serverless Application Repository |
Thank you for contributing! Your pull request will be updated from main and then merged automatically (do not update manually, and be sure to allow changes to be pushed to your fork). |
AWS Glue recently changed API and CloudFormation spec to require
Runtime
parameter for Ray jobs.To address the changes, I am submitting this PR.
This PR is for fixing the issue #25787.
Reference
CloudFormation doc has been also updated for
Runtime
parameter.https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-properties-glue-job-jobcommand.html#cfn-glue-job-jobcommand-runtime
By submitting this pull request, I confirm that my contribution is made under the terms of the Apache-2.0 license