Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

(aws-glue-alpha): S3 path specified in --spark-event-logs-path needs to end with slash #29356

Closed
moomindani opened this issue Mar 4, 2024 · 2 comments · Fixed by #29357 · May be fixed by NOUIY/aws-solutions-constructs#98, NOUIY/aws-solutions-constructs#99 or NOUIY/aws-solutions-constructs#101
Closed

(aws-glue-alpha): S3 path specified in --spark-event-logs-path needs to end with slash #29356

moomindani opened this issue Mar 4, 2024 · 2 comments · Fixed by #29357 · May be fixed by NOUIY/aws-solutions-constructs#98, NOUIY/aws-solutions-constructs#99 or NOUIY/aws-solutions-constructs#101
Labels
@aws-cdk/aws-glue Related to AWS Glue bug This issue is a bug. effort/medium Medium work item – several days of effort p2

Comments

@moomindani
Copy link
Contributor

Describe the bug

When only bucket is provided for spark event log path in job, the s3 path does not end with slash.
It is accepted in job API, but it does not work on actual Spark UI.

Expected Behavior

S3 path specified in --spark-event-logs-path ends with slash

Current Behavior

S3 path specified in --spark-event-logs-path does not end with slash in case only bucket is provided but prefix is not provided

Reproduction Steps

      sparkUI: {
        enabled: true,
      },

Possible Solution

Add trailing slash when it does not end with slash

Additional Information/Context

No response

CDK CLI Version

2.131.0

Framework Version

No response

Node.js Version

20.11.1

OS

macOS Ventura 13.6.4

Language

TypeScript

Language Version

No response

Other information

No response
@moomindani moomindani added bug This issue is a bug. needs-triage This issue or PR still needs to be triaged. labels Mar 4, 2024
@github-actions github-actions bot added the @aws-cdk/aws-glue Related to AWS Glue label Mar 4, 2024
@moomindani moomindani mentioned this issue Mar 4, 2024
Merged
1 task
@tim-finnigan tim-finnigan self-assigned this Mar 6, 2024
@tim-finnigan tim-finnigan added investigating This issue is being investigated and/or work is in progress to resolve the issue. and removed investigating This issue is being investigated and/or work is in progress to resolve the issue. labels Mar 6, 2024
@tim-finnigan
Copy link

Thanks for reaching out and creating the PR - it looks like there is already some feedback that is awaiting a response.

@tim-finnigan tim-finnigan added p2 effort/medium Medium work item – several days of effort labels Mar 6, 2024
@tim-finnigan tim-finnigan removed the needs-triage This issue or PR still needs to be triaged. label Mar 6, 2024
@tim-finnigan tim-finnigan removed their assignment Mar 6, 2024
@mergify mergify bot closed this as completed in #29357 Mar 19, 2024
mergify bot pushed a commit that referenced this issue Mar 19, 2024
@moomindani
fix(glue): s3 path specified in --spark-event-logs-path needs to end …
4ff3565 
…with slash (#29357)

### Issue # (if applicable)

Closes #29356.

### Reason for this change

Currently S3 path specified in `--spark-event-logs-path` does not end with slash in case only bucket is provided but prefix is not provided. This parameter causes errors when viewing the event log through Spark UI / Spark history server.

### Description of changes

Add trailing slash when it does not end with slash.

### Description of how you validated changes

Completed unit test and integ test.

### Checklist
- [x] My code adheres to the [CONTRIBUTING GUIDE](https://github.com/aws/aws-cdk/blob/main/CONTRIBUTING.md) and [DESIGN GUIDELINES](https://github.com/aws/aws-cdk/blob/main/docs/DESIGN_GUIDELINES.md)

----

*By submitting this pull request, I confirm that my contribution is made under the terms of the Apache-2.0 license*
@github-actions GitHub Actions
Copy link

⚠️COMMENT VISIBILITY WARNING⚠️

Comments on closed issues are hard for our team to see.
If you need more assistance, please either tag a team member or open a new issue that references this one.
If you wish to keep having a conversation with other community members under this issue feel free to do so.

ahammond pushed a commit to ahammond/aws-cdk that referenced this issue Mar 26, 2024
@moomindani @ahammond
fix(glue): s3 path specified in --spark-event-logs-path needs to end …
8e96e96 
…with slash (aws#29357)

### Issue # (if applicable)

Closes aws#29356.

### Reason for this change

Currently S3 path specified in `--spark-event-logs-path` does not end with slash in case only bucket is provided but prefix is not provided. This parameter causes errors when viewing the event log through Spark UI / Spark history server.

### Description of changes

Add trailing slash when it does not end with slash.

### Description of how you validated changes

Completed unit test and integ test.

### Checklist
- [x] My code adheres to the [CONTRIBUTING GUIDE](https://github.com/aws/aws-cdk/blob/main/CONTRIBUTING.md) and [DESIGN GUIDELINES](https://github.com/aws/aws-cdk/blob/main/docs/DESIGN_GUIDELINES.md)

----

*By submitting this pull request, I confirm that my contribution is made under the terms of the Apache-2.0 license*
@github-actions github-actions bot mentioned this issue Apr 1, 2024
Closed
This was referenced May 23, 2024
Open
Open
@NOUIY NOUIY mentioned this issue May 30, 2024
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
@aws-cdk/aws-glue Related to AWS Glue bug This issue is a bug. effort/medium Medium work item – several days of effort p2
Projects
None yet
Milestone
No milestone
2 participants
@moomindani @tim-finnigan