feat(ingestion): add source to executionRequest #13614

v-tarasevich-blitz-brain · 2025-05-23T16:07:28Z

This PR contains:

refactoring IngestionSource to type class
refactoring ExecutionRequest to type class
adding source to ExecutionRequest

chriscollins3456

i think this is a reasonable path to be going down here! I have a question about whether it's worth it to adjust loadableTypes here but i can see your path making more sense. interested to hear what you think

datahub-graphql-core/src/main/java/com/linkedin/datahub/graphql/GmsGraphQLEngine.java

...l-core/src/main/java/com/linkedin/datahub/graphql/types/ingestion/IngestionSourceMapper.java

datahub-graphql-core/src/main/java/com/linkedin/datahub/graphql/GmsGraphQLEngine.java

...l-core/src/main/java/com/linkedin/datahub/graphql/types/ingestion/IngestionSourceMapper.java

...hql-core/src/main/java/com/linkedin/datahub/graphql/types/ingestion/IngestionSourceType.java

alwaysmeticulous · 2025-05-26T15:05:34Z

🔴 Meticulous spotted visual differences in 78 of 1354 screens tested: view and approve differences detected.

Meticulous evaluated ~8 hours of user flows against your PR.

_{Last updated for commit 5ca672b. This comment will update as new commits are pushed.}

codecov · 2025-05-26T15:12:58Z

Codecov Report

Attention: Patch coverage is 71.17904% with 66 lines in your changes missing coverage. Please review.

❌ Unsupported file format

Upload processing failed due to unsupported file format. Please review the parser error message:
Error parsing JUnit XML in /home/runner/work/datahub/datahub/metadata-io/build/test-results/test/TEST-com.linkedin.metadata.graph.search.opensearch.SearchGraphServiceOpenSearchTest.xml at 117:1052

Caused by:
    RuntimeError: Error converting computed name to ValidatedString
    
    Caused by:
        string is too long
For more help, visit our troubleshooting guide.

Files with missing lines	Patch %	Lines
...com/linkedin/datahub/graphql/GmsGraphQLEngine.java	0.00%	46 Missing ⚠️
...raphql/types/ingestion/ExecutionRequestMapper.java	86.79%	0 Missing and 7 partials ⚠️
...graphql/types/ingestion/IngestionSourceMapper.java	88.88%	2 Missing and 3 partials ⚠️
.../graphql/types/ingestion/ExecutionRequestType.java	87.50%	3 Missing ⚠️
...s/ingest/source/UpsertIngestionSourceResolver.java	71.42%	1 Missing and 1 partial ⚠️
...b/graphql/types/ingestion/IngestionSourceType.java	95.65%	1 Missing ⚠️
...eact/src/app/ingest/source/IngestionSourceList.tsx	0.00%	1 Missing ⚠️
...ct/src/app/ingestV2/source/IngestionSourceList.tsx	0.00%	1 Missing ⚠️

❌ Your patch status has failed because the patch coverage (71.17%) is below the target coverage (75.00%). You can increase the patch coverage or adjust the target coverage.

📢 Thoughts on this report? Let us know!

chriscollins3456 · 2025-05-29T18:22:29Z

first thing is let's try to add some more test coverage to get code coverage at the 75% goal here

chriscollins3456

this is looking good! only one real comment and then adding tests to get to that 75% code coverage in CI and we can merge this

chriscollins3456 · 2025-05-29T18:38:15Z

...hql-core/src/main/java/com/linkedin/datahub/graphql/types/ingestion/IngestionSourceType.java

+  public static final Set<String> ASPECTS_TO_FETCH =
+      ImmutableSet.of(
+          Constants.INGESTION_SOURCE_ENTITY_NAME,
+          Constants.INGESTION_INFO_ASPECT_NAME,
+          Constants.INGESTION_SOURCE_KEY_ASPECT_NAME);


we'll want to add owners to this once we support that fully

also we want to remove INGESTION_SOURCE_ENTITY_NAME here - that's not an aspect we want to fetch

chriscollins3456 · 2025-05-30T17:31:34Z

we have an issue with :datahub-graphql-core:compileTestJava

… ingestion source to class type

chriscollins3456

nice

github-actions bot added the product PR or Issue related to the DataHub UI/UX label May 23, 2025

vercel bot deployed to Preview May 23, 2025 16:28 View deployment

chriscollins3456 reviewed May 23, 2025

View reviewed changes

mariokhoury4 reviewed May 24, 2025

View reviewed changes

...l-core/src/main/java/com/linkedin/datahub/graphql/types/ingestion/IngestionSourceMapper.java Show resolved Hide resolved

...hql-core/src/main/java/com/linkedin/datahub/graphql/types/ingestion/IngestionSourceType.java Outdated Show resolved Hide resolved

v-tarasevich-blitz-brain force-pushed the vt--add-data-for-executions-tab branch from 6222e16 to 6f84867 Compare May 26, 2025 14:59

github-actions bot deployed to datahub-project-web-react (Preview) May 26, 2025 15:04 View deployment

vercel bot deployed to Preview May 26, 2025 15:18 View deployment

v-tarasevich-blitz-brain force-pushed the vt--add-data-for-executions-tab branch from 6f84867 to 3ed448d Compare May 27, 2025 10:49

vercel bot deployed to Preview May 27, 2025 11:07 View deployment

v-tarasevich-blitz-brain marked this pull request as ready for review May 27, 2025 11:32

v-tarasevich-blitz-brain requested a review from chriscollins3456 May 27, 2025 11:32

datahub-cyborg bot added the needs-review Label for PRs that need review from a maintainer. label May 27, 2025

v-tarasevich-blitz-brain mentioned this pull request May 28, 2025

feat(ingestionSource/ownership): pass ownership to ingestion source #13582

Closed

github-actions bot deployed to datahub-wheels (Preview) May 29, 2025 13:17 View deployment

vercel bot had a problem deploying to Preview May 29, 2025 13:23 Failure

v-tarasevich-blitz-brain force-pushed the vt--add-data-for-executions-tab branch from 8e49aef to b47a4e3 Compare May 29, 2025 13:58

v-tarasevich-blitz-brain changed the title ~~feat(ingestionRunsTab): add source to executionRequest~~ feat(ingestion): add source to executionRequest May 29, 2025

v-tarasevich-blitz-brain mentioned this pull request May 29, 2025

feat(ingestion): add execution log tab #13611

Merged

github-actions bot deployed to datahub-wheels (Preview) May 29, 2025 14:06 View deployment

vercel bot had a problem deploying to Preview May 29, 2025 14:20 Failure

github-actions bot deployed to datahub-project-web-react (Preview) May 29, 2025 14:21 View deployment

v-tarasevich-blitz-brain force-pushed the vt--add-data-for-executions-tab branch from b47a4e3 to 393d552 Compare May 29, 2025 15:43

github-actions bot deployed to datahub-wheels (Preview) May 29, 2025 15:46 View deployment

vercel bot had a problem deploying to Preview May 29, 2025 16:03 Failure

datahub-cyborg bot added pending-submitter-response Issue/request has been reviewed but requires a response from the submitter and removed needs-review Label for PRs that need review from a maintainer. labels May 29, 2025

chriscollins3456 reviewed May 29, 2025

View reviewed changes

github-actions bot deployed to datahub-wheels (Preview) May 30, 2025 14:30 View deployment

github-actions bot deployed to datahub-project-web-react (Preview) May 30, 2025 14:41 View deployment

vercel bot had a problem deploying to Preview May 30, 2025 14:43 Failure

datahub-cyborg bot added pending-submitter-response Issue/request has been reviewed but requires a response from the submitter and removed needs-review Label for PRs that need review from a maintainer. labels May 30, 2025

github-actions bot deployed to datahub-wheels (Preview) May 30, 2025 18:59 View deployment

datahub-cyborg bot added needs-review Label for PRs that need review from a maintainer. and removed pending-submitter-response Issue/request has been reviewed but requires a response from the submitter labels May 30, 2025

github-actions bot deployed to datahub-project-web-react (Preview) May 30, 2025 19:06 View deployment

vercel bot had a problem deploying to Preview May 30, 2025 19:09 Failure

github-actions bot deployed to datahub-wheels (Preview) May 30, 2025 19:44 View deployment

github-actions bot deployed to datahub-project-web-react (Preview) May 30, 2025 19:54 View deployment

vercel bot had a problem deploying to Preview May 30, 2025 19:57 Failure

v-tarasevich-blitz-brain force-pushed the vt--add-data-for-executions-tab branch 2 times, most recently from 7ab7565 to e09bd50 Compare May 30, 2025 20:25

datahub-cyborg bot added pending-submitter-response Issue/request has been reviewed but requires a response from the submitter and removed needs-review Label for PRs that need review from a maintainer. labels May 30, 2025

github-actions bot deployed to datahub-wheels (Preview) May 30, 2025 20:31 View deployment

github-actions bot deployed to datahub-project-web-react (Preview) May 30, 2025 20:39 View deployment

vercel bot had a problem deploying to Preview May 30, 2025 20:50 Failure

feat(ingestion): add ingeston source to execution request and extract…

5ca672b

… ingestion source to class type

v-tarasevich-blitz-brain force-pushed the vt--add-data-for-executions-tab branch from e09bd50 to 5ca672b Compare May 30, 2025 20:54

github-actions bot deployed to datahub-wheels (Preview) May 30, 2025 21:00 View deployment

github-actions bot deployed to datahub-project-web-react (Preview) May 30, 2025 21:06 View deployment

vercel bot had a problem deploying to Preview May 30, 2025 21:42 Failure

chriscollins3456 approved these changes May 30, 2025

View reviewed changes

chriscollins3456 merged commit 313997c into master May 30, 2025
76 of 80 checks passed

chriscollins3456 deleted the vt--add-data-for-executions-tab branch May 30, 2025 23:46

kartikey-visa pushed a commit to kartikey-visa/datahub that referenced this pull request Jul 23, 2025

feat(ingestion): add source to executionRequest (datahub-project#13614)

d953528

feat(ingestion): add source to executionRequest #13614

feat(ingestion): add source to executionRequest #13614

Uh oh!

Conversation

v-tarasevich-blitz-brain commented May 23, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

chriscollins3456 left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

alwaysmeticulous bot commented May 26, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

codecov bot commented May 26, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

❌ Unsupported file format

Uh oh!

chriscollins3456 commented May 29, 2025

Uh oh!

chriscollins3456 left a comment

Choose a reason for hiding this comment

Uh oh!

chriscollins3456 May 29, 2025

Choose a reason for hiding this comment

Uh oh!

chriscollins3456 May 29, 2025

Choose a reason for hiding this comment

Uh oh!

chriscollins3456 commented May 30, 2025

Uh oh!

chriscollins3456 left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

v-tarasevich-blitz-brain commented May 23, 2025 •

edited

Loading

alwaysmeticulous bot commented May 26, 2025 •

edited

Loading

codecov bot commented May 26, 2025 •

edited

Loading