Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

fix(entity-service): handle no-op system-metadata batches #12055

Merged
merged 9 commits into from
Dec 10, 2024
Merged

fix(entity-service): handle no-op system-metadata batches #12055

merged 9 commits into from
Dec 10, 2024

Conversation

david-leifker
Copy link
Collaborator

@david-leifker david-leifker commented Dec 6, 2024 •
edited
Loading

  • disable legacy locking
  • Sequential duplicates in batches must be removed (otherwise results in no rows changed exceptions)
  • Improved detection of duplicates, log as warning only duplicates
  • Inserts/updates are not guaranteed to be in order. Flush inserts when update is executed if no existing item in db
  • Comparisons of systemMetadata using DataTemplateUtil, doesn't always seem to work, using fallback to string

Checklist

  • The PR conforms to DataHub's Contributing Guideline (particularly Commit Message Format)
  • Links to related issues (if applicable)
  • Tests for the changes have been added/updated (if applicable)
  • Docs related to the changes have been added/updated (if applicable). If a new feature has been added a Usage Guide has been added for the same.
  • For any breaking change/potential downtime/deprecation/big changes an entry has been made in Updating DataHub

@david-leifker
fix(entity-service): handle no-op system-metadata batches
d4e8a82 
* revert previous changes
* disable legacy locking
* handle no-op when no change to system metadata
@github-actions github-actions bot added product PR or Issue related to the DataHub UI/UX devops PR or Issue related to DataHub backend & deployment labels Dec 6, 2024
@datahub-cyborg datahub-cyborg bot added the pending-submitter-response Issue/request has been reviewed but requires a response from the submitter label Dec 6, 2024
@datahub-cyborg datahub-cyborg bot added pending-submitter-merge and removed pending-submitter-response Issue/request has been reviewed but requires a response from the submitter labels Dec 6, 2024
@david-leifker david-leifker force-pushed the no-op-system-metadata-batches branch from 8324b14 to 619a6d8 Compare December 9, 2024 20:23
@david-leifker david-leifker force-pushed the no-op-system-metadata-batches branch from f9861a9 to 54d4434 Compare December 10, 2024 00:36
@david-leifker david-leifker merged commit 0a2ac70 into datahub-project:master Dec 10, 2024
45 checks passed
sleeperdeep pushed a commit to sleeperdeep/datahub that referenced this pull request Dec 17, 2024
@david-leifker @sleeperdeep
fix(entity-service): handle no-op system-metadata batches (datahub-pr…
c9c1b95 
…oject#12055)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@RyanHolstien RyanHolstien RyanHolstien approved these changes

Assignees
No one assigned
Labels
devops PR or Issue related to DataHub backend & deployment pending-submitter-merge product PR or Issue related to the DataHub UI/UX release-0.15.0
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@david-leifker @RyanHolstien