Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Lightning: Skip apply ioworkers when seek(0, io.SeekCurrent) #57454

Merged
merged 1 commit into from
Nov 19, 2024
Merged

Lightning: Skip apply ioworkers when seek(0, io.SeekCurrent) #57454

merged 1 commit into from
Nov 19, 2024

Conversation

xuanyu66
Copy link
Contributor

@xuanyu66 xuanyu66 commented Nov 18, 2024 •
edited
Loading

What problem does this PR solve?

Issue Number: close #57413

Problem Summary:
When importing from cloud storage, high concurrency can lead to bottlenecks in mydump.PooledReader.Seek(0, io.SeekCurrent) due to the application of ioWorkers. Seek(0, io.SeekCurrent) does not cause disk IO operations, applying ioWorkers is unnecessary.

What changed and how does it work?

Skip apply ioworkers when seek(0, io.SeekCurrent)

Check List

Tests

  • Unit test
  • Integration test
  • Manual test (add detailed scripts or steps below)
  • No need to test
    • I checked and no code files have been changed.

Side effects

  • Performance regression: Consumes more CPU
  • Performance regression: Consumes more Memory
  • Breaking backward compatibility

Documentation

  • Affects user behaviors
  • Contains syntax changes
  • Contains variable changes
  • Contains experimental features
  • Changes MySQL compatibility

Release note

Please refer to Release Notes Language Style Guide to write a quality release note.

None

@sre-bot
Copy link
Contributor

sre-bot commented Nov 18, 2024 •
edited
Loading

CLA assistant check
All committers have signed the CLA.

@ti-chi-bot ti-chi-bot bot added the size/XS Denotes a PR that changes 0-9 lines, ignoring generated files. label Nov 18, 2024
@tiprow tiprow
Copy link

tiprow bot commented Nov 18, 2024

Hi @xuanyu66. Thanks for your PR.

PRs from untrusted users cannot be marked as trusted with /ok-to-test in this repo meaning untrusted PR authors can never trigger tests themselves. Collaborators can still trigger tests on the PR using /test all.

I understand the commands that are listed here.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

@xuanyu66 xuanyu66 changed the title Skip apply ioworkers when seek(0, io.SeekCurrent) Lightning: Skip apply ioworkers when seek(0, io.SeekCurrent) Nov 18, 2024
@xuanyu66
Copy link
Contributor Author

/test all

@tiprow tiprow
Copy link

tiprow bot commented Nov 18, 2024

@xuanyu66: Cannot trigger testing until a trusted user reviews the PR and leaves an /ok-to-test message.

In response to this:

/test all

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

@codecov Codecov
Copy link

codecov bot commented Nov 18, 2024 •
edited
Loading

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 73.9544%. Comparing base (a22b842) to head (765bc76).
Report is 135 commits behind head on master.

Additional details and impacted files 
@@               Coverage Diff                @@
##             master     #57454        +/-   ##
================================================
+ Coverage   72.8488%   73.9544%   +1.1055%     
================================================
  Files          1672       1702        +30     
  Lines        462666     473747     +11081     
================================================
+ Hits         337047     350357     +13310     
+ Misses       104821     101937      -2884     
- Partials      20798      21453       +655
Flag Coverage Δ
integration 46.5596% <100.0000%> (?)
unit 72.5776% <100.0000%> (+0.3559%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Components Coverage Δ
dumpling 52.7673% <ø> (ø)
parser ∅ <ø> (∅)
br 45.2143% <ø> (+0.0917%) ⬆️

lance6716
lance6716 approved these changes Nov 18, 2024
View reviewed changes
Copy link
Contributor

@lance6716 lance6716 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

/cc @D3Hunter

@ti-chi-bot ti-chi-bot bot requested a review from D3Hunter November 18, 2024 11:53
@ti-chi-bot ti-chi-bot bot added needs-1-more-lgtm Indicates a PR needs 1 more LGTM. approved labels Nov 18, 2024
@ti-chi-bot ti-chi-bot
Copy link

ti-chi-bot bot commented Nov 19, 2024

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: D3Hunter, lance6716

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@ti-chi-bot ti-chi-bot bot added lgtm and removed needs-1-more-lgtm Indicates a PR needs 1 more LGTM. labels Nov 19, 2024
@ti-chi-bot ti-chi-bot
Copy link

ti-chi-bot bot commented Nov 19, 2024

[LGTM Timeline notifier]

Timeline:

  • 2024-11-18 11:53:45.337103529 +0000 UTC m=+875587.527972526: ☑️ agreed by lance6716.
  • 2024-11-19 02:40:53.540144307 +0000 UTC m=+928815.731013303: ☑️ agreed by D3Hunter.

@xuanyu66
Copy link
Contributor Author

/test all

@tiprow tiprow
Copy link

tiprow bot commented Nov 19, 2024

@xuanyu66: Cannot trigger testing until a trusted user reviews the PR and leaves an /ok-to-test message.

In response to this:

/test all

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

@xuanyu66
Copy link
Contributor Author

@lance6716 the merge is ready

@ti-chi-bot ti-chi-bot bot merged commit 0e3efb6 into pingcap:master Nov 19, 2024
25 checks passed
@ti-chi-bot ti-chi-bot bot added the needs-cherry-pick-release-7.5 Should cherry pick this PR to release-7.5 branch. label Dec 3, 2024
@ti-chi-bot
Copy link
Member

In response to a cherrypick label: new pull request created to branch release-7.5: #57930.

@ti-chi-bot ti-chi-bot mentioned this pull request Dec 3, 2024
Merged
13 tasks
ti-chi-bot bot pushed a commit that referenced this pull request Dec 4, 2024
@ti-chi-bot
Lightning: Skip apply ioworkers when seek(0, io.SeekCurrent) (#57454) (…
7723772 
…#57930)

close #57413
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@D3Hunter D3Hunter D3Hunter approved these changes

@lance6716 lance6716 lance6716 approved these changes

Assignees
No one assigned
Labels
approved lgtm needs-cherry-pick-release-7.5 Should cherry pick this PR to release-7.5 branch. release-note-none Denotes a PR that doesn't merit a release note. size/XS Denotes a PR that changes 0-9 lines, ignoring generated files.
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

Lightning: performance issue when importing from cloud storage
5 participants
@xuanyu66 @sre-bot @ti-chi-bot @lance6716 @D3Hunter