Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

planner: Display truncate vector in EXPLAIN #55934

Merged
merged 19 commits into from
Sep 29, 2024
Merged

planner: Display truncate vector in EXPLAIN #55934

merged 19 commits into from
Sep 29, 2024

Conversation

EricZequan
Copy link
Contributor

@EricZequan EricZequan commented Sep 9, 2024 •
edited
Loading

What problem does this PR solve?

Issue Number: ref #54245

Problem Summary:

What changed and how does it work?

Before this PR, when we used explain sql to view the execution plan, it might be affected by the long string or vector entered by the user, making the execution plan difficult to read. This PR brings long constant truncation to more kind of plans.

We added TruncatedStringify in pkg/types/datum.go to handle the explain problem of constant type, interpreting variables with length longer than 64 as xxx...len(x).

At the same time, the truncation of constant will affect the redact processing of some private data in the import into statement (data is lost after truncation). We processed the initParameters function in pkg/executor/importer/import.go so that it will not be affected by truncation.

Check List

Tests

  • Unit test
  • Integration test
  • Manual test (add detailed scripts or steps below)
  • No need to test
    • I checked and no code files have been changed.

Side effects

  • Performance regression: Consumes more CPU
  • Performance regression: Consumes more Memory
  • Breaking backward compatibility

Documentation

  • Affects user behaviors
  • Contains syntax changes
  • Contains variable changes
  • Contains experimental features
  • Changes MySQL compatibility

Release note

Please refer to Release Notes Language Style Guide to write a quality release note.

None

@ti-chi-bot ti-chi-bot bot added do-not-merge/invalid-title do-not-merge/needs-linked-issue do-not-merge/needs-tests-checked release-note-none Denotes a PR that doesn't merit a release note. size/L Denotes a PR that changes 100-499 lines, ignoring generated files. labels Sep 9, 2024
@tiprow tiprow
Copy link

tiprow bot commented Sep 9, 2024

Hi @EricZequan. Thanks for your PR.

PRs from untrusted users cannot be marked as trusted with /ok-to-test in this repo meaning untrusted PR authors can never trigger tests themselves. Collaborators can still trigger tests on the PR using /test all.

I understand the commands that are listed here.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

@codecov Codecov
Copy link

codecov bot commented Sep 9, 2024 •
edited
Loading

Codecov Report

Attention: Patch coverage is 95.16129% with 3 lines in your changes missing coverage. Please review.

Project coverage is 75.3764%. Comparing base (4df3389) to head (6a70a74).
Report is 1 commits behind head on master.

Additional details and impacted files 
@@               Coverage Diff                @@
##             master     #55934        +/-   ##
================================================
+ Coverage   73.3871%   75.3764%   +1.9893%     
================================================
  Files          1623       1623                
  Lines        447156     447243        +87     
================================================
+ Hits         328155     337116      +8961     
+ Misses        98886      89617      -9269     
- Partials      20115      20510       +395
Flag Coverage Δ
integration 48.8889% <46.7741%> (?)
unit 72.4819% <95.1612%> (+0.0015%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Components Coverage Δ
dumpling 52.9478% <ø> (ø)
parser ∅ <ø> (∅)
br 52.2974% <ø> (+6.7666%) ⬆️

@EricZequan
Copy link
Contributor Author

/retest

@tiprow tiprow
Copy link

tiprow bot commented Sep 9, 2024

@EricZequan: Cannot trigger testing until a trusted user reviews the PR and leaves an /ok-to-test message.

In response to this:

/retest

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

@EricZequan
Copy link
Contributor Author

/retest

@tiprow tiprow
Copy link

tiprow bot commented Sep 9, 2024

@EricZequan: Cannot trigger testing until a trusted user reviews the PR and leaves an /ok-to-test message.

In response to this:

/retest

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

@EricZequan EricZequan changed the title Truncate Long vecs planner: Fix more cases that explain does not display a truncated value Sep 9, 2024
@EricZequan
Copy link
Contributor Author

/cc @breezewish PTAL~

@EricZequan
Copy link
Contributor Author

/retest

@tiprow tiprow
Copy link

tiprow bot commented Sep 10, 2024

@EricZequan: Cannot trigger testing until a trusted user reviews the PR and leaves an /ok-to-test message.

In response to this:

/retest

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

@EricZequan
Copy link
Contributor Author

/retest

@tiprow tiprow
Copy link

tiprow bot commented Sep 13, 2024

@EricZequan: Cannot trigger testing until a trusted user reviews the PR and leaves an /ok-to-test message.

In response to this:

/retest

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

EricZequan
EricZequan commented Sep 20, 2024
View reviewed changes
pkg/executor/importer/import.go Outdated Show resolved Hide resolved
pkg/executor/importer/import.go Outdated Show resolved Hide resolved
@EricZequan
Copy link
Contributor Author

/retest

@tiprow tiprow
Copy link

tiprow bot commented Sep 23, 2024

@EricZequan: Cannot trigger testing until a trusted user reviews the PR and leaves an /ok-to-test message.

In response to this:

/retest

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

@EricZequan
Copy link
Contributor Author

/retest

@tiprow tiprow
Copy link

tiprow bot commented Sep 23, 2024

@EricZequan: Cannot trigger testing until a trusted user reviews the PR and leaves an /ok-to-test message.

In response to this:

/retest

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

@EricZequan
Copy link
Contributor Author

/retest

@tiprow tiprow
Copy link

tiprow bot commented Sep 23, 2024

@EricZequan: Cannot trigger testing until a trusted user reviews the PR and leaves an /ok-to-test message.

In response to this:

/retest

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

@EricZequan
Copy link
Contributor Author

@breezewish The option attached to the import statement here are all parameters entered by the user. Weizhen has confirmed that the parameters entered by the user here will be interpreted as constant. The code also processes the option option of the import statement, so we can directly convert it to constant for processing here. Other complex input options such as 1+2 will be regarded as illegal input and will not enter our code. At the same time, the option option does not include vector, so the vector judgment here can also be deleted. We also do not need to do some redact or quote processing on the result of explain here, so it can be directly returned through GetVaule. Please review the current plan~

@EricZequan
Copy link
Contributor Author

/retest

@tiprow tiprow
Copy link

tiprow bot commented Sep 29, 2024

@EricZequan: Cannot trigger testing until a trusted user reviews the PR and leaves an /ok-to-test message.

In response to this:

/retest

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

@EricZequan
fix bazel
4bd63bc 
Signed-off-by: “EricZequan” <[email protected]>
@ti-chi-bot ti-chi-bot bot added lgtm and removed needs-1-more-lgtm Indicates a PR needs 1 more LGTM. labels Sep 29, 2024
@ti-chi-bot ti-chi-bot
Copy link

ti-chi-bot bot commented Sep 29, 2024

[LGTM Timeline notifier]

Timeline:

  • 2024-09-25 08:22:28.249672136 +0000 UTC m=+1640617.990096075: ☑️ agreed by hawkingrei.
  • 2024-09-29 08:06:28.639980614 +0000 UTC m=+169344.060193627: ☑️ agreed by wjhuang2016.

JaySon-Huang
JaySon-Huang approved these changes Sep 29, 2024
View reviewed changes
Copy link
Contributor

@JaySon-Huang JaySon-Huang left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm

@ti-chi-bot ti-chi-bot
Copy link

ti-chi-bot bot commented Sep 29, 2024

@JaySon-Huang: adding LGTM is restricted to approvers and reviewers in OWNERS files.

In response to this:

lgtm

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

windtalker
windtalker approved these changes Sep 29, 2024
View reviewed changes
Copy link
Contributor

@windtalker windtalker left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm

GMHDBJD
GMHDBJD approved these changes Sep 29, 2024
View reviewed changes
Copy link
Contributor

@GMHDBJD GMHDBJD left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@ti-chi-bot ti-chi-bot bot added the approved label Sep 29, 2024
@ti-chi-bot ti-chi-bot bot added sig/planner SIG: Planner size/XXL Denotes a PR that changes 1000+ lines, ignoring generated files. and removed size/L Denotes a PR that changes 100-499 lines, ignoring generated files. approved labels Sep 29, 2024
@ti-chi-bot ti-chi-bot
Copy link

ti-chi-bot bot commented Sep 29, 2024

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: GMHDBJD, hawkingrei, JaySon-Huang, windtalker, wjhuang2016

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@ti-chi-bot ti-chi-bot bot added approved size/L Denotes a PR that changes 100-499 lines, ignoring generated files. and removed size/XXL Denotes a PR that changes 1000+ lines, ignoring generated files. labels Sep 29, 2024
@breezewish breezewish removed the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Sep 29, 2024
@breezewish
Copy link
Member

/merge

@ti-chi-bot ti-chi-bot
Copy link

ti-chi-bot bot commented Sep 29, 2024

@breezewish: We have migrated to builtin LGTM and approve plugins for reviewing.

👉 Please use /approve when you want approve this pull request.

The changes announcement: LGTM plugin changes

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the ti-community-infra/tichi repository.

@ti-chi-bot ti-chi-bot bot merged commit 65d740f into pingcap:master Sep 29, 2024
21 checks passed
This was referenced Oct 7, 2024
Closed
Closed
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@breezewish breezewish breezewish left review comments

@GMHDBJD GMHDBJD GMHDBJD approved these changes

@wjhuang2016 wjhuang2016 wjhuang2016 approved these changes

@JaySon-Huang JaySon-Huang JaySon-Huang approved these changes

@hawkingrei hawkingrei hawkingrei approved these changes

@windtalker windtalker windtalker approved these changes

Assignees
No one assigned
Labels
approved lgtm release-note-none Denotes a PR that doesn't merit a release note. sig/planner SIG: Planner size/L Denotes a PR that changes 100-499 lines, ignoring generated files.
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
7 participants
@EricZequan @breezewish @windtalker @hawkingrei @JaySon-Huang @wjhuang2016 @GMHDBJD