Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

storage: fix block rows not match when filter column is the first non-empty column in the block #9483

Merged
merged 9 commits into from
Sep 29, 2024
Merged

storage: fix block rows not match when filter column is the first non-empty column in the block #9483

merged 9 commits into from
Sep 29, 2024

Conversation

Lloyd-Pottiger
Copy link
Contributor

@Lloyd-Pottiger Lloyd-Pottiger commented Sep 27, 2024 •
edited
Loading

What problem does this PR solve?

Issue Number: ref #9472

Problem Summary:

What is changed and how it works?

Try to fix #9472, failed to reproduce the issue. But when the filter column is the first non-empty column in the block, block.rows() = filter_column->size(), but filter_column is not filtered, so the rows may not match.

But filter_column is usually added to the back of the block, so it is kind of weird.

storage: fix block rows not match when the filter column is the first non-empty column in the block

Check List

Tests

  • Unit test
  • Integration test
  • Manual test (add detailed scripts or steps below)
  • No code

Side effects

  • Performance regression: Consumes more CPU
  • Performance regression: Consumes more Memory
  • Breaking backward compatibility

Documentation

  • Affects user behaviors
  • Contains syntax changes
  • Contains variable changes
  • Contains experimental features
  • Changes MySQL compatibility

Release note

None

@Lloyd-Pottiger
storage: fix block rows not match when filter column is the first non…
bff1388 
…-empty column in the block

Signed-off-by: Lloyd-Pottiger <[email protected]>
@ti-chi-bot ti-chi-bot bot added release-note-none Denotes a PR that doesn't merit a release note. do-not-merge/needs-triage-completed size/M Denotes a PR that changes 30-99 lines, ignoring generated files. and removed do-not-merge/needs-triage-completed labels Sep 27, 2024
@Lloyd-Pottiger
fix
625ed50 
Signed-off-by: Lloyd-Pottiger <[email protected]>
@ti-chi-bot ti-chi-bot bot added needs-cherry-pick-release-7.5 Should cherry pick this PR to release-7.5 branch. needs-cherry-pick-release-8.1 Should cherry pick this PR to release-8.1 branch. needs-cherry-pick-release-7.1 Should cherry pick this PR to release-7.1 branch. labels Sep 27, 2024
@Lloyd-Pottiger
fix
bbb8190 
Signed-off-by: Lloyd-Pottiger <[email protected]>
@Lloyd-Pottiger
check rows
370bc1c 
Signed-off-by: Lloyd-Pottiger <[email protected]>
JinheLin
JinheLin reviewed Sep 29, 2024
View reviewed changes
dbms/src/Core/Block.cpp Outdated Show resolved Hide resolved
@Lloyd-Pottiger
check rows
233a1fe 
Signed-off-by: Lloyd-Pottiger <[email protected]>
JinheLin
JinheLin reviewed Sep 29, 2024
View reviewed changes
dbms/src/Storages/DeltaMerge/LateMaterializationBlockInputStream.cpp
@@ -71,8 +71,6 @@ Block LateMaterializationBlockInputStream::read()
}
for (auto & col : filter_column_block)
{
if (col.name == filter_column_name)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why not try continue filter_column_name like the code below?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

col_filter is not a ColumnRef in filter_column_block, filter_column in filter_column_block can call filter here.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In filterFilterColumnBlock, if header.has(filter_column_name) is false, filter_column will not be filtered.

I mean why not skip filtering filter_column here when header.has(filter_column_name) is false like function filterFilterColumnBlock does?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed

@ti-chi-bot ti-chi-bot bot added needs-1-more-lgtm Indicates a PR needs 1 more LGTM. approved labels Sep 29, 2024
JaySon-Huang
JaySon-Huang approved these changes Sep 29, 2024
View reviewed changes
Copy link
Contributor

@JaySon-Huang JaySon-Huang left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@ti-chi-bot ti-chi-bot
Copy link
Contributor

ti-chi-bot bot commented Sep 29, 2024

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: JaySon-Huang, JinheLin

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:
  • OWNERS [JaySon-Huang,JinheLin]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@ti-chi-bot ti-chi-bot bot added lgtm and removed needs-1-more-lgtm Indicates a PR needs 1 more LGTM. labels Sep 29, 2024
@ti-chi-bot ti-chi-bot
Copy link
Contributor

ti-chi-bot bot commented Sep 29, 2024

[LGTM Timeline notifier]

Timeline:

  • 2024-09-29 08:26:43.202642064 +0000 UTC m=+170558.622855075: ☑️ agreed by JinheLin.
  • 2024-09-29 08:36:15.501271501 +0000 UTC m=+171130.921484505: ☑️ agreed by JaySon-Huang.

@ti-chi-bot ti-chi-bot bot merged commit 69dd613 into pingcap:master Sep 29, 2024
5 checks passed
ti-chi-bot pushed a commit to ti-chi-bot/tiflash that referenced this pull request Sep 29, 2024
@Lloyd-Pottiger @ti-chi-bot
This is an automated cherry-pick of pingcap#9483
bb3e3bf 
Signed-off-by: ti-chi-bot <[email protected]>
@ti-chi-bot
Copy link
Member

In response to a cherrypick label: new pull request created to branch release-7.1: #9493.

@ti-chi-bot ti-chi-bot mentioned this pull request Sep 29, 2024
Merged
12 tasks
ti-chi-bot pushed a commit to ti-chi-bot/tiflash that referenced this pull request Sep 29, 2024
@Lloyd-Pottiger @ti-chi-bot
This is an automated cherry-pick of pingcap#9483
3b18c5c 
Signed-off-by: ti-chi-bot <[email protected]>
@ti-chi-bot
Copy link
Member

In response to a cherrypick label: new pull request created to branch release-7.5: #9494.

@ti-chi-bot ti-chi-bot mentioned this pull request Sep 29, 2024
Closed
12 tasks
ti-chi-bot pushed a commit to ti-chi-bot/tiflash that referenced this pull request Sep 29, 2024
@Lloyd-Pottiger @ti-chi-bot
This is an automated cherry-pick of pingcap#9483
66df47b 
Signed-off-by: ti-chi-bot <[email protected]>
@ti-chi-bot
Copy link
Member

In response to a cherrypick label: new pull request created to branch release-8.1: #9495.

@ti-chi-bot ti-chi-bot mentioned this pull request Sep 29, 2024
Merged
12 tasks
ti-chi-bot bot pushed a commit that referenced this pull request Sep 30, 2024
@ti-chi-bot @Lloyd-Pottiger
storage: fix block rows not match when filter column is the first non…
8c8c97b 
…-empty column in the block (#9483) (#9493)

ref #9472

storage: fix block rows not match when the filter column is the first non-empty column in the block

Signed-off-by: ti-chi-bot <[email protected]>
Signed-off-by: Lloyd-Pottiger <[email protected]>

Co-authored-by: Lloyd-Pottiger <[email protected]>
Co-authored-by: Lloyd-Pottiger <[email protected]>
@Lloyd-Pottiger Lloyd-Pottiger deleted the try-fix branch September 30, 2024 12:34
ti-chi-bot bot pushed a commit that referenced this pull request Oct 8, 2024
@ti-chi-bot @Lloyd-Pottiger
storage: fix block rows not match when filter column is the first non…
5915c6b 
…-empty column in the block (#9483) (#9495)

ref #9472

storage: fix block rows not match when the filter column is the first non-empty column in the block

Signed-off-by: ti-chi-bot <[email protected]>
Signed-off-by: Lloyd-Pottiger <[email protected]>

Co-authored-by: Lloyd-Pottiger <[email protected]>
Co-authored-by: Lloyd-Pottiger <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@JinheLin JinheLin JinheLin approved these changes

@JaySon-Huang JaySon-Huang JaySon-Huang approved these changes

Assignees
No one assigned
Labels
approved lgtm needs-cherry-pick-release-7.1 Should cherry pick this PR to release-7.1 branch. needs-cherry-pick-release-7.5 Should cherry pick this PR to release-7.5 branch. needs-cherry-pick-release-8.1 Should cherry pick this PR to release-8.1 branch. release-note-none Denotes a PR that doesn't merit a release note. size/M Denotes a PR that changes 30-99 lines, ignoring generated files.
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

query count(*) failed
4 participants
@Lloyd-Pottiger @ti-chi-bot @JaySon-Huang @JinheLin