Skip to content

[BugFix] follow the output type after eliminating agg (backport #52741)#53486

Merged
wanpengfei-git merged 4 commits intobranch-3.3from
mergify/bp/branch-3.3/pr-52741
Dec 11, 2024
Merged

[BugFix] follow the output type after eliminating agg (backport #52741)#53486
wanpengfei-git merged 4 commits intobranch-3.3from
mergify/bp/branch-3.3/pr-52741

Conversation

@mergify
Copy link
Contributor

@mergify mergify bot commented Dec 3, 2024 •
edited by wanpengfei-git
Loading

Why I'm doing:

When eliminating aggregation by a unique key, the aggregation node is rewritten to a project node. The output of COUNT aggregation function is BIGINT, but it is rewritten to TINYINT.

Refresh materialized view lineitem_agg_mv1 failed after retrying 1 times(try-lock 0 times), error-msg : com.starrocks.common.UserException: type of exprs is not match slot's, expr_type=1, slot_type=7, slot_name=count_qty backend [id=10001] [host=172.26.95.192]
	at com.starrocks.qe.DefaultCoordinator.handleErrorExecution(DefaultCoordinator.java:719)
	at com.starrocks.qe.scheduler.Deployer.waitForDeploymentCompletion(Deployer.java:251)
	at com.starrocks.qe.scheduler.Deployer.deployFragments(Deployer.java:121)
	at com.starrocks.qe.scheduler.dag.AllAtOnceExecutionSchedule.schedule(AllAtOnceExecutionSchedule.java:45)
	at com.starrocks.qe.DefaultCoordinator.deliverExecFragments(DefaultCoordinator.java:645)
	at com.starrocks.qe.DefaultCoordinator.startScheduling(DefaultCoordinator.java:543)
	at com.starrocks.qe.scheduler.Coordinator.startScheduling(Coordinator.java:115)
	at com.starrocks.qe.scheduler.Coordinator.exec(Coordinator.java:94)
	at com.starrocks.qe.StmtExecutor.handleDMLStmt(StmtExecutor.java:2300)
 [wrapped] com.starrocks.common.UserException: type of exprs is not match slot's, expr_type=1, slot_type=7, slot_name=count_qty backend [id=10001] [host=172.26.95.192]
	at com.starrocks.qe.StmtExecutor.handleDMLStmt(StmtExecutor.java:2571)
	at com.starrocks.load.InsertOverwriteJobRunner.executeInsert(InsertOverwriteJobRunner.java:356)
	at com.starrocks.load.InsertOverwriteJobRunner.doLoad(InsertOverwriteJobRunner.java:170)
	at com.starrocks.load.InsertOverwriteJobRunner.handle(InsertOverwriteJobRunner.java:150)
	at com.starrocks.load.InsertOverwriteJobRunner.transferTo(InsertOverwriteJobRunner.java:211)
	at com.starrocks.load.InsertOverwriteJobRunner.prepare(InsertOverwriteJobRunner.java:255)
	at com.starrocks.load.InsertOverwriteJobRunner.handle(InsertOverwriteJobRunner.java:147)
	at com.starrocks.load.InsertOverwriteJobRunner.run(InsertOverwriteJobRunner.java:135)
	at com.starrocks.load.InsertOverwriteJobMgr.executeJob(InsertOverwriteJobMgr.java:91)
	at com.starrocks.qe.StmtExecutor.handleInsertOverwrite(StmtExecutor.java:2100)
	at com.starrocks.qe.StmtExecutor.handleDMLStmt(StmtExecutor.java:2195)
	at com.starrocks.qe.StmtExecutor.handleDMLStmtWithProfile(StmtExecutor.java:2109)
	at com.starrocks.scheduler.PartitionBasedMvRefreshProcessor.refreshMaterializedView(PartitionBasedMvRefreshProcessor.java:1159)
	at com.starrocks.scheduler.PartitionBasedMvRefreshProcessor.doRefreshMaterializedView(PartitionBasedMvRefreshProcessor.java:467)
	at com.starrocks.scheduler.PartitionBasedMvRefreshProcessor.doRefreshMaterializedViewWithRetry(PartitionBasedMvRefreshProcessor.java:373)
	at com.starrocks.scheduler.PartitionBasedMvRefreshProcessor.doMvRefresh(PartitionBasedMvRefreshProcessor.java:332)
	at com.starrocks.scheduler.PartitionBasedMvRefreshProcessor.processTaskRun(PartitionBasedMvRefreshProcessor.java:204)
	at com.starrocks.scheduler.TaskRun.executeTaskRun(TaskRun.java:270)
	at com.starrocks.scheduler.TaskRunExecutor.lambda$executeTaskRun$0(TaskRunExecutor.java:58)
	at java.base/java.util.concurrent.CompletableFuture$AsyncSupply.run(CompletableFuture.java:1700)
	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
	at java.base/java.lang.Thread.run(Thread.java:829)

What I'm doing:

Follow the output type of the aggregation after eliminating agg.

Fixes #issue

What type of PR is this:

  • BugFix
  • Feature
  • Enhancement
  • Refactor
  • UT
  • Doc
  • Tool

Does this PR entail a change in behavior?

  • Yes, this PR will result in a change in behavior.
  • No, this PR will not result in a change in behavior.

If yes, please specify the type of change:

  • Interface/UI changes: syntax, type conversion, expression evaluation, display information
  • Parameter changes: default values, similar parameters but with different default values
  • Policy changes: use new policy to replace old one, functionality automatically enabled
  • Feature removed
  • Miscellaneous: upgrade & downgrade compatibility, etc.

Checklist:

  • I have added test cases for my bug fix or my new feature
  • This pr needs user documentation (for new or modified features or behaviors)
    • I have added documentation for my new feature or new function
  • This is a backport pr

Bugfix cherry-pick branch check:

  • I have checked the version labels which the pr will be auto-backported to the target branch
    • 3.3
    • 3.2
    • 3.1
    • 3.0
    • 2.5
This is an automatic backport of pull request #52741 done by [Mergify](https://mergify.com). ## Why I'm doing:

When eliminating aggregation by a unique key, the aggregation node is rewritten to a project node. The output of COUNT aggregation function is BIGINT, but it is rewritten to TINYINT.

Refresh materialized view lineitem_agg_mv1 failed after retrying 1 times(try-lock 0 times), error-msg : com.starrocks.common.UserException: type of exprs is not match slot's, expr_type=1, slot_type=7, slot_name=count_qty backend [id=10001] [host=172.26.95.192]
	at com.starrocks.qe.DefaultCoordinator.handleErrorExecution(DefaultCoordinator.java:719)
	at com.starrocks.qe.scheduler.Deployer.waitForDeploymentCompletion(Deployer.java:251)
	at com.starrocks.qe.scheduler.Deployer.deployFragments(Deployer.java:121)
	at com.starrocks.qe.scheduler.dag.AllAtOnceExecutionSchedule.schedule(AllAtOnceExecutionSchedule.java:45)
	at com.starrocks.qe.DefaultCoordinator.deliverExecFragments(DefaultCoordinator.java:645)
	at com.starrocks.qe.DefaultCoordinator.startScheduling(DefaultCoordinator.java:543)
	at com.starrocks.qe.scheduler.Coordinator.startScheduling(Coordinator.java:115)
	at com.starrocks.qe.scheduler.Coordinator.exec(Coordinator.java:94)
	at com.starrocks.qe.StmtExecutor.handleDMLStmt(StmtExecutor.java:2300)
 [wrapped] com.starrocks.common.UserException: type of exprs is not match slot's, expr_type=1, slot_type=7, slot_name=count_qty backend [id=10001] [host=172.26.95.192]
	at com.starrocks.qe.StmtExecutor.handleDMLStmt(StmtExecutor.java:2571)
	at com.starrocks.load.InsertOverwriteJobRunner.executeInsert(InsertOverwriteJobRunner.java:356)
	at com.starrocks.load.InsertOverwriteJobRunner.doLoad(InsertOverwriteJobRunner.java:170)
	at com.starrocks.load.InsertOverwriteJobRunner.handle(InsertOverwriteJobRunner.java:150)
	at com.starrocks.load.InsertOverwriteJobRunner.transferTo(InsertOverwriteJobRunner.java:211)
	at com.starrocks.load.InsertOverwriteJobRunner.prepare(InsertOverwriteJobRunner.java:255)
	at com.starrocks.load.InsertOverwriteJobRunner.handle(InsertOverwriteJobRunner.java:147)
	at com.starrocks.load.InsertOverwriteJobRunner.run(InsertOverwriteJobRunner.java:135)
	at com.starrocks.load.InsertOverwriteJobMgr.executeJob(InsertOverwriteJobMgr.java:91)
	at com.starrocks.qe.StmtExecutor.handleInsertOverwrite(StmtExecutor.java:2100)
	at com.starrocks.qe.StmtExecutor.handleDMLStmt(StmtExecutor.java:2195)
	at com.starrocks.qe.StmtExecutor.handleDMLStmtWithProfile(StmtExecutor.java:2109)
	at com.starrocks.scheduler.PartitionBasedMvRefreshProcessor.refreshMaterializedView(PartitionBasedMvRefreshProcessor.java:1159)
	at com.starrocks.scheduler.PartitionBasedMvRefreshProcessor.doRefreshMaterializedView(PartitionBasedMvRefreshProcessor.java:467)
	at com.starrocks.scheduler.PartitionBasedMvRefreshProcessor.doRefreshMaterializedViewWithRetry(PartitionBasedMvRefreshProcessor.java:373)
	at com.starrocks.scheduler.PartitionBasedMvRefreshProcessor.doMvRefresh(PartitionBasedMvRefreshProcessor.java:332)
	at com.starrocks.scheduler.PartitionBasedMvRefreshProcessor.processTaskRun(PartitionBasedMvRefreshProcessor.java:204)
	at com.starrocks.scheduler.TaskRun.executeTaskRun(TaskRun.java:270)
	at com.starrocks.scheduler.TaskRunExecutor.lambda$executeTaskRun$0(TaskRunExecutor.java:58)
	at java.base/java.util.concurrent.CompletableFuture$AsyncSupply.run(CompletableFuture.java:1700)
	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
	at java.base/java.lang.Thread.run(Thread.java:829)

What I'm doing:

Follow the output type of the aggregation after eliminating agg.

Fixes #issue

What type of PR is this:

  • BugFix
  • Feature
  • Enhancement
  • Refactor
  • UT
  • Doc
  • Tool

Does this PR entail a change in behavior?

  • Yes, this PR will result in a change in behavior.
  • No, this PR will not result in a change in behavior.

If yes, please specify the type of change:

  • Interface/UI changes: syntax, type conversion, expression evaluation, display information
  • Parameter changes: default values, similar parameters but with different default values
  • Policy changes: use new policy to replace old one, functionality automatically enabled
  • Feature removed
  • Miscellaneous: upgrade & downgrade compatibility, etc.

Checklist:

  • I have added test cases for my bug fix or my new feature
  • This pr needs user documentation (for new or modified features or behaviors)
    • I have added documentation for my new feature or new function
  • This is a backport pr

@mergify mergify bot added the conflicts label Dec 3, 2024
@mergify
Copy link
Contributor Author

mergify bot commented Dec 3, 2024

Cherry-pick of b094453 has failed:

On branch mergify/bp/branch-3.3/pr-52741
Your branch is up to date with 'origin/branch-3.3'.

You are currently cherry-picking commit b094453a90.
  (fix conflicts and run "git cherry-pick --continue")
  (use "git cherry-pick --skip" to skip this patch)
  (use "git cherry-pick --abort" to cancel the cherry-pick operation)

Unmerged paths:
  (use "git add/rm <file>..." as appropriate to mark resolution)
	both modified:   fe/fe-core/src/main/java/com/starrocks/sql/optimizer/rule/transformation/EliminateAggRule.java
	deleted by us:   fe/fe-core/src/test/java/com/starrocks/sql/plan/AggregateWithUKFKTest.java

no changes added to commit (use "git add" and/or "git commit -a")

To fix up this pull request, you can check it out locally. See documentation: https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/reviewing-changes-in-pull-requests/checking-out-pull-requests-locally

@mergify mergify bot mentioned this pull request Dec 3, 2024
Merged
24 tasks
@mergify
Copy link
Contributor Author

mergify bot commented Dec 3, 2024

@mergify[bot]: Backport conflict, please reslove the conflict and resubmit the pr

@mergify mergify bot closed this Dec 3, 2024
@Seaven Seaven reopened this Dec 3, 2024
@wanpengfei-git wanpengfei-git enabled auto-merge (squash) December 3, 2024 09:10
@Seaven Seaven force-pushed the mergify/bp/branch-3.3/pr-52741 branch from d43ad03 to 3eb1e99 Compare December 3, 2024 09:12
ZiheLiu
ZiheLiu previously approved these changes Dec 10, 2024
View reviewed changes
@ZiheLiu
[BugFix] follow the output type after eliminating agg (#52741)
8cbd0b6 
Signed-off-by: zihe.liu <[email protected]>
(cherry picked from commit b094453)
@ZiheLiu
fix ut
f2602c0 
Signed-off-by: zihe.liu <[email protected]>
@ZiheLiu
wip
43efbb5 
Signed-off-by: zihe.liu <[email protected]>
@ZiheLiu
ignore ustable ut
4ec3b98 
Signed-off-by: zihe.liu <[email protected]>
@ZiheLiu ZiheLiu force-pushed the mergify/bp/branch-3.3/pr-52741 branch from 78250a3 to 4ec3b98 Compare December 11, 2024 06:12
@sonarqubecloud
Copy link

sonarqubecloud bot commented Dec 11, 2024

Quality Gate Passed Quality Gate passed

Issues
0 New issues
0 Accepted issues

Measures
0 Security Hotspots
0.0% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarQube Cloud

@wanpengfei-git wanpengfei-git merged commit 16bb564 into branch-3.3 Dec 11, 2024
@wanpengfei-git wanpengfei-git deleted the mergify/bp/branch-3.3/pr-52741 branch December 11, 2024 06:57
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@ZiheLiu ZiheLiu ZiheLiu approved these changes

Assignees

@ZiheLiu ZiheLiu

Labels

automerge conflicts version:3.3.8

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@ZiheLiu @Seaven @wanpengfei-git