[CALCITE-7311] Support the syntax ROW(*) to create a nested ROW type with all columns #4665

xiedeyantu · 2025-12-04T14:58:31Z

mihaibudiu · 2025-12-04T22:17:27Z

core/src/main/java/org/apache/calcite/sql/validate/SqlValidatorImpl.java

    }
  }

+  private List<SqlNode> expandRowStar(SqlIdentifier identifier, SelectScope scope) {


Once we merge #4662 this should probably be enhanced to support EXCLUDE list in ROW(*) as well.

Perhaps we can consider supporting this in the next PR, because there is currently a limitation: the EXCLUDE syntax is only enabled in Babel. Should we also keep ROW(* EXCLUDE(...)) supported in Babel?

They both probably need to be in the same parser

Doing it in a subsequent PR is fine.

They both probably need to be in the same parser

I want to confirm, are you suggesting that the ROW(*) capability should also be fully supported in Babel? I’ve read through all the comments, and it seems this is the only remaining issue to address, right?

it is fine to support ROW(*) in the main parser and ROW(* EXCLUDE) only in Babel

mihaibudiu · 2025-12-04T22:19:43Z

core/src/main/java/org/apache/calcite/sql/validate/SqlValidatorImpl.java

          new CallCopyingArgHandler(call, false);
      call.getOperator().acceptCall(this, call, true, argHandler);
-      final SqlNode result = argHandler.result();
+      final SqlNode result = expandRow(argHandler.result());


I find is a bit weird to have expandRow happen as a side-effect of calculatePermuteOffset; maybe this is a sign that the function name should be updated.

I have rename expandRow to expandStarInRow.

mihaibudiu · 2025-12-04T22:21:08Z

core/src/main/java/org/apache/calcite/sql/validate/SqlValidatorImpl.java

+        return node;
+      }
+      final SqlValidatorScope scope = getScope();
+      if (!(scope instanceof SelectScope)) {


what happens if the scope is not Select? Will eventually an error be produced?
Or is this the right place to produce the error?

I added error here.

mihaibudiu · 2025-12-04T22:22:55Z

core/src/main/codegen/templates/Parser.jj

+    }
+
+    private void popRowValueStar() {
+        if (rowValueStarCount > 0) {


Can this be false?
If not, maybe this can be an assert.

mihaibudiu · 2025-12-04T22:24:54Z

core/src/main/codegen/templates/Parser.jj

    LOOKAHEAD(3)
    <ROW> {
        s = span();
+        pushRowValueStar();


good point about nested ROW constructors.
I wonder what spark does.
It seems that ROW(*, ROW(*)) is legit

Pgsql can support this syntax, following:

SELECT ROW(DEPT.*, ROW(DEPT.*)) FROM DEPT; row ----------------------------------------------------------- (10,ACCOUNTING,"NEW YORK","(10,ACCOUNTING,""NEW YORK"")") (20,RESEARCH,DALLAS,"(20,RESEARCH,DALLAS)") (30,SALES,CHICAGO,"(30,SALES,CHICAGO)") (40,OPERATIONS,BOSTON,"(40,OPERATIONS,BOSTON)") (4 rows)

It doesn't support ROW(*, ROW(*)). But I think we can support this form.

I configured a Spark environment and tested the following SQL statements, all of which were executable.

spark-sql (mydb)> SELECT STRUCT(*) AS row_data FROM dept; {"deptno":10,"dname":"ACCOUNTING","loc":"NEW YORK"} {"deptno":40,"dname":"OPERATIONS","loc":"BOSTON"} {"deptno":20,"dname":"RESEARCH","loc":"DALLAS"} {"deptno":30,"dname":"SALES","loc":"CHICAGO"} Time taken: 0.207 seconds, Fetched 4 row(s) spark-sql (mydb)> SELECT STRUCT(*, STRUCT(*)) AS row_data FROM dept; {"deptno":10,"dname":"ACCOUNTING","loc":"NEW YORK","col2":{"deptno":10,"dname":"ACCOUNTING","loc":"NEW YORK"}} {"deptno":40,"dname":"OPERATIONS","loc":"BOSTON","col2":{"deptno":40,"dname":"OPERATIONS","loc":"BOSTON"}} {"deptno":20,"dname":"RESEARCH","loc":"DALLAS","col2":{"deptno":20,"dname":"RESEARCH","loc":"DALLAS"}} {"deptno":30,"dname":"SALES","loc":"CHICAGO","col2":{"deptno":30,"dname":"SALES","loc":"CHICAGO"}} Time taken: 0.103 seconds, Fetched 4 row(s) spark-sql (mydb)> SELECT STRUCT(*, STRUCT(dept.*)) AS row_data FROM dept; {"deptno":10,"dname":"ACCOUNTING","loc":"NEW YORK","col2":{"deptno":10,"dname":"ACCOUNTING","loc":"NEW YORK"}} {"deptno":40,"dname":"OPERATIONS","loc":"BOSTON","col2":{"deptno":40,"dname":"OPERATIONS","loc":"BOSTON"}} {"deptno":20,"dname":"RESEARCH","loc":"DALLAS","col2":{"deptno":20,"dname":"RESEARCH","loc":"DALLAS"}} {"deptno":30,"dname":"SALES","loc":"CHICAGO","col2":{"deptno":30,"dname":"SALES","loc":"CHICAGO"}} Time taken: 0.075 seconds, Fetched 4 row(s)

mihaibudiu · 2025-12-04T22:26:02Z

core/src/main/codegen/templates/Parser.jj

        e = ContextVariable()
    |
        e = CompoundIdentifier()
+    |


The natural question that follows is about ROW(T.*). But that can be part of a follow-up PR, including perhaps ROW(T.* EXCLUDE(list))

mihaibudiu · 2025-12-04T22:26:52Z

core/src/main/codegen/templates/Parser.jj

    |
        e = CompoundIdentifier()
+    |
+        LOOKAHEAD({ allowRowValueStar() })


didn't know that LOOKAHEAD allows a function call

xiedeyantu · 2025-12-04T22:39:55Z

This PR was assisted by Copilot. While it basically meets the requirements, the implementation is not optimal and conflicts with #4662. Thank you very much for your review. I will carefully consider these issues and continue with this PR after #4662 is completed.

xiedeyantu · 2025-12-09T01:49:38Z

Now that #4662 is merged, I plan to restart this work within the next few days.

…with all columns

sonarqubecloud · 2025-12-18T10:28:42Z

Quality Gate passed

Issues
2 New issues
0 Accepted issues

Measures
0 Security Hotspots
87.9% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarQube Cloud

xiedeyantu marked this pull request as draft December 4, 2025 14:58

mihaibudiu reviewed Dec 4, 2025

View reviewed changes

xiedeyantu force-pushed the CALCITE-7311 branch from 0009193 to 90c3ad0 Compare December 17, 2025 13:41

xiedeyantu marked this pull request as ready for review December 17, 2025 14:02

mihaibudiu approved these changes Dec 17, 2025

View reviewed changes

xiedeyantu added the LGTM-will-merge-soon Overall PR looks OK. Only minor things left. label Dec 18, 2025

[CALCITE-7311] Support the syntax ROW(*) to create a nested ROW type …

b6ba7a8

…with all columns

xiedeyantu force-pushed the CALCITE-7311 branch from 90c3ad0 to b6ba7a8 Compare December 18, 2025 08:47

xiedeyantu merged commit 1a4b00b into apache:main Dec 19, 2025
20 checks passed

[CALCITE-7311] Support the syntax ROW(*) to create a nested ROW type with all columns #4665

[CALCITE-7311] Support the syntax ROW(*) to create a nested ROW type with all columns #4665

Uh oh!

Conversation

xiedeyantu commented Dec 4, 2025

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

mihaibudiu Dec 17, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

xiedeyantu commented Dec 4, 2025

Uh oh!

xiedeyantu commented Dec 9, 2025

Uh oh!

sonarqubecloud bot commented Dec 18, 2025

Quality Gate passed

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

mihaibudiu Dec 17, 2025 •

edited

Loading