Skip to content

Pipe: account page decode memory in scan parser#17807

Merged
jt2594838 merged 3 commits into
apache:masterfrom
Caideyipi:fix-pipe-page-memory
Jun 3, 2026
Merged

Pipe: account page decode memory in scan parser#17807
jt2594838 merged 3 commits into
apache:masterfrom
Caideyipi:fix-pipe-page-memory

Conversation

@Caideyipi

Copy link
Copy Markdown
Collaborator

Summary

  • Account page decode memory before IChunkReader#nextPageData() in the Pipe scan parser.
  • Add a memory-controlled chunk reader wrapper for normal and aligned multi-page chunks.
  • Estimate page decompression plus BatchData/vector object memory and use it for splitting aligned chunks.
  • Add a regression test for high-compression aligned multi-page chunks.

Tests

  • mvn -pl iotdb-core/datanode spotless:apply
  • git diff --check
  • mvn -pl iotdb-core/datanode -DskipTests compile failed due to unrelated existing datanode compile errors in schema/query modules.

@jt2594838 jt2594838 merged commit 59b79ef into apache:master Jun 3, 2026
41 checks passed
@jt2594838 jt2594838 deleted the fix-pipe-page-memory branch June 3, 2026 07:13
jt2594838 pushed a commit that referenced this pull request Jun 3, 2026
…#17833)

* Pipe: account page decode memory in scan parser (#17807)

* Pipe: account page decode memory in scan parser

* Fix pipe scan parser single page row count

* Fix pipe scan parser page memory test

* Fix tsfile writer API in pipe test
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants