Skip to content

Commit 5e8a38a

Browse files
JackieTien97JackieTien97
authored
Add CLAUDE.md files and update .gitignore (#788)
1 parent c934f78 commit 5e8a38a

5 files changed

Lines changed: 386 additions & 0 deletions

File tree

‎.gitignore‎

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -46,3 +46,9 @@ cpp/third_party/zlib-1.3.1
4646
build/*
4747
cpp/third_party/zlib-1.3.1/treebuild.xml
4848
cpp/third_party/zlib-1.3.1/zlib-1.3.1/treebuild.xml
49+
50+
# Claude Code
51+
.claude/settings.local.json
52+
.claude/todos/
53+
.claude/worktrees/
54+
.claude/scheduled_tasks.json

‎CLAUDE.md‎

Lines changed: 67 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,67 @@
1+
<!--
2+
3+
Licensed to the Apache Software Foundation (ASF) under one
4+
or more contributor license agreements. See the NOTICE file
5+
distributed with this work for additional information
6+
regarding copyright ownership. The ASF licenses this file
7+
to you under the Apache License, Version 2.0 (the
8+
"License"); you may not use this file except in compliance
9+
with the License. You may obtain a copy of the License at
10+
11+
http://www.apache.org/licenses/LICENSE-2.0
12+
13+
Unless required by applicable law or agreed to in writing,
14+
software distributed under the License is distributed on an
15+
"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
16+
KIND, either express or implied. See the License for the
17+
specific language governing permissions and limitations
18+
under the License.
19+
20+
-->
21+
22+
# CLAUDE.md
23+
24+
This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.
25+
26+
## Project Overview
27+
28+
Apache TsFile is a columnar storage file format designed for time series data. The project provides implementations in three languages, each in its own top-level directory. The main branch is `develop`.
29+
30+
TsFile uses a hierarchical storage model: **Page → Chunk → ChunkGroup → File**. Data is organized by device, with each device's measurements stored as individual time series. Two write models exist: **aligned** (all measurements share timestamps) and **non-aligned** (independent timestamps per measurement).
31+
32+
## Repository Structure
33+
34+
| Directory | Language | Build System | Details |
35+
|-----------|----------|--------------|---------|
36+
| `java/` | Java | Maven | [java/CLAUDE.md](java/CLAUDE.md) |
37+
| `cpp/` | C++ | CMake | [cpp/CLAUDE.md](cpp/CLAUDE.md) |
38+
| `python/` | Python | setuptools + Cython | [python/CLAUDE.md](python/CLAUDE.md) |
39+
40+
The root `pom.xml` orchestrates all three via Maven profiles:
41+
42+
```bash
43+
./mvnw clean verify -P with-java # Java only
44+
./mvnw clean verify -P with-cpp # C++ only
45+
./mvnw clean verify -P with-python # Python (requires C++ built first)
46+
```
47+
48+
## Cross-Module Dependencies
49+
50+
- **Python depends on C++**: The Python module uses Cython to bind to the C++ shared library (`libtsfile`). C++ must be built before Python (`-P with-python` implies C++ build).
51+
- **Java is independent**: The Java implementation shares no build-time dependency with C++ or Python.
52+
- All three implementations read/write the same TsFile binary format.
53+
54+
## Code Formatting
55+
56+
```bash
57+
./mvnw spotless:apply # Format all languages (Java: Google Java Format, C++: clang-format, Python: Black)
58+
./mvnw spotless:check # Check formatting without modifying
59+
```
60+
61+
## License Header
62+
63+
Every new file must include the Apache License 2.0 header at the top. Use the comment style appropriate for the file type (e.g., `<!-- -->` for Markdown, `/* */` for Java/C++, `#` for Python). See any existing file for the exact wording.
64+
65+
## Git Commit
66+
67+
- Do NOT add `Co-Authored-By` trailer to commit messages.

‎cpp/CLAUDE.md‎

Lines changed: 108 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,108 @@
1+
<!--
2+
3+
Licensed to the Apache Software Foundation (ASF) under one
4+
or more contributor license agreements. See the NOTICE file
5+
distributed with this work for additional information
6+
regarding copyright ownership. The ASF licenses this file
7+
to you under the Apache License, Version 2.0 (the
8+
"License"); you may not use this file except in compliance
9+
with the License. You may obtain a copy of the License at
10+
11+
http://www.apache.org/licenses/LICENSE-2.0
12+
13+
Unless required by applicable law or agreed to in writing,
14+
software distributed under the License is distributed on an
15+
"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
16+
KIND, either express or implied. See the License for the
17+
specific language governing permissions and limitations
18+
under the License.
19+
20+
-->
21+
22+
# CLAUDE.md — C++ Module
23+
24+
This file provides guidance to Claude Code (claude.ai/code) when working with the C++ module.
25+
26+
## Build Commands
27+
28+
```bash
29+
# Build (Release, default)
30+
bash build.sh
31+
32+
# Build variants
33+
bash build.sh -t=Debug
34+
bash build.sh -t=RelWithDebInfo
35+
bash build.sh -a=ON # Enable AddressSanitizer
36+
bash build.sh -c=ON # Enable code coverage
37+
38+
# Disable optional compression libraries
39+
bash build.sh --disable-snappy --disable-lz4 --disable-lzokay --disable-zlib
40+
41+
# Or use CMake directly
42+
mkdir -p build/Release && cd build/Release
43+
cmake ../.. -DCMAKE_BUILD_TYPE=Release -DBUILD_TEST=ON
44+
make -j$(nproc)
45+
46+
# Via Maven from repo root
47+
./mvnw clean verify -P with-cpp
48+
49+
# Run all tests
50+
./build/Release/lib/TsFile_Test
51+
52+
# Run specific test suite
53+
./build/Release/lib/TsFile_Test --gtest_filter=SnappyCompressorTest.*
54+
```
55+
56+
## Build Options (CMake)
57+
58+
All `ON` by default unless noted:
59+
60+
| Option | Purpose |
61+
|--------|---------|
62+
| `BUILD_TEST` | Compile tests (GTest 1.12.1, auto-downloaded) |
63+
| `ENABLE_ANTLR4` | ANTLR4 parser runtime |
64+
| `ENABLE_SNAPPY` / `ENABLE_LZ4` / `ENABLE_LZOKAY` / `ENABLE_ZLIB` | Compression libraries |
65+
| `ENABLE_THREADS` | Multi-threaded read/write via pthreads |
66+
| `ENABLE_ASAN` | AddressSanitizer (`OFF` by default) |
67+
| `ENABLE_SIMDE` | SIMD Everywhere (`OFF` by default) |
68+
69+
## Source Structure
70+
71+
```
72+
cpp/src/
73+
├── common/ # Core types: Schema, Tablet, DeviceId, TsBlock, allocators, config
74+
├── compress/ # Compression: Snappy, LZ4, LZOKAY, Zlib (factory pattern)
75+
├── encoding/ # Encoding: Plain, TS2Diff, Gorilla, Dictionary, RLE, Zigzag, SPRINTZ
76+
├── file/ # File I/O: TsFileIOReader/Writer, RestorableTsFileIOWriter
77+
├── reader/ # Read path: TsFileReader, QueryExecutor, filters, result sets
78+
├── writer/ # Write path: TsFileWriter, TsFileTableWriter, ChunkWriter, PageWriter
79+
├── parser/ # ANTLR4 path parser (grammars + generated code)
80+
├── cwrapper/ # C language bindings (used by Python module)
81+
└── utils/ # Utilities: error codes, date handling, fault injection
82+
```
83+
84+
## Architecture Notes
85+
86+
- **C++11** standard, targets CMake 3.11+
87+
- Dual data model: **tree-view** (`TsFileTreeWriter/Reader`) and **table-view** (`TsFileTableWriter`, `TableQueryExecutor`)
88+
- Parallel column encoding in table write path, controlled by `ENABLE_THREADS`
89+
- Third-party libraries are bundled under `third_party/` (ANTLR4, Snappy, LZ4, LZOKAY, Zlib, SIMDe)
90+
- `cwrapper/` provides the C API that the Python module binds to via Cython
91+
92+
## Code Style
93+
94+
- **Formatter**: clang-format (Google style), configured in `.clang-format`
95+
96+
## Testing
97+
98+
- **Framework**: Google Test 1.12.1 (auto-downloaded during build, or supply `third_party/googletest-release-1.12.1.zip`)
99+
- Tests in `cpp/test/`, mirroring `src/` structure
100+
- Test discovery via `gtest_discover_tests()`
101+
102+
## License Header
103+
104+
Every new file must include the Apache License 2.0 header at the top. For C/C++ files, use the `/* */` block comment style. See any existing `.h` or `.cc` file for the exact wording.
105+
106+
## Git Commit
107+
108+
- Do NOT add `Co-Authored-By` trailer to commit messages.

‎java/CLAUDE.md‎

Lines changed: 101 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,101 @@
1+
<!--
2+
3+
Licensed to the Apache Software Foundation (ASF) under one
4+
or more contributor license agreements. See the NOTICE file
5+
distributed with this work for additional information
6+
regarding copyright ownership. The ASF licenses this file
7+
to you under the Apache License, Version 2.0 (the
8+
"License"); you may not use this file except in compliance
9+
with the License. You may obtain a copy of the License at
10+
11+
http://www.apache.org/licenses/LICENSE-2.0
12+
13+
Unless required by applicable law or agreed to in writing,
14+
software distributed under the License is distributed on an
15+
"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
16+
KIND, either express or implied. See the License for the
17+
specific language governing permissions and limitations
18+
under the License.
19+
20+
-->
21+
22+
# CLAUDE.md — Java Module
23+
24+
This file provides guidance to Claude Code (claude.ai/code) when working with the Java module.
25+
26+
## Build Commands
27+
28+
Maven with wrapper (`./mvnw` from repo root). All commands require `-P with-java`.
29+
30+
```bash
31+
# Build (skip tests)
32+
./mvnw clean package -P with-java -DskipTests
33+
34+
# Install to local Maven repo
35+
./mvnw install -P with-java -DskipTests
36+
37+
# Run all tests (unit + integration)
38+
./mvnw clean verify -P with-java
39+
40+
# Run unit tests only
41+
./mvnw test -P with-java
42+
43+
# Run integration tests only
44+
./mvnw verify -P with-java -DskipUTs=true
45+
46+
# Run a single test class (use -pl to target submodule)
47+
./mvnw test -P with-java -pl java/tsfile -Dtest=TsFileWriterTest
48+
49+
# Run a single test method
50+
./mvnw test -P with-java -pl java/tsfile -Dtest=TsFileWriterTest#testWrite
51+
52+
# Format code (requires Java 17+ runtime)
53+
./mvnw spotless:apply
54+
55+
# Check formatting
56+
./mvnw spotless:check
57+
```
58+
59+
## Module Structure
60+
61+
Four submodules under `java/`:
62+
63+
- **common** — Shared types: `TSDataType`, `ColumnCategory`, `Binary`, `BitMap`, constants
64+
- **tsfile** — Core implementation: read/write APIs, encoding, compression, file format
65+
- **examples** — Reference implementations for read/write operations
66+
- **tools** — CLI utilities for TsFile inspection
67+
68+
## Architecture
69+
70+
Key packages in `java/tsfile/src/main/java/org/apache/tsfile/`:
71+
72+
- `write/` — Write path: `TsFileWriter`, `Tablet` (batch API), `TSRecord` (row API)
73+
- `read/` — Read path: `TsFileReader`, `TsFileSequenceReader`, `QueryExpression`
74+
- `encoding/` — Encoding algorithms (RLE, TS_2DIFF, GORILLA, DICTIONARY)
75+
- `compress/` — Compression codecs (Snappy, LZ4, ZSTD, XZ)
76+
- `file/` — File format structures (metadata, headers, footers)
77+
- `compatibility/` — Version compatibility handling
78+
- `parser/` — ANTLR4-generated path parser (grammar in `src/main/antlr4/`)
79+
80+
Code generation: FreeMarker templates in `tsfile/src/main/codegen/` generate type-specific implementations (e.g., per-datatype readers/writers) into `target/generated-sources/`.
81+
82+
## Code Style
83+
84+
- **Formatter**: Spotless with Google Java Format 1.28.0 (Spotless requires Java 17+ to run but the project targets Java 8 bytecode)
85+
- **Checkstyle**: Google style variant in root `checkstyle.xml`, 100 char line limit
86+
- **Import order**: `org.apache.tsfile`, `javax`, `java`, static imports
87+
88+
## Testing Conventions
89+
90+
- **Framework**: JUnit 4
91+
- Unit tests: `*Test.java` — run via `mvn test`
92+
- Integration tests: `*IT.java` — run via `mvn verify`
93+
- Tests run in random order with `reuseForks=false`
94+
95+
## License Header
96+
97+
Every new file must include the Apache License 2.0 header at the top. For Java files, use the `/* */` block comment style. See any existing `.java` file for the exact wording.
98+
99+
## Git Commit
100+
101+
- Do NOT add `Co-Authored-By` trailer to commit messages.

‎python/CLAUDE.md‎

Lines changed: 104 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,104 @@
1+
<!--
2+
3+
Licensed to the Apache Software Foundation (ASF) under one
4+
or more contributor license agreements. See the NOTICE file
5+
distributed with this work for additional information
6+
regarding copyright ownership. The ASF licenses this file
7+
to you under the Apache License, Version 2.0 (the
8+
"License"); you may not use this file except in compliance
9+
with the License. You may obtain a copy of the License at
10+
11+
http://www.apache.org/licenses/LICENSE-2.0
12+
13+
Unless required by applicable law or agreed to in writing,
14+
software distributed under the License is distributed on an
15+
"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
16+
KIND, either express or implied. See the License for the
17+
specific language governing permissions and limitations
18+
under the License.
19+
20+
-->
21+
22+
# CLAUDE.md — Python Module
23+
24+
This file provides guidance to Claude Code (claude.ai/code) when working with the Python module.
25+
26+
## Prerequisites
27+
28+
The Python module depends on the C++ shared library (`libtsfile`). **Build C++ first:**
29+
30+
```bash
31+
# From repo root
32+
./mvnw clean package -P with-cpp -DskipTests
33+
```
34+
35+
## Build Commands
36+
37+
```bash
38+
# Build everything via Maven (builds C++ then Python)
39+
./mvnw clean verify -P with-python
40+
41+
# Or build directly (after C++ is built)
42+
cd python
43+
python setup.py build_ext --inplace
44+
45+
# Run tests
46+
pytest tests/
47+
48+
# Run a single test
49+
pytest tests/test_write_and_read.py -k "test_name"
50+
51+
# Format code
52+
cd ../ && ./mvnw spotless:apply # Uses Black 26.3.1
53+
```
54+
55+
## Source Structure
56+
57+
```
58+
python/tsfile/
59+
├── __init__.py # Public API exports
60+
├── constants.py # Enums: TSDataType, TSEncoding, Compressor, ColumnCategory
61+
├── schema.py # TimeseriesSchema, DeviceSchema, TableSchema, ColumnSchema
62+
├── tablet.py # Tablet class for batch columnar data
63+
├── field.py / row_record.py # Row-based data structures
64+
├── tag_filter.py # Tag filtering DSL for queries
65+
├── tsfile_table_writer.py # High-level table writer API
66+
├── utils.py # to_dataframe(), dataframe_to_tsfile()
67+
├── exceptions.py # Custom exception types
68+
│
69+
├── tsfile_py_cpp.pyx # Cython core: Python ↔ C++ type conversion
70+
├── tsfile_reader.pyx # Cython reader wrapper
71+
├── tsfile_writer.pyx # Cython writer wrapper
72+
├── tsfile_cpp.pxd # C++ interface declarations (cwrapper)
73+
│
74+
└── dataset/ # High-level multi-file API
75+
├── dataframe.py # TsFileDataFrame (pandas-like interface)
76+
├── timeseries.py # Timeseries, AlignedTimeseries
77+
├── reader.py # Multi-file reader coordination
78+
└── merge.py # K-way merge for overlapping time ranges
79+
```
80+
81+
## Architecture Notes
82+
83+
- **Layered design**: C++ core → Cython bridge (`*.pyx`) → Pure Python API
84+
- `setup.py` copies pre-built C++ headers and shared libraries from `../cpp/target/build/`
85+
- Cython binds to `cwrapper/tsfile_cwrapper.h` from the C++ module
86+
- C error codes are mapped to Python exceptions via `check_error()` in `tsfile_py_cpp.pyx`
87+
- Requires Python 3.9+, numpy >= 2.0, pandas >= 2.0, pyarrow >= 16.0
88+
89+
## Code Style
90+
91+
- **Formatter**: Black 26.3.1 (via Spotless Maven plugin)
92+
93+
## Testing
94+
95+
- **Framework**: pytest
96+
- Tests in `python/tests/`, 19 test files covering write, read, filtering, DataFrame conversion, and Arrow format
97+
98+
## License Header
99+
100+
Every new file must include the Apache License 2.0 header at the top. For Python files, use `#` line comments. For `.pyx`/`.pxd` files, also use `#` comments. See any existing file for the exact wording.
101+
102+
## Git Commit
103+
104+
- Do NOT add `Co-Authored-By` trailer to commit messages.

0 commit comments

Comments
 (0)