[FEA] Add `bitset_filter` for CAGRA indices removal #1837

lowener · 2023-09-22T00:30:39Z

Closes #1600.
To be merged after #1803 and #1811
This PR adds bitset_filter to filter an index with a bitset.

copy-pr-bot · 2023-09-22T00:30:42Z

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

lowener · 2023-09-22T14:01:59Z

Current benchmark performance on my local machine (with the corresponding removed_ratio of 2%, 4%, 8%, ..):

AnnCagra/float/uint32_t/21/manual_time       4.09 ms         4.13 ms          171 block_size=256 dataset (GiB)=0.953674 degree=64 graph (GiB)=0.476837 iterations=36 itopk_size=64 k=32 n_cols=128 n_queries=1000 n_rows=2M removed_ratio=0 search_width=2
AnnCagra/float/uint32_t/22/manual_time       4.10 ms         4.14 ms          171 block_size=256 dataset (GiB)=0.953674 degree=64 graph (GiB)=0.476837 iterations=36 itopk_size=64 k=32 n_cols=128 n_queries=1000 n_rows=2M removed_ratio=0.02 search_width=2
AnnCagra/float/uint32_t/23/manual_time       4.10 ms         4.14 ms          171 block_size=256 dataset (GiB)=0.953674 degree=64 graph (GiB)=0.476837 iterations=36 itopk_size=64 k=32 n_cols=128 n_queries=1000 n_rows=2M removed_ratio=0.04 search_width=2
AnnCagra/float/uint32_t/24/manual_time       4.11 ms         4.15 ms          170 block_size=256 dataset (GiB)=0.953674 degree=64 graph (GiB)=0.476837 iterations=36 itopk_size=64 k=32 n_cols=128 n_queries=1000 n_rows=2M removed_ratio=0.08 search_width=2
AnnCagra/float/uint32_t/25/manual_time       4.10 ms         4.14 ms          171 block_size=256 dataset (GiB)=0.953674 degree=64 graph (GiB)=0.476837 iterations=36 itopk_size=64 k=32 n_cols=128 n_queries=1000 n_rows=2M removed_ratio=0.16 search_width=2
AnnCagra/float/uint32_t/26/manual_time       4.09 ms         4.13 ms          171 block_size=256 dataset (GiB)=0.953674 degree=64 graph (GiB)=0.476837 iterations=36 itopk_size=64 k=32 n_cols=128 n_queries=1000 n_rows=2M removed_ratio=0.32 search_width=2
AnnCagra/float/uint32_t/27/manual_time       4.09 ms         4.13 ms          171 block_size=256 dataset (GiB)=0.953674 degree=64 graph (GiB)=0.476837 iterations=36 itopk_size=64 k=32 n_cols=128 n_queries=1000 n_rows=2M removed_ratio=0.64 search_width=2

cjnolet · 2023-09-26T20:30:20Z

/ok to test

lowener · 2023-09-26T21:30:57Z

/ok to test

Signed-off-by: Mickael Ide <[email protected]>

lowener · 2023-09-26T21:50:53Z

/ok to test

Co-authored-by: Micka <[email protected]>

…g' into 23.10-cagra-remove

lowener · 2023-09-28T20:24:38Z

/ok to test

lowener · 2023-09-28T20:34:30Z

Latest benchmark results on my local machine. The difference between none_filter and bitset_filter is definitely noticeable now.

AnnCagra/float/uint32_t/21/manual_time       3.72 ms         3.76 ms          188 block_size=256 dataset (GiB)=0.953674 degree=64 graph (GiB)=0.476837 iterations=36 itopk_size=64 k=32 n_cols=128 n_queries=1000 n_rows=2M removed_ratio=0 search_width=2
AnnCagra/float/uint32_t/22/manual_time       6.15 ms         6.19 ms          114 block_size=256 dataset (GiB)=0.953674 degree=64 graph (GiB)=0.476837 iterations=36 itopk_size=64 k=32 n_cols=128 n_queries=1000 n_rows=2M removed_ratio=0.02 search_width=2
AnnCagra/float/uint32_t/23/manual_time       6.17 ms         6.22 ms          113 block_size=256 dataset (GiB)=0.953674 degree=64 graph (GiB)=0.476837 iterations=36 itopk_size=64 k=32 n_cols=128 n_queries=1000 n_rows=2M removed_ratio=0.04 search_width=2
AnnCagra/float/uint32_t/24/manual_time       6.18 ms         6.22 ms          113 block_size=256 dataset (GiB)=0.953674 degree=64 graph (GiB)=0.476837 iterations=36 itopk_size=64 k=32 n_cols=128 n_queries=1000 n_rows=2M removed_ratio=0.08 search_width=2
AnnCagra/float/uint32_t/25/manual_time       6.19 ms         6.23 ms          113 block_size=256 dataset (GiB)=0.953674 degree=64 graph (GiB)=0.476837 iterations=36 itopk_size=64 k=32 n_cols=128 n_queries=1000 n_rows=2M removed_ratio=0.16 search_width=2
AnnCagra/float/uint32_t/26/manual_time       6.20 ms         6.24 ms          113 block_size=256 dataset (GiB)=0.953674 degree=64 graph (GiB)=0.476837 iterations=36 itopk_size=64 k=32 n_cols=128 n_queries=1000 n_rows=2M removed_ratio=0.32 search_width=2
AnnCagra/float/uint32_t/27/manual_time       6.25 ms         6.28 ms          112 block_size=256 dataset (GiB)=0.953674 degree=64 graph (GiB)=0.476837 iterations=36 itopk_size=64 k=32 n_cols=128 n_queries=1000 n_rows=2M removed_ratio=0.64 search_width=2

lowener · 2023-09-29T15:16:53Z

Current result of benchmark of bench-prims on 10M points, batch size 1, 10, 10lk

AnnCagra/float/uint32_t/35/manual_time      0.299 ms        0.340 ms         2343 block_size=256 dataset (GiB)=4.76837 degree=64 graph (GiB)=2.38419 iterations=36 itopk_size=64 k=32 n_cols=128 n_queries=1 n_rows=10M removed_ratio=0 search_width=2
AnnCagra/float/uint32_t/36/manual_time      0.332 ms        0.373 ms         2111 block_size=256 dataset (GiB)=4.76837 degree=64 graph (GiB)=2.38419 iterations=36 itopk_size=64 k=32 n_cols=128 n_queries=1 n_rows=10M removed_ratio=0.02 search_width=2
AnnCagra/float/uint32_t/37/manual_time      0.333 ms        0.375 ms         2100 block_size=256 dataset (GiB)=4.76837 degree=64 graph (GiB)=2.38419 iterations=36 itopk_size=64 k=32 n_cols=128 n_queries=1 n_rows=10M removed_ratio=0.04 search_width=2
AnnCagra/float/uint32_t/38/manual_time      0.339 ms        0.381 ms         2062 block_size=256 dataset (GiB)=4.76837 degree=64 graph (GiB)=2.38419 iterations=36 itopk_size=64 k=32 n_cols=128 n_queries=1 n_rows=10M removed_ratio=0.08 search_width=2
AnnCagra/float/uint32_t/39/manual_time      0.351 ms        0.393 ms         1991 block_size=256 dataset (GiB)=4.76837 degree=64 graph (GiB)=2.38419 iterations=36 itopk_size=64 k=32 n_cols=128 n_queries=1 n_rows=10M removed_ratio=0.16 search_width=2
AnnCagra/float/uint32_t/40/manual_time      0.369 ms        0.411 ms         1895 block_size=256 dataset (GiB)=4.76837 degree=64 graph (GiB)=2.38419 iterations=36 itopk_size=64 k=32 n_cols=128 n_queries=1 n_rows=10M removed_ratio=0.32 search_width=2
AnnCagra/float/uint32_t/41/manual_time      0.382 ms        0.424 ms         1831 block_size=256 dataset (GiB)=4.76837 degree=64 graph (GiB)=2.38419 iterations=36 itopk_size=64 k=32 n_cols=128 n_queries=1 n_rows=10M removed_ratio=0.64 search_width=2
AnnCagra/float/uint32_t/42/manual_time      0.308 ms        0.347 ms         2271 block_size=256 dataset (GiB)=4.76837 degree=64 graph (GiB)=2.38419 iterations=36 itopk_size=64 k=32 n_cols=128 n_queries=10 n_rows=10M removed_ratio=0 search_width=2
AnnCagra/float/uint32_t/43/manual_time      0.342 ms        0.381 ms         2047 block_size=256 dataset (GiB)=4.76837 degree=64 graph (GiB)=2.38419 iterations=36 itopk_size=64 k=32 n_cols=128 n_queries=10 n_rows=10M removed_ratio=0.02 search_width=2
AnnCagra/float/uint32_t/44/manual_time      0.344 ms        0.384 ms         2032 block_size=256 dataset (GiB)=4.76837 degree=64 graph (GiB)=2.38419 iterations=36 itopk_size=64 k=32 n_cols=128 n_queries=10 n_rows=10M removed_ratio=0.04 search_width=2
AnnCagra/float/uint32_t/45/manual_time      0.353 ms        0.392 ms         1982 block_size=256 dataset (GiB)=4.76837 degree=64 graph (GiB)=2.38419 iterations=36 itopk_size=64 k=32 n_cols=128 n_queries=10 n_rows=10M removed_ratio=0.08 search_width=2
AnnCagra/float/uint32_t/46/manual_time      0.360 ms        0.399 ms         1945 block_size=256 dataset (GiB)=4.76837 degree=64 graph (GiB)=2.38419 iterations=36 itopk_size=64 k=32 n_cols=128 n_queries=10 n_rows=10M removed_ratio=0.16 search_width=2
AnnCagra/float/uint32_t/47/manual_time      0.382 ms        0.421 ms         1834 block_size=256 dataset (GiB)=4.76837 degree=64 graph (GiB)=2.38419 iterations=36 itopk_size=64 k=32 n_cols=128 n_queries=10 n_rows=10M removed_ratio=0.32 search_width=2
AnnCagra/float/uint32_t/48/manual_time      0.393 ms        0.433 ms         1779 block_size=256 dataset (GiB)=4.76837 degree=64 graph (GiB)=2.38419 iterations=36 itopk_size=64 k=32 n_cols=128 n_queries=10 n_rows=10M removed_ratio=0.64 search_width=2
AnnCagra/float/uint32_t/49/manual_time       37.4 ms         37.4 ms           19 block_size=256 dataset (GiB)=4.76837 degree=64 graph (GiB)=2.38419 iterations=36 itopk_size=64 k=32 n_cols=128 n_queries=10k n_rows=10M removed_ratio=0 search_width=2
AnnCagra/float/uint32_t/50/manual_time       62.0 ms         62.0 ms           11 block_size=256 dataset (GiB)=4.76837 degree=64 graph (GiB)=2.38419 iterations=36 itopk_size=64 k=32 n_cols=128 n_queries=10k n_rows=10M removed_ratio=0.02 search_width=2
AnnCagra/float/uint32_t/51/manual_time       62.1 ms         62.1 ms           11 block_size=256 dataset (GiB)=4.76837 degree=64 graph (GiB)=2.38419 iterations=36 itopk_size=64 k=32 n_cols=128 n_queries=10k n_rows=10M removed_ratio=0.04 search_width=2
AnnCagra/float/uint32_t/52/manual_time       62.3 ms         62.3 ms           11 block_size=256 dataset (GiB)=4.76837 degree=64 graph (GiB)=2.38419 iterations=36 itopk_size=64 k=32 n_cols=128 n_queries=10k n_rows=10M removed_ratio=0.08 search_width=2
AnnCagra/float/uint32_t/53/manual_time       62.8 ms         62.4 ms           11 block_size=256 dataset (GiB)=4.76837 degree=64 graph (GiB)=2.38419 iterations=36 itopk_size=64 k=32 n_cols=128 n_queries=10k n_rows=10M removed_ratio=0.16 search_width=2
AnnCagra/float/uint32_t/54/manual_time       62.4 ms         62.4 ms           11 block_size=256 dataset (GiB)=4.76837 degree=64 graph (GiB)=2.38419 iterations=36 itopk_size=64 k=32 n_cols=128 n_queries=10k n_rows=10M removed_ratio=0.32 search_width=2
AnnCagra/float/uint32_t/55/manual_time       62.9 ms         62.9 ms           11 block_size=256 dataset (GiB)=4.76837 degree=64 graph (GiB)=2.38419 iterations=36 itopk_size=64 k=32 n_cols=128 n_queries=10k n_rows=10M removed_ratio=0.64 search_width=2

lowener · 2023-09-29T16:55:14Z

/ok to test

lowener · 2023-10-02T09:31:32Z

AnnCagra/float/uint32_t/35/manual_time       2.52 ms         2.56 ms          277 block_size=256 dataset (GiB)=4.76837 degree=64 graph (GiB)=2.38419 iterations=161 itopk_size=300 k=255 n_cols=128 n_queries=1 n_rows=10M removed_ratio=0 search_width=2
AnnCagra/float/uint32_t/36/manual_time       2.33 ms         2.36 ms          301 block_size=256 dataset (GiB)=4.76837 degree=64 graph (GiB)=2.38419 iterations=161 itopk_size=300 k=255 n_cols=128 n_queries=1 n_rows=10M removed_ratio=0.02 search_width=2
AnnCagra/float/uint32_t/37/manual_time       2.41 ms         2.45 ms          290 block_size=256 dataset (GiB)=4.76837 degree=64 graph (GiB)=2.38419 iterations=161 itopk_size=300 k=255 n_cols=128 n_queries=1 n_rows=10M removed_ratio=0.04 search_width=2
AnnCagra/float/uint32_t/38/manual_time       2.58 ms         2.62 ms          271 block_size=256 dataset (GiB)=4.76837 degree=64 graph (GiB)=2.38419 iterations=161 itopk_size=300 k=255 n_cols=128 n_queries=1 n_rows=10M removed_ratio=0.08 search_width=2
AnnCagra/float/uint32_t/39/manual_time       2.83 ms         2.86 ms          247 block_size=256 dataset (GiB)=4.76837 degree=64 graph (GiB)=2.38419 iterations=161 itopk_size=300 k=255 n_cols=128 n_queries=1 n_rows=10M removed_ratio=0.16 search_width=2
AnnCagra/float/uint32_t/40/manual_time       3.37 ms         3.41 ms          207 block_size=256 dataset (GiB)=4.76837 degree=64 graph (GiB)=2.38419 iterations=161 itopk_size=300 k=255 n_cols=128 n_queries=1 n_rows=10M removed_ratio=0.32 search_width=2
AnnCagra/float/uint32_t/41/manual_time       3.86 ms         3.90 ms          181 block_size=256 dataset (GiB)=4.76837 degree=64 graph (GiB)=2.38419 iterations=161 itopk_size=300 k=255 n_cols=128 n_queries=1 n_rows=10M removed_ratio=0.64 search_width=2
AnnCagra/float/uint32_t/42/manual_time       2.59 ms         2.63 ms          270 block_size=256 dataset (GiB)=4.76837 degree=64 graph (GiB)=2.38419 iterations=161 itopk_size=300 k=255 n_cols=128 n_queries=10 n_rows=10M removed_ratio=0 search_width=2
AnnCagra/float/uint32_t/43/manual_time       2.52 ms         2.56 ms          278 block_size=256 dataset (GiB)=4.76837 degree=64 graph (GiB)=2.38419 iterations=161 itopk_size=300 k=255 n_cols=128 n_queries=10 n_rows=10M removed_ratio=0.02 search_width=2
AnnCagra/float/uint32_t/44/manual_time       2.59 ms         2.63 ms          270 block_size=256 dataset (GiB)=4.76837 degree=64 graph (GiB)=2.38419 iterations=161 itopk_size=300 k=255 n_cols=128 n_queries=10 n_rows=10M removed_ratio=0.04 search_width=2
AnnCagra/float/uint32_t/45/manual_time       2.73 ms         2.77 ms          256 block_size=256 dataset (GiB)=4.76837 degree=64 graph (GiB)=2.38419 iterations=161 itopk_size=300 k=255 n_cols=128 n_queries=10 n_rows=10M removed_ratio=0.08 search_width=2
AnnCagra/float/uint32_t/46/manual_time       3.01 ms         3.05 ms          232 block_size=256 dataset (GiB)=4.76837 degree=64 graph (GiB)=2.38419 iterations=161 itopk_size=300 k=255 n_cols=128 n_queries=10 n_rows=10M removed_ratio=0.16 search_width=2
AnnCagra/float/uint32_t/47/manual_time       3.64 ms         3.67 ms          192 block_size=256 dataset (GiB)=4.76837 degree=64 graph (GiB)=2.38419 iterations=161 itopk_size=300 k=255 n_cols=128 n_queries=10 n_rows=10M removed_ratio=0.32 search_width=2
AnnCagra/float/uint32_t/48/manual_time       4.02 ms         4.06 ms          174 block_size=256 dataset (GiB)=4.76837 degree=64 graph (GiB)=2.38419 iterations=161 itopk_size=300 k=255 n_cols=128 n_queries=10 n_rows=10M removed_ratio=0.64 search_width=2
AnnCagra/float/uint32_t/49/manual_time        485 ms          485 ms            2 block_size=256 dataset (GiB)=4.76837 degree=64 graph (GiB)=2.38419 iterations=161 itopk_size=300 k=255 n_cols=128 n_queries=10k n_rows=10M removed_ratio=0 search_width=2
AnnCagra/float/uint32_t/50/manual_time        632 ms          631 ms            1 block_size=256 dataset (GiB)=4.76837 degree=64 graph (GiB)=2.38419 iterations=161 itopk_size=300 k=255 n_cols=128 n_queries=10k n_rows=10M removed_ratio=0.02 search_width=2
AnnCagra/float/uint32_t/51/manual_time        629 ms          629 ms            1 block_size=256 dataset (GiB)=4.76837 degree=64 graph (GiB)=2.38419 iterations=161 itopk_size=300 k=255 n_cols=128 n_queries=10k n_rows=10M removed_ratio=0.04 search_width=2
AnnCagra/float/uint32_t/52/manual_time        631 ms          624 ms            1 block_size=256 dataset (GiB)=4.76837 degree=64 graph (GiB)=2.38419 iterations=161 itopk_size=300 k=255 n_cols=128 n_queries=10k n_rows=10M removed_ratio=0.08 search_width=2
AnnCagra/float/uint32_t/53/manual_time        617 ms          617 ms            1 block_size=256 dataset (GiB)=4.76837 degree=64 graph (GiB)=2.38419 iterations=161 itopk_size=300 k=255 n_cols=128 n_queries=10k n_rows=10M removed_ratio=0.16 search_width=2
AnnCagra/float/uint32_t/54/manual_time        608 ms          607 ms            1 block_size=256 dataset (GiB)=4.76837 degree=64 graph (GiB)=2.38419 iterations=161 itopk_size=300 k=255 n_cols=128 n_queries=10k n_rows=10M removed_ratio=0.32 search_width=2
AnnCagra/float/uint32_t/55/manual_time        605 ms          604 ms            1 block_size=256 dataset (GiB)=4.76837 degree=64 graph (GiB)=2.38419 iterations=161 itopk_size=300 k=255 n_cols=128 n_queries=10k n_rows=10M removed_ratio=0.64 search_width=2

lowener · 2023-10-02T14:55:06Z

/ok to test

This PR fixes a bug in the filtering operations in the CAGRA multi-kernel search implementation. This bug caused the test of #1837 to fail. Authors: - tsuki (https://github.com/enp1s0) Approvers: - Micka (https://github.com/lowener) - Corey J. Nolet (https://github.com/cjnolet)

cjnolet

LGTM!

cjnolet · 2023-10-02T16:44:08Z

@lowener, I've approved this PR and I think the implementation looks good. The only thing lacking now is documentation. We should add a separate usage example to raft::cagra::search_with_filtering for specifying the bitset sample filter.

lowener · 2023-10-02T19:05:41Z

/ok to test

cjnolet · 2023-10-02T23:31:57Z

/ok to test

github-actions bot added cpp CMake labels Sep 22, 2023

lowener added the 2 - In Progress Currenty a work in progress label Sep 22, 2023

lowener added feature request New feature or request non-breaking Non-breaking change labels Sep 22, 2023

cjnolet assigned lowener Sep 26, 2023

lowener force-pushed the 23.10-cagra-remove branch from f21f826 to d4067e2 Compare September 26, 2023 20:24

lowener added 4 commits September 26, 2023 23:48

Initial bitset indices removal for CAGRA

4ab04c2

Signed-off-by: Mickael Ide <[email protected]>

Fix build and pre-filtering bug

02eebe5

Add benchmark for cagra remove function

0f09c37

Fix style

53b45dc

lowener force-pushed the 23.10-cagra-remove branch from 2165a6b to 53b45dc Compare September 26, 2023 21:49

github-actions bot removed the CMake label Sep 26, 2023

lowener mentioned this pull request Sep 27, 2023

23.10 cagra removal #1856

Closed

lowener and others added 4 commits September 27, 2023 18:39

Use std::optional for removed_indices

bad676a

Fix filter bug

1e4491e

Add filter validation

be97ffb

Fix error message

1c1f028

enp1s0 mentioned this pull request Sep 28, 2023

[BUG] Fix a bug in the filtering operation in CAGRA multi-kernel #1862

Merged

enp1s0 and others added 6 commits September 28, 2023 21:25

Fix info log in nn-descent

36c94ff

Fix log message in NN-descent

31639dc

Merge branch 'branch-23.10' into 23.10-cagra-remove

95d2aa3

Update cpp/include/raft/neighbors/detail/cagra/search_plan.cuh

694bcd2

Co-authored-by: Micka <[email protected]>

Merge remote-tracking branch 'enp1s0/cagra-fix-multi-kernel-filter-bu…

4f16f1d

…g' into 23.10-cagra-remove

Fix style

059f8e6

lowener added 3 commits September 28, 2023 22:10

Remove bitset from cagra class

d8e0421

Merge remote-tracking branch 'enp1s0/cagra-fix-multi-kernel-filter-bu…

0cf6cc0

…g' into 23.10-cagra-remove

Fix style

e62b540

lowener marked this pull request as ready for review September 28, 2023 20:24

lowener requested a review from a team as a code owner September 28, 2023 20:24

lowener added 3 - Ready for Review and removed 2 - In Progress Currenty a work in progress labels Sep 28, 2023

lowener changed the title ~~[FEA] Add CAGRA indices removal~~ [FEA] Add bitset_filter for CAGRA indices removal Sep 28, 2023

lowener added 2 commits September 29, 2023 17:04

Use 10M points and different batch in bench

66cbe8a

Fix batch size

aedb973

Use k=255 for cagra bench

cbea209

cjnolet approved these changes Oct 2, 2023

View reviewed changes

Add usage example for search_with_filtering

7ebf87f

Merge branch 'branch-23.10' into 23.10-cagra-remove

378fc52

raydouglass merged commit bf3f85a into rapidsai:branch-23.10 Oct 3, 2023
54 of 56 checks passed

lowener deleted the 23.10-cagra-remove branch November 3, 2023 13:12

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[FEA] Add `bitset_filter` for CAGRA indices removal #1837

[FEA] Add `bitset_filter` for CAGRA indices removal #1837

lowener commented Sep 22, 2023 •

edited

Loading

copy-pr-bot bot commented Sep 22, 2023

lowener commented Sep 22, 2023

cjnolet commented Sep 26, 2023

lowener commented Sep 26, 2023

lowener commented Sep 26, 2023

lowener commented Sep 28, 2023

lowener commented Sep 28, 2023

lowener commented Sep 29, 2023

lowener commented Sep 29, 2023

lowener commented Oct 2, 2023

lowener commented Oct 2, 2023

cjnolet left a comment

cjnolet commented Oct 2, 2023

lowener commented Oct 2, 2023

cjnolet commented Oct 2, 2023

[FEA] Add bitset_filter for CAGRA indices removal #1837

[FEA] Add bitset_filter for CAGRA indices removal #1837

Conversation

lowener commented Sep 22, 2023 • edited Loading

copy-pr-bot bot commented Sep 22, 2023

lowener commented Sep 22, 2023

cjnolet commented Sep 26, 2023

lowener commented Sep 26, 2023

lowener commented Sep 26, 2023

lowener commented Sep 28, 2023

lowener commented Sep 28, 2023

lowener commented Sep 29, 2023

lowener commented Sep 29, 2023

lowener commented Oct 2, 2023

lowener commented Oct 2, 2023

cjnolet left a comment

Choose a reason for hiding this comment

cjnolet commented Oct 2, 2023

lowener commented Oct 2, 2023

cjnolet commented Oct 2, 2023

[FEA] Add `bitset_filter` for CAGRA indices removal #1837

[FEA] Add `bitset_filter` for CAGRA indices removal #1837

lowener commented Sep 22, 2023 •

edited

Loading