Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add end-to-end CUDA ann-benchmarks to raft #1304

Merged
merged 49 commits into from
Mar 28, 2023
Merged

Add end-to-end CUDA ann-benchmarks to raft #1304

merged 49 commits into from
Mar 28, 2023

Conversation

cjnolet
Copy link
Member

@cjnolet cjnolet commented Feb 24, 2023 •
edited
Loading

New bench/ann artifact for comparing (C++ APIs for) GPU-acclerated algorithms end-to-end. Working on this w/ @tfeher but had to squash the original commits into a single commit.

Things left to do:

  • Separate benchmarks executables for each different algorithm
  • Separate build targets for ggnn and hnswlib
  • Revise bench/ann docs
  • Break factory.cuh abd benchmark.cu / benchmark.cpp into individual files for each different algorithm to make it easier to plug in new algorithms.
  • Separate into its own conda package

closes #1211

@cjnolet cjnolet added improvement Improvement / enhancement to an existing function non-breaking Non-breaking change labels Feb 24, 2023
@cjnolet cjnolet self-assigned this Feb 24, 2023
@cjnolet cjnolet changed the title Fea 2304 cuann benchmark cuann-benchmark tool for comparing benchmark algorithms end-to-end Feb 24, 2023
@cjnolet cjnolet changed the title cuann-benchmark tool for comparing benchmark algorithms end-to-end Add cuann-benchmarks to raft Feb 25, 2023
@github-actions github-actions bot added the ci label Feb 25, 2023
@cjnolet cjnolet changed the title Add cuann-benchmarks to raft Add end-to-end CUDA ann-benchmarks to raft Feb 25, 2023
@cjnolet cjnolet mentioned this pull request Feb 28, 2023
Closed
@cjnolet cjnolet marked this pull request as ready for review March 6, 2023 23:34
@cjnolet
Copy link
Member Author

cjnolet commented Mar 24, 2023 •
edited
Loading

Can we also publish the output of these benchmarks somewhere in the source tree, as well as instructions on how to run yourself and get the datasets?

@benfred the docs for the ann benchmarks might have been collapsed when you were reviewing. At least I found them hard to find when viewing the changes in Github and I knew what I was looking for.

Does this file, which we're adding to the docs, help alleviate this concern? Or are you wanting us to either duplicate or point right to to the table of datasets in the big-ann-benchmarks?

(And I also just realized I still had a couple todos in there. I'll fix those before we merge this)

@benfred
Copy link
Member

benfred commented Mar 28, 2023

Does this file, which we're adding to the docs, help alleviate this concern? Or are you wanting us to either duplicate or point right to to the table of datasets in the big-ann-benchmarks?

yup, that looks good! I must have missed it there

benfred
benfred approved these changes Mar 28, 2023
View reviewed changes
Copy link
Member

@benfred benfred left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

looks good!

tfeher
tfeher approved these changes Mar 28, 2023
View reviewed changes
Copy link
Contributor

@tfeher tfeher left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks Corey for the updates on this PR. It looks good to me, just two minor comment, that can be addressed in the follow-up as well.

cpp/bench/ann/conf/deep-100M.json Outdated Show resolved Hide resolved
cpp/bench/ann/src/raft/raft_ivf_flat_wrapper.h Outdated Show resolved Hide resolved
@github-actions github-actions bot added the ci label Mar 28, 2023
@cjnolet
Copy link
Member Author

cjnolet commented Mar 28, 2023

/merge

@rapids-bot rapids-bot bot merged commit 22ebc72 into rapidsai:branch-23.04 Mar 28, 2023
@ChuckHastings ChuckHastings mentioned this pull request Mar 29, 2023
Merged
rapids-bot bot pushed a commit to rapidsai/cugraph that referenced this pull request Mar 29, 2023
@naimnv
Replace CUDA_TRY with RAFT_CUDA_TRY (#3389)
b45fb8d 
Replace CUDA_TRY with RAFT_CUDA_TRY in three files.

Also replaced CHECK_CUDA with RAFT_CHECK_CUDA

rapidsai/raft#1304 removed the old macros, we still had a few references.


Currently blocked by rapidsai/cugraph-ops#448... once that is merged we should be able to finish building and merge this.

Authors:
  - Naim (https://github.com/naimnv)
  - Chuck Hastings (https://github.com/ChuckHastings)

Approvers:
  - Seunghwa Kang (https://github.com/seunghwak)
  - Chuck Hastings (https://github.com/ChuckHastings)

URL: #3389
@ahendriksen
Copy link
Contributor

Was it intended behavior that the benchmarks are not built anymore when you build with ./build.sh bench-prims? This currently sets -DBUILD_PRIMS_BENCH, but in the bench CmakeLists.txt, it checks for if(BUILD_BENCH).

@ahendriksen ahendriksen mentioned this pull request Mar 31, 2023
Merged
@ahendriksen
Copy link
Contributor

In any case, I have attempted a fix in #1389.

rapids-bot bot pushed a commit that referenced this pull request Mar 31, 2023
@ahendriksen
Fix primitives benchmarks (#1389)
09720d8 
Due to some typos in #1304, the benchmarks are not built any more. 

I have fixed this in this PR. I think it is important that this PR goes into 23.04, otherwise we will not have benchmarks for two months in the stable release.

Authors:
  - Allard Hendriksen (https://github.com/ahendriksen)

Approvers:
  - Corey J. Nolet (https://github.com/cjnolet)

URL: #1389
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@tfeher tfeher tfeher approved these changes

@raydouglass raydouglass raydouglass approved these changes

@benfred benfred benfred approved these changes

Assignees

@cjnolet cjnolet

Labels
ci CMake cpp improvement Improvement / enhancement to an existing function non-breaking Non-breaking change
Projects
VS/ML/DM Primitives Release Board
Archived in project
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

[FEA] Integrate cuda ANN benchmarks
5 participants
@cjnolet @benfred @ahendriksen @raydouglass @tfeher