Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

End-to-end vector search tutorial in docs #1776

Merged
merged 16 commits into from
Oct 13, 2023
Merged

End-to-end vector search tutorial in docs #1776

merged 16 commits into from
Oct 13, 2023

Conversation

cjnolet
Copy link
Member

@cjnolet cjnolet commented Aug 26, 2023 •
edited
Loading

Closes #1745

@cjnolet cjnolet added documentation Improvements or additions to documentation non-breaking Non-breaking change Vector Search labels Aug 26, 2023
@cjnolet cjnolet self-assigned this Aug 26, 2023
@cjnolet cjnolet added doc Documentation and removed Vector Search documentation Improvements or additions to documentation labels Aug 26, 2023
@cjnolet
Copy link
Member Author

cjnolet commented Aug 26, 2023 •
edited
Loading

@wphicks it would be great if you have cycles to add some verbiage and examples around raft::device_resource_manager (and raft::mdbuffer) to this tutorial as well. Please also feel free to push directly to my branch. The goal of this tutorial is to cover the basics end to end so someone who's not a CUDA developer can understand the core principals enough to know how to invoke our APIs. Further, they should be able to integrate our vector search APIs end-to-end.

@wphicks wphicks self-assigned this Aug 29, 2023
@copy-pr-bot
Copy link

copy-pr-bot bot commented Aug 30, 2023

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

@wphicks
Copy link
Contributor

wphicks commented Aug 31, 2023

Just pushed some text on device_resources_manager. Feel free to edit directly as desired.

@wphicks wphicks removed their assignment Oct 2, 2023
@cjnolet cjnolet changed the base branch from branch-23.10 to branch-23.12 October 5, 2023 23:45
@cjnolet
Copy link
Member Author

cjnolet commented Oct 12, 2023

/ok to test

benfred
benfred approved these changes Oct 12, 2023
View reviewed changes
Copy link
Member

@benfred benfred left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I really like this tutorial - I think its very clear and well written, and will be a great introduction on how to use raft for ANN.

Some minor comments:

  • There is a great introduction on using RAFT and device_resources in the cuda development section - but we don't tie it into the example code using the BF and CAGRA example code blocks later. I think we should add a res.sync_stream() to the BF/CAGRA example code blocks , and maybe also a snippet that copies the results back to host memory
  • The section on using a stream_pool doesn't need to be here (though this is super useful, and we should keep for other parts of the documentation somewhere). Consumers of our vector search indices don't typically need a stream_pool, so this information might be best put elsewhere

@cjnolet
Copy link
Member Author

cjnolet commented Oct 13, 2023

/ok to test

@cjnolet
Copy link
Member Author

cjnolet commented Oct 13, 2023

/merge

@rapids-bot rapids-bot bot merged commit cf08558 into rapidsai:branch-23.12 Oct 13, 2023
divyegala pushed a commit to divyegala/raft that referenced this pull request Oct 17, 2023
@cjnolet @divyegala
End-to-end vector search tutorial in docs (rapidsai#1776)
985d3ef 
Closes rapidsai#1745

Authors:
  - Corey J. Nolet (https://github.com/cjnolet)
  - William Hicks (https://github.com/wphicks)
  - Divye Gala (https://github.com/divyegala)

Approvers:
  - Ben Frederickson (https://github.com/benfred)

URL: rapidsai#1776
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@benfred benfred benfred approved these changes

Assignees

@cjnolet cjnolet

Labels
doc Documentation non-breaking Non-breaking change
Projects
VS/ML/DM Primitives Release Board
Archived in project
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

[DOC] Tutorial for end-to-end vector search in C++
4 participants
@cjnolet @wphicks @benfred @divyegala