Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ANN bench options to specify CAGRA graph and dataset locations #1896

Merged
merged 91 commits into from
Nov 16, 2023

Conversation

cjnolet
Copy link
Member

@cjnolet cjnolet commented Oct 12, 2023

CAGRA index can be constructed using existing device_mdarrays, in which case just reference to these arrays are stored.
This way allocations are managed outside the index, and we can customize how these allocations are made.

This PR

  • modifies the CAGRA ANN bench wrapper to manage the allocations locally,
  • add options for the json file to specify whether the graph / dataset is allocated in device / host_pinned / host_huge_page memory.

tfeher and others added 30 commits July 22, 2023 00:09
@cjnolet
Copy link
Member Author

cjnolet commented Nov 9, 2023

/ok to test

@cjnolet
Copy link
Member Author

cjnolet commented Nov 14, 2023

/ok to test

@cjnolet
Copy link
Member Author

cjnolet commented Nov 14, 2023

/ok to test

@cjnolet
Copy link
Member Author

cjnolet commented Nov 14, 2023

/ok to test

@tfeher tfeher added the 5 - DO NOT MERGE Hold off on merging; see PR for details label Nov 14, 2023
@tfeher tfeher removed the 5 - DO NOT MERGE Hold off on merging; see PR for details label Nov 15, 2023
@cjnolet
Copy link
Member Author

cjnolet commented Nov 15, 2023

/ok to test

@cjnolet
Copy link
Member Author

cjnolet commented Nov 16, 2023

/ok to test

@cjnolet
Copy link
Member Author

cjnolet commented Nov 16, 2023

/ok to test

@cjnolet
Copy link
Member Author

cjnolet commented Nov 16, 2023

/ok to test

@cjnolet
Copy link
Member Author

cjnolet commented Nov 16, 2023

/ok to test

@cjnolet
Copy link
Member Author

cjnolet commented Nov 16, 2023

/merge

@rapids-bot rapids-bot bot merged commit 0f6ce22 into rapidsai:branch-23.12 Nov 16, 2023
62 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
cpp improvement Improvement / enhancement to an existing function non-breaking Non-breaking change python Vector Search
Projects
Development

Successfully merging this pull request may close these issues.

7 participants