Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add cuGraph devcontainers #3838

Merged
merged 66 commits into from
Sep 27, 2023
Merged

Conversation

trxcllnt
Copy link
Collaborator

Description

This PR adds some devcontainers to help simplify building the cuGraph C++ and Python libraries.

It also adds an optional job to the pr.yaml to build the cuGraph libs in each devcontainer, so the build caches are populated for devs by CI.

A devcontainer can be launched by clicking the "Reopen in Container" button that VSCode shows when opening the repo (or by using the "Rebuild and Reopen in Container" command from the command palette):
image

Clicking this button will cause VSCode to prompt the user to select one of these devcontainer variants:
image

On startup, the devcontainer creates or updates the conda/pip environment using cugraph/dependencies.yaml. The envs/package caches are cached on the host via volume mounts, which are described in more detail in .devcontainer/README.md.

The container includes convenience functions to clean, configure, and build the various cuGraph components:

$ clean-cugraph-cpp # only cleans the C++ build dir
$ clean-cugraph-python # only cleans the Python build dir
$ clean-cugraph # cleans both C++ and Python build dirs

$ configure-cugraph-cpp # only configures cugraph C++ lib

$ build-cugraph-cpp # only builds cugraph C++ lib
$ build-cugraph-python # only builds cugraph Python lib
$ build-cugraph # builds both C++ and Python libs
  • The C++ build script is a small wrapper around cmake -S ~/cugraph/cpp -B ~/cugraph/cpp/build and cmake --build ~/cugraph/cpp/build
  • The Python build script is a small wrapper around pip install --editable ~/cugraph/cpp

Unlike build.sh, these convenience scripts don't install the libraries after building them. Instead, they automatically inject the correct arguments to build the C++ libraries from source and use their build dirs as package roots:

$ cmake -S ~/cugraph/cpp -B ~/cugraph/cpp/build
$ CMAKE_ARGS="-Dcugraph_ROOT=~/cugraph/cpp/build" \ # <-- this argument is automatic
  pip install -e ~/cugraph/cpp

Checklist

  • I am familiar with the Contributing Guidelines.
  • New or existing tests cover these changes.
  • The documentation is up to date with these changes.

trxcllnt added 30 commits July 17, 2023 10:11
@trxcllnt
Copy link
Collaborator Author

/ok to test

@trxcllnt
Copy link
Collaborator Author

/ok to test

@trxcllnt trxcllnt removed the DO NOT MERGE Hold off on merging; see PR for details label Sep 21, 2023
@trxcllnt
Copy link
Collaborator Author

/ok to test

1 similar comment
@trxcllnt
Copy link
Collaborator Author

/ok to test

@trxcllnt
Copy link
Collaborator Author

/ok to test

@trxcllnt
Copy link
Collaborator Author

/ok to test

@trxcllnt
Copy link
Collaborator Author

/ok to test

@BradReesWork BradReesWork added this to the 23.10 milestone Sep 25, 2023
@trxcllnt
Copy link
Collaborator Author

/ok to test

rapids-bot bot pushed a commit to rapidsai/cuspatial that referenced this pull request Sep 25, 2023
Removes the unified and isolated devcontainers in favor of just the single one. The unified devcontainers are [still available](https://github.com/rapidsai/devcontainers/blob/d970ac17a894f5d1047fcfd17cffa6d14fe66ec8/.devcontainer/cuda12.0-conda/devcontainer.json#L23-L31) in the [`rapidsai/devcontainers`](https://github.com/rapidsai/devcontainers) repository.


Adds an optional job to the `pr.yaml` to [build the cuGraph libs in each devcontainer](https://github.com/trxcllnt/cuspatial/blob/fea/devcontainers/.github/workflows/pr.yaml#L106-L111), so the build caches are populated for devs by CI.

Other PRs:

* rapidsai/rmm#1328
* rapidsai/kvikio#273
* rapidsai/cudf#14015
* rapidsai/raft#1791
* rapidsai/cumlprims_mg#149
* rapidsai/cuml#5568
* rapidsai/cugraph-ops#538
* rapidsai/cugraph#3838

Authors:
  - Paul Taylor (https://github.com/trxcllnt)
  - H. Thomson Comer (https://github.com/thomcom)

Approvers:
  - Mark Harris (https://github.com/harrism)
  - Jake Awe (https://github.com/AyodeAwe)

URL: #1263
@trxcllnt
Copy link
Collaborator Author

/ok to test

@rlratzel
Copy link
Contributor

Hi @trxcllnt - I vaguely remember someone mentioning that these images could be built outside of VSCode and the corresponding containers can be used by non-VSCode users. Is that correct, and if so, are there instructions on how to do that?

I still have not installed VSCode yet and did not see how to try these changes in my dev environment.

@trxcllnt
Copy link
Collaborator Author

@rlratzel yes, you can use the devcontainers CLI

@trxcllnt
Copy link
Collaborator Author

/ok to test

@BradReesWork
Copy link
Member

/merge

@rapids-bot rapids-bot bot merged commit 8b87915 into rapidsai:branch-23.10 Sep 27, 2023
69 checks passed
@raydouglass raydouglass mentioned this pull request Oct 2, 2023
rapids-bot bot pushed a commit that referenced this pull request Oct 17, 2023
A couple PRs were merged after `branch-23.12` was created and contained RAPIDS versions that need to be updated in `branch-23.12`.

Ref:
- #3838
- #3852

Authors:
  - Ray Douglass (https://github.com/raydouglass)
  - Rick Ratzel (https://github.com/rlratzel)

Approvers:
  - AJ Schmidt (https://github.com/ajschmidt8)
  - Brad Rees (https://github.com/BradReesWork)

URL: #3905
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
improvement Improvement / enhancement to an existing function non-breaking Non-breaking change
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants