add devcontainers

rapidsai · rapids-bot · Sep 13, 2023 · Jul 26, 2023 · Jul 26, 2023 · Jul 26, 2023
commit c3fee80cf04de75afc0c3ee03bd44c2b66b3c387
diff --git a/.devcontainer/Dockerfile b/.devcontainer/Dockerfile
@@ -0,0 +1,33 @@
+# syntax=docker/dockerfile:1.5
+
+ARG CUDA=12.0
+ARG LLVM=16
+ARG RAPIDS=23.08
+ARG DISTRO=ubuntu22.04
+ARG REPO=rapidsai/devcontainers
+
+ARG PYTHON_PACKAGE_MANAGER=conda
+
+FROM ${REPO}:${RAPIDS}-cpp-llvm${LLVM}-cuda${CUDA}-${DISTRO} as pip-base
+
+FROM ${REPO}:${RAPIDS}-cpp-mambaforge-${DISTRO} as conda-base
+
+COPY --from=pip-base /etc/skel/.config/clangd/config.yaml /etc/skel/.config/clangd/config.yaml
+
+FROM ${PYTHON_PACKAGE_MANAGER}-base
+
+ARG CUDA
+ENV CUDAARCHS="RAPIDS"
+ENV CUDA_VERSION="${CUDA_VERSION:-${CUDA}}"
+
+ARG PYTHON_PACKAGE_MANAGER
+ENV PYTHON_PACKAGE_MANAGER="${PYTHON_PACKAGE_MANAGER}"
+
+ENV PYTHONSAFEPATH="1"
+ENV PYTHONUNBUFFERED="1"
+ENV PYTHONDONTWRITEBYTECODE="1"
+
+ENV SCCACHE_REGION="us-east-2"
+ENV SCCACHE_BUCKET="rapids-sccache-devs"
+ENV VAULT_HOST="https://vault.ops.k8s.rapids.ai"
+ENV HISTFILE="/home/coder/.cache/._bash_history"
diff --git a/.devcontainer/README.md b/.devcontainer/README.md
@@ -0,0 +1,103 @@
+# KvikIO Development Containers
+
+This directory contains [devcontainer configurations](https://containers.dev/implementors/json_reference/) for using VSCode to [develop in a container](https://code.visualstudio.com/docs/devcontainers/containers) via the `Remote Containers` [extension](https://marketplace.visualstudio.com/items?itemName=ms-vscode-remote.remote-containers) or [GitHub Codespaces](https://github.com/codespaces).
+
+This container is a turnkey development environment for building and testing the KvikIO C++ and Python libraries.
+
+## Table of Contents
+
+* [Prerequisites](#prerequisites)
+* [Host bind mounts](#host-bind-mounts)
+* [Launch a Dev Container](#launch-a-dev-container)
+  * [via VSCode](#via-vscode)
+  * [via `launch.sh`](#via-launchsh)
+    * [Single mode](#single-mode)
+    * [Unified mode](#unified-mode)
+    * [Isolated mode](#isolated-mode)
+
+## Prerequisites
+
+* [VSCode](https://code.visualstudio.com/download)
+* [VSCode Remote Containers extension](https://marketplace.visualstudio.com/items?itemName=ms-vscode-remote.remote-containers)
+
+## Host bind mounts
+
+By default, the following directories are bind-mounted into the devcontainer:
+
+* `${repo}:/home/coder/kvikio`
+* `${repo}/../.aws:/home/coder/.aws`
+* `${repo}/../.local:/home/coder/.local`
+* `${repo}/../.cache:/home/coder/.cache`
+* `${repo}/../.conda:/home/coder/.conda`
+* `${repo}/../.config:/home/coder/.config`
+
+This ensures caches, configurations, dependencies, and your commits are persisted on the host across container runs.
+
+## Launch a Dev Container
+
+A devcontainer can be launched directly by VSCode, or via a custom `launch.sh` shell script.
+
+### via VSCode
+
+To launch a devcontainer from VSCode, open the KvikIO repo and select the "Reopen in Container" button in the bottom right:<br/><img src="https://user-images.githubusercontent.com/178183/221771999-97ab29d5-e718-4e5f-b32f-2cdd51bba25c.png"/>
+
+Alternatively, open the VSCode command palette (typically `cmd/ctrl + shift + P`) and run the "Rebuild and Reopen in Container" command.
+
+### via `launch.sh`
+
+Use the `.devcontainer/launch.sh` script to start a new instance of the development container and launch a fresh VSCode window connected to it.
+
+VSCode extends its [single-window-per-folder](https://github.com/microsoft/vscode/issues/2686) process model to devcontainers. Opening the same devcontainer in separate windows doesn't create two separate containers -- instead you have two VSCode windows each connected to the same running container.
+
+`launch.sh` takes two arguments, a `mode` and a `package manager`.
+
+* The `mode` argument determines how the devcontainer interacts with the files on the host.
+* The `package manager` argument can be either `conda`, or `pip`. This determines whether the devcontainer uses `conda` or `pip` to install the dependencies (the default is `conda`). `pip` is experimental/not working for normal dev, and is currently meant to aid in pip packaging work.
+
+#### Single mode
+
+`.devcontainer/launch.sh single` launches the devcontainer with the [default bind mounts](#host-bind-mounts). KvikIO and cuDF are installed via the package manager.
+
+Example:
+```bash
+# Launch a devcontainer that only mounts kvikio and installs dependencies via conda
+$ .devcontainer/launch.sh single conda
+
+# or installs dependencies via pip
+$ .devcontainer/launch.sh single pip
+```
+
+#### Unified mode
+
+`.devcontainer/launch.sh unified` launches the devcontainer with the [default bind mounts](#host-bind-mounts), as well as additional `kvikio` and `cudf` bind mounts (assumes KvikIO and cuDF are siblings to the kvikio repository):
+
+* `${repo}/../.kvikio:/home/coder/kvikio`
+* `${repo}/../.cudf:/home/coder/cudf`
+
+In this mode, KvikIO and cuDF will not be installed, but the devcontainer will install the dependencies necessary to build all three.
+
+Example:
+```bash
+# Launch a devcontainer that mounts kvikio, cudf, and kvikio from the host and installs dependencies via conda
+$ .devcontainer/launch.sh unified conda
+
+# or installs dependencies via pip
+$ .devcontainer/launch.sh unified pip
+```
+
+#### Isolated mode
+
+`.devcontainer/launch.sh isolated` launches the devcontainer without the deps/repo bind mounts, and instead contains a unique copy of the `kvikio` source in the container's file system.
+
+Use this mode to launch multiple isolated development containers that can be checked out to separate branches of `kvikio`.
+
+**Be sure to push any commits you want to persist. Once this container is removed, any unpushed changes will be lost!**
+
+Examples:
+```bash
+# Launch a devcontainer that is isolated from changes on the host and installs dependencies via conda
+$ .devcontainer/launch.sh isolated conda
+
+# or installs dependencies via pip
+$ .devcontainer/launch.sh isolated pip
+```
diff --git a/.devcontainer/conda/isolated/.devcontainer/devcontainer.json b/.devcontainer/conda/isolated/.devcontainer/devcontainer.json
@@ -0,0 +1,92 @@
+{
+  "shutdownAction": "stopContainer",
+
+  "build": {
+    "context": "${localWorkspaceFolder}/.devcontainer",
+    "dockerfile": "${localWorkspaceFolder}/.devcontainer/Dockerfile",
+    "args": {
+      "CUDA": "12.0",
+      "LLVM": "16",
+      "PYTHON_PACKAGE_MANAGER": "conda"
+    }
+  },
+  "hostRequirements": {
+    "gpu": true
+  },
+
+  "features": {
+    "ghcr.io/rapidsai/devcontainers/features/rapids-build-utils:23.8": {}
+  },
+
+  "overrideFeatureInstallOrder": [
+    "ghcr.io/rapidsai/devcontainers/features/rapids-build-utils"
+  ],
+
+  "initializeCommand": [
+    "/bin/bash", "-c", "mkdir -m 0755 -p ${localWorkspaceFolder}/../.{aws,cache,config,conda/pkgs}"
+  ],
+
+  "updateContentCommand": [
+    "/bin/bash", "-c", "mkdir -m 0755 -p ~/.config/clangd && cp -n /etc/skel/.config/clangd/config.yaml ~/.config/clangd/config.yaml && cp -ar /workspaces/${localWorkspaceFolderBasename} /home/coder/${localWorkspaceFolderBasename} && rapids-make-vscode-workspace --update"
+  ],
+
+  "containerEnv": {
+    "DEFAULT_CONDA_ENV": "rapids"
+  },
+
+  "workspaceFolder": "/home/coder",
+  "workspaceMount": "source=${localWorkspaceFolder},target=/workspaces/${localWorkspaceFolderBasename},type=bind,consistency=consistent",
+  "mounts": [
+    "source=${localWorkspaceFolder}/../.aws,target=/home/coder/.aws,type=bind,consistency=consistent",
+    "source=${localWorkspaceFolder}/../.cache,target=/home/coder/.cache,type=bind,consistency=consistent",
+    "source=${localWorkspaceFolder}/../.config,target=/home/coder/.config,type=bind,consistency=consistent",
+    "source=${localWorkspaceFolder}/../.conda/pkgs,target=/home/coder/.conda/pkgs,type=bind,consistency=consistent"
+  ],
+
+  "customizations": {
+    "vscode": {
+      "extensions": [
+        "llvm-vs-code-extensions.vscode-clangd",
+        "mutantdino.resourcemonitor",
+        "ms-vscode.cpptools",
+        "nvidia.nsight-vscode-edition",
+        "seaube.clangformat",
+        "tamasfe.even-better-toml"
+      ],
+      "settings": {
+        "C_Cpp.vcpkg.enabled": false,
+        "C_Cpp.formatting": "disabled",
+        "C_Cpp.autocomplete": "disabled",
+        "C_Cpp.errorSquiggles": "disabled",
+        "C_Cpp.intelliSenseEngine": "disabled",
+        "C_Cpp.configurationWarnings": "disabled",
+        "C_Cpp.autoAddFileAssociations": false,
+        "clang-format.fallbackStyle": "none",
+        "files.trimFinalNewlines": true,
+        "files.insertFinalNewline": true,
+        "files.trimTrailingWhitespace": true,
+        "files.associations": {
+          "*.cu": "cuda-cpp",
+          "*.cuh": "cuda-cpp",
+          "**/libcudacxx/include/**/*": "cpp",
+          "**/libcudacxx-src/include/**/*": "cpp"
+        },
+        "files.watcherExclude": {
+          "**/build/**": true,
+          "**/_skbuild/**": true,
+          "**/target/**": true
+        },
+        "python.linting.flake8Enabled": true,
+        "[c]": {
+          "editor.defaultFormatter": "seaube.clangformat"
+        },
+        "[cpp]": {
+          "editor.defaultFormatter": "seaube.clangformat"
+        },
+        "[cuda-cpp]": {
+          "editor.defaultFormatter": "seaube.clangformat"
+        }
+      }
+    }
+  }
+}
diff --git a/.devcontainer/conda/single/.devcontainer/devcontainer.json b/.devcontainer/conda/single/.devcontainer/devcontainer.json
@@ -0,0 +1,95 @@
+{
+  "shutdownAction": "stopContainer",
+
+  "build": {
+    "context": "${localWorkspaceFolder}/.devcontainer",
+    "dockerfile": "${localWorkspaceFolder}/.devcontainer/Dockerfile",
+    "args": {
+      "CUDA": "12.0",
+      "LLVM": "16",
+      "PYTHON_PACKAGE_MANAGER": "conda"
+    }
+  },
+  "hostRequirements": {
+    "gpu": true
+  },
+
+  "features": {
+    "ghcr.io/rapidsai/devcontainers/features/rapids-build-utils:23.8": {}
+  },
+
+  "overrideFeatureInstallOrder": [
+    "ghcr.io/rapidsai/devcontainers/features/rapids-build-utils"
+  ],
+
+  "initializeCommand": [
+    "/bin/bash", "-c", "mkdir -m 0755 -p ${localWorkspaceFolder}/../.{aws,cache,config,conda/pkgs,conda/${localWorkspaceFolderBasename}/single}"
+  ],
+
+  "updateContentCommand": [
+    "/bin/bash",
+    "-c",
+    "mkdir -m 0755 -p ~/.config/clangd && cp -n /etc/skel/.config/clangd/config.yaml ~/.config/clangd/config.yaml"
+  ],
+
+  "containerEnv": {
+    "DEFAULT_CONDA_ENV": "rapids"
+  },
+
+  "workspaceFolder": "/home/coder",
+  "workspaceMount": "source=${localWorkspaceFolder},target=/home/coder/${localWorkspaceFolderBasename},type=bind,consistency=consistent",
+  "mounts": [
+    "source=${localWorkspaceFolder}/../.aws,target=/home/coder/.aws,type=bind,consistency=consistent",
+    "source=${localWorkspaceFolder}/../.cache,target=/home/coder/.cache,type=bind,consistency=consistent",
+    "source=${localWorkspaceFolder}/../.config,target=/home/coder/.config,type=bind,consistency=consistent",
+    "source=${localWorkspaceFolder}/../.conda/pkgs,target=/home/coder/.conda/pkgs,type=bind,consistency=consistent",
+    "source=${localWorkspaceFolder}/../.conda/${localWorkspaceFolderBasename}/single,target=/home/coder/.conda/envs,type=bind,consistency=consistent"
+  ],
+
+  "customizations": {
+    "vscode": {
+      "extensions": [
+        "llvm-vs-code-extensions.vscode-clangd",
+        "mutantdino.resourcemonitor",
+        "ms-vscode.cpptools",
+        "nvidia.nsight-vscode-edition",
+        "seaube.clangformat",
+        "tamasfe.even-better-toml"
+      ],
+      "settings": {
+        "C_Cpp.vcpkg.enabled": false,
+        "C_Cpp.formatting": "disabled",
+        "C_Cpp.autocomplete": "disabled",
+        "C_Cpp.errorSquiggles": "disabled",
+        "C_Cpp.intelliSenseEngine": "disabled",
+        "C_Cpp.configurationWarnings": "disabled",
+        "C_Cpp.autoAddFileAssociations": false,
+        "clang-format.fallbackStyle": "none",
+        "files.trimFinalNewlines": true,
+        "files.insertFinalNewline": true,
+        "files.trimTrailingWhitespace": true,
+        "files.associations": {
+          "*.cu": "cuda-cpp",
+          "*.cuh": "cuda-cpp",
+          "**/libcudacxx/include/**/*": "cpp",
+          "**/libcudacxx-src/include/**/*": "cpp"
+        },
+        "files.watcherExclude": {
+          "**/build/**": true,
+          "**/_skbuild/**": true,
+          "**/target/**": true
+        },
+        "python.linting.flake8Enabled": true,
+        "[c]": {
+          "editor.defaultFormatter": "seaube.clangformat"
+        },
+        "[cpp]": {
+          "editor.defaultFormatter": "seaube.clangformat"
+        },
+        "[cuda-cpp]": {
+          "editor.defaultFormatter": "seaube.clangformat"
+        }
+      }
+    }
+  }
+}