Skip to content

Commit

Permalink
Stop using versioneer to manage versions (#1301)
Browse files Browse the repository at this point in the history
This PR replaces usage of versioneer with hard-coded version numbers in setup.py and init.py. Since raft needs to manage versions across a wide range of file types (CMake, C++, Sphinx and doxygen docs, etc), versioneer cannot be relied on as a single source of truth and therefore does not allow us to single-source our versioning to the Git repo as is intended. Additionally, since the primary means of installing raft is via conda packages (or now, pip packages), information from the package manager tends to be far more informative than the version strings for troubleshooting and debugging purposes. Conversely, the nonstandard version strings that it produces tend to be problematic for other tools, which at best will ignore such versions but at worst will simply fail.

This PR also replaces usage of an environment variable to set the package name for wheels in setup.py, instead moving the renaming logic into the same sed script used to update package versions. This change makes setup.py essentially static, paving the way for migration to pyproject.toml.

Authors:
  - Vyas Ramasubramani (https://github.com/vyasr)

Approvers:
  - AJ Schmidt (https://github.com/ajschmidt8)
  - Corey J. Nolet (https://github.com/cjnolet)
  - Ben Frederickson (https://github.com/benfred)

URL: #1301
  • Loading branch information
vyasr authored Feb 28, 2023
1 parent 90c8da1 commit 754ebc3
Show file tree
Hide file tree
Showing 18 changed files with 55 additions and 5,065 deletions.
2 changes: 2 additions & 0 deletions .github/workflows/build.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -74,6 +74,7 @@ jobs:
package-name: pylibraft
package-dir: python/pylibraft
skbuild-configure-options: "-DRAFT_BUILD_WHEELS=ON -DDETECT_CONDA_ENV=OFF -DFIND_RAFT_CPP=OFF"
uses-setup-env-vars: false
wheel-publish-pylibraft:
needs: wheel-build-pylibraft
secrets: inherit
Expand All @@ -96,6 +97,7 @@ jobs:
package-name: raft_dask
package-dir: python/raft-dask
skbuild-configure-options: "-DRAFT_BUILD_WHEELS=ON -DDETECT_CONDA_ENV=OFF -DFIND_RAFT_CPP=OFF"
uses-setup-env-vars: false
wheel-publish-raft-dask:
needs: wheel-build-raft-dask
secrets: inherit
Expand Down
2 changes: 2 additions & 0 deletions .github/workflows/pr.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -73,6 +73,7 @@ jobs:
package-name: pylibraft
package-dir: python/pylibraft
skbuild-configure-options: "-DRAFT_BUILD_WHEELS=ON -DDETECT_CONDA_ENV=OFF -DFIND_RAFT_CPP=OFF"
uses-setup-env-vars: false
wheel-tests-pylibraft:
needs: wheel-build-pylibraft
secrets: inherit
Expand All @@ -95,6 +96,7 @@ jobs:
package-dir: python/raft-dask
before-wheel: "RAPIDS_PY_WHEEL_NAME=pylibraft_cu11 rapids-download-wheels-from-s3 ./local-wheelhouse"
skbuild-configure-options: "-DRAFT_BUILD_WHEELS=ON -DDETECT_CONDA_ENV=OFF -DFIND_RAFT_CPP=OFF"
uses-setup-env-vars: false
wheel-tests-raft-dask:
needs: wheel-build-raft-dask
secrets: inherit
Expand Down
2 changes: 0 additions & 2 deletions .pre-commit-config.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,6 @@ repos:
files: python/.*
# Explicitly specify the pyproject.toml at the repo root, not per-project.
args: ["--config", "pyproject.toml"]
exclude: .*_version.py,.*versioneer.py
- repo: https://github.com/PyCQA/flake8
rev: 5.0.4
hooks:
Expand All @@ -36,7 +35,6 @@ repos:
"python/pylibraft/pylibraft",
"python/raft-dask/raft_dask"]
pass_filenames: false
exclude: .*_version.py
- repo: https://github.com/PyCQA/pydocstyle
rev: 6.1.1
hooks:
Expand Down
27 changes: 27 additions & 0 deletions ci/release/apply_wheel_modifications.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,27 @@
#!/bin/bash
# Copyright (c) 2023, NVIDIA CORPORATION.
#
# Usage: bash apply_wheel_modifications.sh <new_version> <cuda_suffix>

VERSION=${1}
CUDA_SUFFIX=${2}

# __init__.py versions
sed -i "s/__version__ = .*/__version__ = \"${VERSION}\"/g" python/pylibraft/pylibraft/__init__.py
sed -i "s/__version__ = .*/__version__ = \"${VERSION}\"/g" python/raft-dask/raft_dask/__init__.py

# setup.py versions
sed -i "s/version=.*,/version=\"${VERSION}\",/g" python/pylibraft/pylibraft/__init__.py
sed -i "s/version=.*,/version=\"${VERSION}\",/g" python/raft-dask/raft_dask/__init__.py

# pylibraft setup.py cuda suffixes
sed -i "s/name=\"pylibraft\"/name=\"pylibraft${CUDA_SUFFIX}\"/g" python/pylibraft/setup.py
sed -i "s/rmm/rmm${CUDA_SUFFIX}/g" python/pylibraft/setup.py

# raft-dask setup.py cuda suffixes
sed -i "s/name=\"raft-dask\"/name=\"raft-dask${CUDA_SUFFIX}\"/g" python/raft-dask/setup.py
sed -i "s/ucx-py/ucx-py${CUDA_SUFFIX}/g" python/raft-dask/setup.py
sed -i "s/pylibraft/pylibraft${CUDA_SUFFIX}/g" python/raft-dask/setup.py

# Dependency versions in pyproject.toml
sed -i "s/rmm/rmm${CUDA_SUFFIX}/g" python/pylibraft/pyproject.toml
12 changes: 11 additions & 1 deletion ci/release/update-version.sh
Original file line number Diff line number Diff line change
Expand Up @@ -38,6 +38,14 @@ sed_runner 's/'"pylibraft_version .*)"'/'"pylibraft_version ${NEXT_FULL_TAG})"'/
sed_runner 's/'"raft_dask_version .*)"'/'"raft_dask_version ${NEXT_FULL_TAG})"'/g' python/raft-dask/CMakeLists.txt
sed_runner 's/'"branch-.*\/RAPIDS.cmake"'/'"branch-${NEXT_SHORT_TAG}\/RAPIDS.cmake"'/g' fetch_rapids.cmake

# Python __init__.py updates
sed_runner "s/__version__ = .*/__version__ = \"${NEXT_FULL_TAG}\"/g" python/pylibraft/pylibraft/__init__.py
sed_runner "s/__version__ = .*/__version__ = \"${NEXT_FULL_TAG}\"/g" python/raft-dask/raft_dask/__init__.py

# Python setup.py updates
sed_runner "s/version=.*,/version=\"${NEXT_FULL_TAG}\",/g" python/pylibraft/pylibraft/__init__.py
sed_runner "s/version=.*,/version=\"${NEXT_FULL_TAG}\",/g" python/raft-dask/raft_dask/__init__.py

# Docs update
sed_runner 's/version = .*/version = '"'${NEXT_SHORT_TAG}'"'/g' docs/source/conf.py
sed_runner 's/release = .*/release = '"'${NEXT_FULL_TAG}'"'/g' docs/source/conf.py
Expand All @@ -64,11 +72,13 @@ NEXT_UCX_PY_SHORT_TAG_PEP440=$(python -c "from setuptools.extern import packagin

# Wheel builds install intra-RAPIDS dependencies from same release
sed_runner "s/{cuda_suffix}[^\"].*\",/{cuda_suffix}==${NEXT_SHORT_TAG_PEP440}.*\",/g" python/pylibraft/setup.py
sed_runner "s/{cuda_suffix}.*\"\]/{cuda_suffix}==${NEXT_SHORT_TAG_PEP440}.*\"\]/g" python/pylibraft/_custom_build/backend.py
sed_runner "s/dask-cuda==.*\",/dask-cuda==${NEXT_SHORT_TAG_PEP440}.*\",/g" python/raft-dask/setup.py
sed_runner "s/pylibraft{cuda_suffix}.*\",/pylibraft{cuda_suffix}==${NEXT_SHORT_TAG_PEP440}.*\",/g" python/raft-dask/setup.py
sed_runner "s/ucx-py{cuda_suffix}.*\",/ucx-py{cuda_suffix}==${NEXT_UCX_PY_SHORT_TAG_PEP440}.*\",/g" python/raft-dask/setup.py

# Dependency versions in pyproject.toml
sed_runner "s/rmm==.*\",/rmm==${NEXT_SHORT_TAG_PEP440}.*\",/g" python/pylibraft/pyproject.toml

for FILE in .github/workflows/*.yaml; do
sed_runner "/shared-action-workflows/ s/@.*/@branch-${NEXT_SHORT_TAG}/g" "${FILE}"
done
Expand Down
36 changes: 0 additions & 36 deletions python/pylibraft/_custom_build/backend.py

This file was deleted.

6 changes: 1 addition & 5 deletions python/pylibraft/pylibraft/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -13,8 +13,4 @@
# limitations under the License.
#

import pylibraft.config
from pylibraft._version import get_versions

__version__ = get_versions()["version"]
del get_versions
__version__ = "23.04.00"
Loading

0 comments on commit 754ebc3

Please sign in to comment.