-
Notifications
You must be signed in to change notification settings - Fork 197
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Remove NumPy <2 pin #2414
Remove NumPy <2 pin #2414
Conversation
Now that CuPy 13.3.0 is out, restarting CI and marking this ready for review |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Changes looked good to me.
I double-checked that this project doesn't have a build dependency on numpy
, so no need to add any pins for build time.
Looked around at all the other uses of numpy
and I think this got all of them. Assuming CI passes, I think this is good to go.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks Sebastian and James! 🙏
The RAFT-Dask package doesn't actually `import numpy` anywhere inside. This may have been different in the past. However today RAFT-Dask does not use NumPy. Further pylibraft, which does use NumPy and has a dependency on it, is already a dependency of RAFT-Dask. As a result, NumPy is pulled into the environment anyways. The conda package for raft-dask already doesn't have NumPy and has been working ok for a while. So line up the wheels to match this behavior.
"numpy>=1.23,<2.0a0", | ||
"pylibraft==24.10.*,>=0.0.0a0", |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
AFAICT numpy
is not used in raft-dask
$ git grep numpy -- python/raft-dask/ | wc -l
0
Also the Conda package doesn't include it
raft/conda/recipes/raft-dask/meta.yaml
Lines 61 to 76 in f1cc0fb
run: | |
{% if cuda_major == "11" %} | |
- cudatoolkit | |
{% else %} | |
- cuda-cudart | |
{% endif %} | |
- {{ pin_compatible('cuda-version', max_pin='x', min_pin='x') }} | |
- dask-cuda ={{ minor_version }} | |
- rapids-dask-dependency ={{ minor_version }} | |
- joblib >=0.11 | |
- nccl >=2.9.9 | |
- pylibraft {{ version }} | |
- python x.x | |
- rmm ={{ minor_version }} | |
- ucx-py {{ ucx_py_version }} | |
- distributed-ucxx {{ ucxx_version }} |
The origin of this numpy
line in raft-dask
's wheels traces back to the addition of wheels in PR ( #1013 ). Though not seeing any discussion about why numpy
was added there. Even at the time conda raft-dask
packages did not depend on numpy
. Am guessing this was just copied into the different RAFT wheel packages. So it is likely just an oversight that this hasn't already been cleaned up
Though pylibraft
, used by raft-dask
, does use numpy
$ git grep numpy -- python/pylibraft/ | wc -l
52
Also pylibraft
already includes numpy
as a dependency
raft/conda/recipes/pylibraft/meta.yaml
Line 68 in a773fa5
- numpy >=1.23,<3.0a0 |
raft/python/pylibraft/pyproject.toml
Line 35 in a773fa5
"numpy>=1.23,<3.0a0", |
So we can be confident numpy
will be in the environment raft-dask
is installed into even though it appears raft-dask
itself doesn't need it
Given all of this, took this opportunity to clean this up and align our packages on the handling of this dependency
There was a network error in this CI job. Will wait for running CI jobs to complete and then restart failed jobs Edit: Looks like that fixed it |
/merge |
Thanks Sebastian and James! 🙏 |
This PR removes the NumPy<2 pin which is expected to work for
RAPIDS projects once CuPy 13.3.0 is released (CuPy 13.2.0 had
some issues preventing the use with NumPy 2).