Pin to Python 3.11. #388

bdice · 2024-09-05T17:39:26Z

Pin devcontainers to Python 3.11.

During the migration to Python 3.12, we are getting py=3.12 as a matrix selector for rapids-dependency-file-generator, which in turn selects python=3.12 for repositories that have added support. However, for repositories that have not migrated yet, we get the fallback matrix python>=3.10,<3.12 which conflicts with the other pinning. To avoid this, we can pin devcontainers to use PYTHON_VERSION="3.11" which will be passed through to the logic here:

devcontainers/features/src/rapids-build-utils/opt/rapids-build-utils/bin/make-conda-dependencies.sh

Lines 62 to 73 in f9155e8

    
           local python_version="${PYTHON_VERSION:-$(python3 --version 2>&1 | cut -d' ' -f2)}"; 
        
           python_version="$(cut -d'.' -f3 --complement <<< "${python_version}")"; 
        
           local -a _matrix_selectors=( 
        
               arch="$(uname -m)" 
        
               cuda="${cuda_version}" 
        
               py="${python_version}" 
        
           ); 
        
           # add extra arguments (if there are conflicts, e.g. 'py=3.10;py=3.11', it's fine... the last one will win) 
        
           test ${#matrix_entry[@]} -gt 0 && _matrix_selectors+=("${matrix_entry[@]}"); 
        
           local -r matrix_selectors=$(IFS=";"; echo "${_matrix_selectors[*]}")

jameslamb

Thanks for looking into this!

I hadn't considered the way that Python version detection here would have been silently hitting the fallback matrices until at least one repo had a match.

I've updated rapidsai/build-planning#40 with checklist items so when we go to add Python 3.13 in the future, we remember to do this again.

Contributes to rapidsai/build-planning#40 #388 pinned to Python 3.11 for building the mambaforge images here, to prevent issues in the situation where some but not all RAPIDS repos had added Python 3.12 support. This proposes moving that pin to Python 3.12. Keeping this hard-coded will prevent similar issues the next time we go to add a Python version (Python 3.13) across RAPIDS.

Pin to Python 3.11.

9e27957

bdice requested a review from a team as a code owner September 5, 2024 17:39

bdice requested review from jameslamb and removed request for a team September 5, 2024 17:39

trxcllnt approved these changes Sep 5, 2024

View reviewed changes

jameslamb approved these changes Sep 5, 2024

View reviewed changes

trxcllnt merged commit 68807e3 into rapidsai:branch-24.10 Sep 5, 2024
211 of 213 checks passed

This was referenced Sep 10, 2024

Add support for Python 3.12 rapidsai/build-planning#40

Closed

pin mambaforge images to Python 3.12 #391

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Pin to Python 3.11. #388

Pin to Python 3.11. #388

bdice commented Sep 5, 2024

jameslamb left a comment

	local python_version="${PYTHON_VERSION:-$(python3 --version 2>&1 \| cut -d' ' -f2)}";
	python_version="$(cut -d'.' -f3 --complement <<< "${python_version}")";

	local -a _matrix_selectors=(
	arch="$(uname -m)"
	cuda="${cuda_version}"
	py="${python_version}"
	);

	# add extra arguments (if there are conflicts, e.g. 'py=3.10;py=3.11', it's fine... the last one will win)
	test ${#matrix_entry[@]} -gt 0 && _matrix_selectors+=("${matrix_entry[@]}");
	local -r matrix_selectors=$(IFS=";"; echo "${_matrix_selectors[*]}")

Pin to Python 3.11. #388

Pin to Python 3.11. #388

Conversation

bdice commented Sep 5, 2024

jameslamb left a comment

Choose a reason for hiding this comment