Seeing build issues like this when building pylibraft:

    Error compiling Cython file:
    ------------------------------------------------------------
    ...
  
        shape = (cai.shape[0], cai.shape[1] if len(cai.shape) == 2 else 1)
  
        return make_device_matrix_view[int64_t, int64_t, row_major](
  
            <int64_t*><uintptr_t>cai.data, shape[0], shape[1])
 
    cdef device_matrix_view[const_float, int64_t, row_major] \ 
    ^
 
    ------------------------------------------------------------
  
    /home/coder/raft/python/pylibraft/pylibraft/common/mdspan.pyx:203:0: Function signature does not match previous declaration

(build link)

Note that this is only happening in pip devcontainer builds, not in the other wheel builds of pylibraft here.

Pasting my message from Slack:

My guess is that that this is happening because the pip devcontainer is using the Cython 3.1 alpha, while the normal wheel builds are not. The alpha was released five days ago. I'm guessing there are some bugs in there that need sorting out. The issue is that (if you recall) our devcontainer builds install with --pre because of rapidsai/build-planning#85 (otherwise we could specify alpha versions explicitly for dask/distributed and not need the global --pre)

We should probably report a bug upstream if we can root cause this, but in the meantime raft may just have to introduce an upper bound. I wouldn't bother doing this globally in other RAPIDS packages unless we start seeing failures more broadly though; likely this is some edge case due to raft's complex templating, and it may even be that raft's Cython has a bug that the newer Cython version is catching while the old one didn't due to less strict parsing. I encountered that with cuml before and fixed a real bug.

Thanks @vyasr ! It looks like adding the ceiling on Cython worked 🎉

Put up #2491 to track the work of relaxing that ceiling in the future.

Some CI jobs here are failing for what look like temporary issues, but I don't want to stop the in-progress C++ test job to restart everything, because those take several hours to run 😬

I'll keep checking back and hopefully we can get a clean CI run and then merge this.

/merge