-
Notifications
You must be signed in to change notification settings - Fork 26
Description
Hello,
I'm encountering an issue when using pydata/sparse. The problem is coming from Finch returning Julia-native dtypes like Julia: Float64, which are not recognized by NumPy's np.dtype(...).
The error I am getting is TypeError: Cannot interpret 'Julia: Float64' as a data type. It happens in the normalize_chunks(...) function, where this block is called:
if dtype and not isinstance(dtype, np.dtype):
dtype = np.dtype(dtype) # Fails here
To reproduce the error:
import os
os.environ["SPARSE_BACKEND"] = "Finch"
import cubed
import numpy as np
import sparse
os.environ["CUBED_BACKEND_ARRAY_API_MODULE"] = "sparse"
sa_finch = sparse.asarray(np.eye(4), format="coo")
print(type(sa_finch))
wrapped = cubed.asarray(sa_finch) # fails here
This seems to be caused by a mismatch between assumptions in NumPy-oriented code and behavior in newer array API–compatible backends. Finch, for example, returns dtypes like Julia: Float64, which aren’t compatible with np.dtype(...). In normalize_chunks(...), there’s currently an implicit expectation that all dtype values will work with np.dtype(...), which may not hold for backends like Finch or other non-NumPy implementations.
Tagging @ilan-gold as discussed.