Thanks @r-devulap . Let's see if they persist.

It failed again (#23659) and my suspicion was right. It fails on older processors without AVX512. https://github.com/numpy/numpy/actions/runs/4800304956/jobs/8541285721?pr=23659

Confirmed that this is a bug in the Intel SDE. I am able to reproduce this consistently on an older hardware without AVX512. I wonder if we can disable this CI action if we don't detect AVX-512.

minimal reproducer (for my records and to file an SDE bug):

import numpy as np
from numpy.core._multiarray_tests import get_fpu_mode

old = get_fpu_mode()
np.exp(1)
new = get_fpu_mode()
print("old fpu mode =", hex(old))
print("new fpu mode =", hex(new))

$ sde -skx -- python3 test_umath_complex.py
old fpu mode = 0x37f
new fpu mode = 0x77f

SDE version: Intel(R) Software Development Emulator. Version: 9.14.0 external

I'd be happy to skip the test if there were an easy way to detect the hardware. Any idea how long it will take for Intel to respond to the bug report?

@r-devulap what triggers this bug? After a bit of searching, it sounds like this sets round-down bit which would be wrong and could have weird long-term issues.

Or is this triggered by the test itself? In that case skipping it is fine of course.

@seberg its a bug in the Intel SDE and nothing to do with NumPy at all. The bug shows up when the CI job intel_spr_sde_test lands on a machine without AVX-512 (My guess is the emulation of some AVX-512 instruction in the SDE modifies the FPU mode bits and doesn't reset it).

@charris Looking at the previous release of SDE, looks like they have a new release out every 6 months roughly. Given v9.21 was released just last week, my guess we will have to wait a few months to have a version with this bug fix which isn't ideal because this CI provides coverage for PR #23435 and #23351 and hence stalls those PR's.

I would like to get those PR's merged sooner rather than later. If the maintainers are okay with it, I propose to disable this CI for now and I can run a nightly job on my local SKX machine until we find a permanent fix for it.

@r-devulap For the pending PRs, it is probably good enough if it passes now and then :) The only annoyance is that all the failures need to be checked. Maybe we should just disable the CI test for now and rely on your home testing?

The only annoyance is that all the failures need to be checked. Maybe we should just disable the CI test for now and rely on your home testing?

Yeah, that is extra burden on the maintainers. I will be happy to set up something locally until we find a long term fix.

See #23682