gh-142414: Unify UNBOUND in Lib/concurrent/interpreters/_queues.py and Lib/concurrent/interpreters/_crossinterp.py. #142515
Conversation
note35
changed the title
note35
changed the title
ZeroIntensity
left a comment
ZeroIntensity
left a comment
There was a problem hiding this comment.
I think it would also be good to add a dedicated test for this, similar to Victor's small repro in the original issue. This bug only occurs under some special circumstances in the current test suite, which isn't ideal.
Misc/NEWS.d/next/Library/2025-12-10-15-27-58.gh-issue-142414.zTHgP-.rst
Outdated
Show resolved
Hide resolved
|
I also created another PR to remove singleton - #142553 |
…est_interpreter_pool"
Misc/NEWS.d/next/Library/2025-12-10-15-27-58.gh-issue-142414.zTHgP-.rst
Outdated
Show resolved
Hide resolved
…THgP-.rst Co-authored-by: Peter Bierma <[email protected]>
We don't need to make singleton picklabe based on the current tests.
| except queues.QueueEmpty: | ||
| pass |
There was a problem hiding this comment.
./python -m test test_interpreters test_concurrent_futures.test_interpreter_pool
Using random seed: 1867170309
0:00:00 load avg: 0.27 Run 2 tests sequentially in a single process
0:00:00 load avg: 0.27 [1/2] test_interpreters
0:00:24 load avg: 0.48 [1/2] test_interpreters passed
0:00:24 load avg: 0.48 [2/2] test_concurrent_futures.test_interpreter_pool
... (running forever without catching queues.QueueEmpty)
def test_blocking_with_limited_workers(self):
ready = queues.create()
blocker = queues.create()
def run(taskid, ready, blocker):
ready.put_nowait(taskid)
blocker.get()
numtasks = 10
futures = []
with self.executor_type(4) as executor:
for i in range(numtasks):
fut = executor.submit(run, i, ready, blocker)
futures.append(fut)
pending = numtasks
while pending > 0:
done = 0
for _ in range(pending):
try:
ready.get(timeout=1)
# https://github.com/python/cpython/blob/2a820e2b9cf9e470d9f5342019fca3fe7f4ed7bc/Lib/concurrent/interpreters/_queues.py#L261C20-L261C30
# test_interpreters test_concurrent_futures.test_interpreter_pool
# Before: queue.get() throws interpreters.QueueEmpty
# After: queue.get() throws queues.QueueEmpty
# test_concurrent_futures.test_interpreter_pool
# Before: queue.get() throws interpreters.QueueEmpty.
# After: queue.get() throws interpreters.QueueEmpty
except interpreters.QueueEmpty:
pass
except queues.QueueEmpty:
pass
else:
# If the error is not caught, the loop is running forever.
done += 1
pending -= done
for _ in range(done):
blocker.put_nowait(None)Why does running test_interpreters + test_concurrent_futures.test_interpreter_pool make the queue.get() throws queues.QueueEmpty after having the same UNBOUND object in _queues.py and _crossinterp.py?
FWIK, cpython/Lib/concurrent/interpreters/init.py loads queues.QueueEmpty as interpreters.QueueEmpty, and before this PR, queue.get() is expected to throw interpreters.QueueEmpty,
For some reasons, queue.get() throws _queues.QueueEmpty after running test_interpreters. This may be another thing to explore further (I think it depends on how the sub-interpreter is implemented. Do you have any knowledge about this?
Given this is a test fix, maybe it's okay to have a further dive deep separately? I think this so far can be reproduced only during a combination of test_interpreters + test_concurrent_futures.test_interpreter_pool. (which is likely rare and not reported?) I'd also like to add a TODO gh-XXX, but we don't yet have enough context to make an issue...)
2 participants
The fix is aligned with bug reporter's observation.
Test with
./python -m test test_interpreters test_struct(This one triggers OP's issue.)./python -m test test_interpreters test_concurrent_futures.test_interpreter_pool(This one triggers the forever loop after the initial fix.)./python -m test test_interpreters test_structfails with: KeyError: concurrent.interpreters._queues.UNBOUND #142414