Open
Description
I don't know if this is supposed to be supported at all.
Describe the bug
The multiprocessing spawn method cannot be changed when using scalene.
To Reproduce
The following code will crash with the error:
Error in program being profiled:
context has already been set
Traceback (most recent call last):
File "/home/USER/anaconda3/envs/ox/lib/python3.12/site-packages/scalene/scalene_profiler.py", line 1795, in profile_code
exec(code, the_globals, the_locals)
File "/home/USER/Dokumente/mptest/test.py", line 6, in <module>
multiprocessing.set_start_method("spawn")
File "/home/USER/anaconda3/envs/ox/lib/python3.12/multiprocessing/context.py", line 247, in set_start_method
raise RuntimeError('context has already been set')
RuntimeError: context has already been set
import multiprocessing
import multiprocessing.process
multiprocessing.set_start_method("spawn")
def test():
print("At", multiprocessing.process.current_process())
with multiprocessing.Pool(4) as p:
for i in range(8):
p.apply_async(test)
p.close()
p.join()Expected behavior
No crash
Desktop (please complete the following information):
- OS: Ubuntu 24.04.1 LTS
- Browser doesn't matter
- Version Scalene version 1.5.45 (2024.10.01) - also tried the current git version
Additional context
The error can be mitigated by using multiprocessing.set_start_method("spawn", force=True) but in that case it will crash when spawning the processes.
Error in program being profiled:
Can't pickle <class 'scalene.replacement_sem_lock.replacement_semlock.ReplacementSemLock'>: attribute lookup replacement_semlock.ReplacementSemLock on scalene.replacement_sem_lock failed
Traceback (most recent call last):
File "/home/USER/anaconda3/envs/ox/lib/python3.12/site-packages/scalene/scalene_profiler.py", line 1795, in profile_code
exec(code, the_globals, the_locals)
File "/home/USER/Dokumente/mptest/test.py", line 13, in <module>
with multiprocessing.Pool(4) as p:
^^^^^^^^^^^^^^^^^^^^^^^
File "/home/USER/anaconda3/envs/ox/lib/python3.12/multiprocessing/context.py", line 119, in Pool
return Pool(processes, initializer, initargs, maxtasksperchild,
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/USER/anaconda3/envs/ox/lib/python3.12/multiprocessing/pool.py", line 215, in __init__
self._repopulate_pool()
File "/home/USER/anaconda3/envs/ox/lib/python3.12/multiprocessing/pool.py", line 306, in _repopulate_pool
return self._repopulate_pool_static(self._ctx, self.Process,
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/USER/anaconda3/envs/ox/lib/python3.12/multiprocessing/pool.py", line 329, in _repopulate_pool_static
w.start()
File "/home/USER/anaconda3/envs/ox/lib/python3.12/multiprocessing/process.py", line 121, in start
self._popen = self._Popen(self)
^^^^^^^^^^^^^^^^^
File "/home/USER/anaconda3/envs/ox/lib/python3.12/multiprocessing/context.py", line 289, in _Popen
return Popen(process_obj)
^^^^^^^^^^^^^^^^^^
File "/home/USER/anaconda3/envs/ox/lib/python3.12/multiprocessing/popen_spawn_posix.py", line 32, in __init__
super().__init__(process_obj)
File "/home/USER/anaconda3/envs/ox/lib/python3.12/multiprocessing/popen_fork.py", line 19, in __init__
self._launch(process_obj)
File "/home/USER/anaconda3/envs/ox/lib/python3.12/multiprocessing/popen_spawn_posix.py", line 47, in _launch
reduction.dump(process_obj, fp)
File "/home/USER/anaconda3/envs/ox/lib/python3.12/multiprocessing/reduction.py", line 60, in dump
ForkingPickler(file, protocol).dump(obj)
_pickle.PicklingError: Can't pickle <class 'scalene.replacement_sem_lock.replacement_semlock.ReplacementSemLock'>: attribute lookup replacement_semlock.ReplacementSemLock on scalene.replacement_sem_lock failed
Metadata
Metadata
Assignees
Labels
No labels