Skip to content

Multiprocessing usage results in ZMQError: Address already in use #6109

Closed

Description

Some custom python code that uses multiprocessing was occasionally throwing ZMQErrors for me when used in a notebook.

Here's a simple test case I made that always reproduces the issue for me. It creates a process that creates a process.

from multiprocessing import Process
def worker_worker():
    d = Process(target=worker)
    d.start()
    d.join()    

def worker():
    pass

p = Process(target=worker_worker)
p.start()
p.join()

Running this as a script with python, it finishes correctly. Running it in an ipython notebook causes a hang. When interrupted the exception shown is:

Exception in thread Thread-4:
Traceback (most recent call last):
  File "/usr/local/Cellar/python/2.7.7_1/Frameworks/Python.framework/Versions/2.7/lib/python2.7/threading.py", line 810, in __bootstrap_inner
    self.run()
  File "/Users/rkhadiwala/Library/Python/2.7/lib/python/site-packages/zmq/utils/garbage.py", line 34, in run
    s.bind(self.gc.url)
  File "socket.pyx", line 444, in zmq.backend.cython.socket.Socket.bind (zmq/backend/cython/socket.c:4076)
  File "checkrc.pxd", line 21, in zmq.backend.cython.checkrc._check_rc (zmq/backend/cython/socket.c:6032)
ZMQError: Address already in use

system information (I've also tried this on another box (linux) and saw the same issue):

python -c "import IPython; print(IPython.sys_info())"
{'commit_hash': '681fd77',
 'commit_source': 'installation',
 'default_encoding': 'UTF-8',
 'ipython_path': '/Users/rkhadiwala/Library/Python/2.7/lib/python/site-packages/IPython',
 'ipython_version': '2.1.0',
 'os_name': 'posix',
 'platform': 'Darwin-13.3.0-x86_64-i386-64bit',
 'sys_executable': '/usr/bin/python',
 'sys_platform': 'darwin',
 'sys_version': '2.7.5 (default, Mar  9 2014, 22:15:05) \n[GCC 4.2.1 Compatible Apple LLVM 5.0 (clang-500.0.68)]'}

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Assignees

Labels

Type

No type

Projects

No projects

Relationships

None yet

Development

No branches or pull requests

Issue actions