[IPython-dev] threadsafe widgets

Brian Granger ellisonbg at gmail.com
Fri Jan 9 18:34:43 EST 2015

Yuck :( but I am not surprised at this...

> I believe this is because all widgets seem to share the same iopub_socket
> and write to it whenever the user changes a traitlet.  And of course zmq
> sockets aren't threadsafe.
Yep, they definitely share a single iopub socket.

> === A few potential solutions ===
> 1) "Quick" -- Wrap iopub_socket in a threadsafe wrapper.  Surround every
> important function call in a mutex.

Probably my second choice.

> 2) "Hintjens-approved" -- Give every single widget its own inproc PUB
> socket.  We'd have to spin up an extra thread (or a greenlet in some
> pre-extant thread?  I don't really understand what the different IPython
> threads do...but I could find out...) with a SUB socket to collect them all
> together and forward them all out on iopub_socket.

Definitely not going to happen. One of the issues we are running into,
especially in multiuser deployments of the notebook is that file
descriptors get consumed. If anything we are wanting to *reduce* the number
of sockets used by ipython.

> 3) "Dirty" -- Wrap IPython.kernel.zmq.session.Session.send inside a
> mutex.  Are there use-cases where anybody besides Session actually does
> anything with iopub_socket?
There shouldn't be any other way that send gets called, so this is probably
my top choice.

> === Motivation ===
> I think there could be valid use-cases for widgets which display results
> from polling external resources, e.g. widgets which keep track of EC2
> instances, widgets which monitor IPython.parallel workerbees, etc.
I have already started to use widgets with threads, but in my case I didn't
quite run into issue you are seeing - almost though. This is definitely
going to show up more.

> === Conclusion ===
> I'd be happy to attack this problem, if you guys think it would be
> useful.  If so, let me know what would be a good general direction, and
> I'll put together a pull request.

@minrk is our expert on the zeromq stuff. Can you open an issue on the
ipython/ipython repo and mention @minrk and @ellisonbg and @jdfreder



> Keep warm!
> Jackson Loper
> Pattern Theory Group
> Division of Applied Math
> Brown University
> PS. I love the new text editor feature in the ipython master branch!  My
> days of vim-over-ssh aren't over yet (NERDTree and push-notification of
> filechange detection are still too useful to me when logging on from
> multiple machines) but I can see the end in sight!
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev

Brian E. Granger
Cal Poly State University, San Luis Obispo
@ellisonbg on Twitter and GitHub
bgranger at calpoly.edu and ellisonbg at gmail.com
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20150109/23b886e3/attachment.html>

More information about the IPython-dev mailing list