[IPython-dev] [matplotlib-devel] Uniform GUI support across matplotlib, ets and ipython

Brian Granger ellisonbg at gmail.com
Fri Sep 3 14:28:11 EDT 2010


On Fri, Sep 3, 2010 at 9:22 AM, Michiel de Hoon <mjldehoon at yahoo.com> wrote:
> --- On Wed, 9/1/10, Brian Granger <ellisonbg at gmail.com> wrote:
>> > So right now you're in a loop in which you let qt4 (or
>> wx) watch the file descriptors qt4 needs, then zeromq the
>> file descriptors that zeromq needs, and so on?
>> ZMQ sockets are not really sockets in that they do not have
>> a file descriptor interface.
> That may be, but at the bottom of the ZMQ event loop is a call to select(), which expects a pair of ordinary file descriptors. At the same time, at the bottom of the qt4 event loop is also a call to select(), also using a set of file descriptors.

What you are saying is mostly right.  ZMQ uses the best polling
mechanism for the platform (select/poll/epoll/kqueue/etc).  In some
cases, there are actual file descriptors in the internals of zeromq.
As I understand it:

* Zeromq runs on other transports other than TCP and not all of those
transports have file descriptors.
* Even if zeromq is using TCP for transport, a single zeromq socket
may be implemented with many low level sockets during its lifetime.
These sockets are not in any way exposed in the public zeromq API.
* zeromq sockets are connectionless.  If you were able to get those
low level file descriptors, the model would become leaky because you
would need to start handling the connection related events.

Thus, it is currently not an option to get those low level file
descriptors and do what you are proposing.

> Currently your event loop works approximately as follows:
> 1) Run the qt4 event loop, which calls select() inside qt4, watching the file descriptors that qt4 knows about.
> 2) Interrupt the qt4 event loop, and enter the ZMQ event loop, which calls select() inside ZMQ to watch the file descriptors ZMQ knows about.
> 3) If none if the ZMQ file descriptors have been signaled, go to 1).
> Ideally, you would want to call select() only once, and have it watch both the file descriptors qt4 knows about, as well as the file descriptors ZMQ knows about.
> So how to implement this?
> Think about just the ZMQ event loop for now. Just before entering the ZMQ event loop, ZMQ knows which file descriptors it should watch when it calls select(). Which means that ZMQ maintains in some form or another a list of file descriptors to watch. Now I don't know the details of ZMQ, but I suppose there is some function inside ZMQ that effectively adds one file descriptor to the ZMQ list of file descriptors to watch. Now, instead of this function, call an ipython-specific function that effectively adds this file descriptor to the list of file descriptors that qt should watch for (probably using QSocketNotifier). Now you don't need to enter the ZMQ event loop at all: When you enter the qt event loop, and there is any activity on one of the ZMQ file descriptors, the qt event loop will notice and take the appropriate action.
> Does this make sense?

Definitely, it makes total sense, it is just not possible today with
how zeromq is implemented.  The zeromq devs are working on a file
descriptor interface, but it will not be fully compatible with the
usual one in that you won't be able to distinguish between read, write
and error events with only the file descriptor.  I am not sure the
file descriptor interface that zeromq will eventually have will
actually work with what qt is expecting.

The other issue is one of maintainability.  Currently the GUI code and
zeromq event loop code is completely separate and orthogonal.  We can
easily change one without affecting the other.  If we were to combine
the event loops, we would loose that nice separation that makes our
code more robust, stable and maintainable.  Also, I should mention
that the current approach functions perfectly well.  Both the GUI and
IPython remain extremely responsive at all times.

I should also mention that the help we need with matplotlib right now
is *completely* independent of these issues of how we integrate the
two event loops.

Thanks for the ideas though.



> --Michiel.

Brian E. Granger, Ph.D.
Assistant Professor of Physics
Cal Poly State University, San Luis Obispo
bgranger at calpoly.edu
ellisonbg at gmail.com

More information about the IPython-dev mailing list