[IPython-dev] Engine Queue sockets

[MinRK]

I'm working on the controller, making pretty decent progress, but keep
running into design questions.

The current one:
I have been thinking of the Controller-Engine connection as PAIR sockets,
but it seems like it could also be just one XREP on the Controller and XREQs
on the engines. Then the controller, rather than switching on actual
sockets, switches on engine IDs, since XREP uses the first entry in
send_multipart to determine the destination.

For a crude task model, we could reverse those connections and allow XREQ on
the controller to load balance. Of course, then we would lose any
information about which engine is doing what until jobs complete.

Could there be an issue in terms of scaling for the controller to be
creating thousands of PAIR sockets versus managing thousands of connections
to one XREP socket?

-MinRK

Also: I'm in the #ipython IRC channel now, and am generally trying to be
online there while I work on IPython stuff.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20100718/ee3ebcc7/attachment.html>