[IPython-dev] Starting to plan for 0.11 (this time for real)

[Brian Granger]

> As for non-blockers, we have:
>
> - the parallel code is not in a good situation right now: we have a
> few regressions re. the Twisted 0.10.1 code (e.g. the SGE code isn't
> ported yet), the Twisted winhpc scheduler is only in 0.11, and while
> the new zmq tools are looking great, they are NOT production-ready
> quite yet.  In summary, we'll have to warn in bright, blinking pink
> letters 1995-style, everyone who uses the parallel code in production
> systems to stick with the 0.10 series for a little longer.  Annoying,
> yes, but unfortunately such is life.

I want to at least proposed the following solution to this:

Remove all of the twisted stuff from 0.11 and put the new zmq stuff in
place as a prototype.

Here is my logic:

* The Twisted parallel stuff is *already* broken in 0.11 and if anyone
has stable code running on it, they should be using 0.10.
* If someone is happy to run non-production ready code, there is no
reason they should be using the Twisted stuff, they should use the
pyzmq stuff.
* Twisted is a *massive* burden on our code base:
  - For package managers, it brings in Twisted, Foolscap and zope.interface.
  - It makes our test suite unstable and fragile because we have to
run tests in subprocesses and use trial sometimes and nose other
times.
  - It is a huge # of LOC.
  - It means that most of our codebase is Python 3 ready.

There are lots of cons to this proposal:

* That is really quick to drop support for the Twisted stuff.
* We may piss some people off.
* It possibly means maintaining the 0.10 series longer than we imagined.
* We don't have a security story for the pyzmq parallel stuff yet.

I am not convinced this is the right thing to do, but the benefits are
significant.

Cheers,

Brian

-- 
Brian E. Granger, Ph.D.
Assistant Professor of Physics
Cal Poly State University, San Luis Obispo
bgranger at calpoly.edu
ellisonbg at gmail.com