[IPython-dev] Converting ipython notebook from tornado to gevent?

[MinRK]

I would reiterate Matthias‘s points - it’s not necessarily difficult to use
gevent+wsgi to serve the notebook webapp, but there would be a lot of code
that is not shared (essentially all web handler code), which would be a
substantial maintenance burden to keep in sync with IPython development,
since you would really be on your own.

Basic points:

   1. I agree with Matthias that probably the best (certainly the easiest
   to maintain) choice is to run the existing notebook server (or at least a
   subset or superset of its handlers) unmodified in a subprocess, and
   proxying requests to that. I would look at that first, and only consider
   porting if it fails.
   2. Technically, you can run the tornado app itself in a gevent universe
   by telling it to use the Poller object from zmq.green.eventloop instead
   of zmq.eventloop. It will not play especially nicely with fellow
   greenlets, but at least where it spends 99% of its time (waiting in the
   poller), it would cooperate properly.

 One of the tasks we have on our todo list is to convert the IPython
Notebook from Tornado to Gevent.

I hope you reconsider this goal. A true wsgi-gevent notebook webapp would
really mean a fork of the notebook, with all of the long-term burden and
unpleasantness that entails. Getting the existing notebook to sit somewhere
inside a webapp that happens to use a different toolkit is probably more
sustainable.


-MinRK


On Wed, Jun 26, 2013 at 12:29 PM, Matthias BUSSONNIER <
bussonniermatthias at gmail.com> wrote:

> Hi Steve,
>
> Le 26 juin 2013 à 20:32, Stephen Chan a écrit :
>
> > Hi,
> >    A group of us in Adam Arkin's KBase group at LBL has started working
> on a bioinformatics notebook using IPython Notebook. So far it looks like
> the ideal platform and we're glad to be finally under way.
> >
> >    One of the tasks we have on our todo list is to convert the IPython
> Notebook from Tornado to Gevent. We aren't considering this out of any
> perverse desire to create additional work for ourselves, but mainly because
> our existing deployment environment is based on uwsgi+gevent and the fact
> that gevent does a pretty good job of moving asynchrony back into the
> plumbing (where some of us think it belongs) so that we can code using a
> more conventional python idiom.
> >
> >    I think that this came up in f2f conversations with Fernando and
> MinRK, but I just wanted to bring it up again and ask if anyone considered
> a port to specifically to gevent, and if there's any idea of the scope of
> work? Is it something like a week of work for a solid python developer, or
> is it more like a couple of weeks or more? Are there any gotchas that may
> arise due to code written against tornado?
>
> Sorry  if my answer does not totally respond to your question,
>
> I don't know what your exact requirement, and I'm not the most familiar
> with the handler side of IPython notebook.
> >From the few things I read about tornado, I've seen here and there that
> integrating gevent with tornado should be possible in some cases.
> The big problem I think would be to keep the thing up to date with the
> following development.
>
> Not much should rely on tornado outside of the server, if it does it is
> probably a bug. It should even be possible I guess to write a server in
> another language that talk to the kernel/browser and write/read ipynb files
> to disk. as long as you conform to the web socket and zmq protocol it
> should be transparent for the client.
>
> Have you also considered a middleware ? You probably have your reasons to
> want to use gevent, but maybe what you might want is a bridge that only
> forward websocket/http(s) ? It will with higher chances stay compatible
> across version also.
>
> --
> Matthias
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20130627/591af50f/attachment.html>