[IPython-dev] Twisted reactor + IPython0

[Brian Granger]

>  I must clarify my idea:
>  My main objective is to provide a set of classes that anybody can use in
>  QT/WX/whatelse to create easily some ipython console GUI widget
>  (QTIpython,WXIpython,TKIpython etc...)

Great!

>  My work is based on WX but can be easily used for QT or other.I'm using
>  Scintilla for the display so... can be also easy to be ported.
>  (will see this weekend with all the guy how I could interface with
>  pyreadline)
>
>  So,first I wrote a little piece of code to get ipython shell inside a wx
>  widget.
>  It was based on user keyboard interaction with no thread.
>  I used for this a piece of gtk console and IPInteractive Shell class where I
>  could launch some commands line by line.
>  Then arise the problem of infinite loop:
>  While 1:
>         Pass
>  With this my widget and all my GUI was frozen, using a thread where I sent
>  line by line my command to an IPInteractive Shell solved this issue.
>  That's why I needed thread glenn ;)
>
>  A little Ascii drawing:
>
>  Thread wich feeds Ipython instance line by line  <--- communicate -->  WxApp
>  Main loop Thread
>
>  Now I've got this problem:
>  User want to  be able to use Wx Ipython console widget with for example QT4
>  support or WX support...
>  So I see no other idea to give them this class run in an other process where
>  they could launch any GUI unrelated to my widget GUI loop.
>  That's why I wanted xml-rpc and twisted support in ipython0.
>  (If you take a look at this it looks like that ipython1 should be the WAY to
>  do, but I don't know if ipython1 has interactive support like ipython0 and
>  ipython0 NEEDS GUI widgets ;) )

>  So perhaps twisted integration in ipython0 for that is not a clever idea,
>  perhaps a simpler xmlrpc server loop can be OK.
>  I really want to have GUI loop+XML/RPC loop+ipython interactivity in a class
>  that I could call to do all of this.
>
>  Guys,what is the best way?

It really depends on your goals.  If you need something super quick
and are OK hacking together something with IPython0, then just do
that.  But just a warning.  You will end up hacking something together
that only half works and as IPython1 matures, you will end up
re-writing it to use IPython1.

IPython1 is being designed from the ground up to support this type of
thing.  Does IPython1 have interactive support - absolutely!!! -
overwise it would have the "I" in front.  You could start developing
such GUI widgets today using IPython1 and the result would be robust,
well designed code that will last for years.

Example:  Just last week, Barry Wark created a very nice Cocoa App for
IPython1 using PyObjC.  While the resulting app doesn't have all of
the features of IPython0 (magics, tab completion, shell commands,
etc.), those are the easy part.  The important and hard part is
getting the whole design right.  And that is already done in IPython1.
 Depending on how you want to design the GUI stuff, you might need to
learn a bit about twisted, but that is not too bad.

The key point is that if you go this route, you will end up having fun
coding things that are well designed and will stand the test of time,
rather than hacking around trying to square a circle.

We are more than willing to point you to the relevant code in IPython1.

>  Ville:
>  I'll take a look at your file this week end, when gael and others guys in
>  france will have explained me all the bzr stuff :)

Sounds fun!

>  Glenn:
>
>
>  >So, your sketch using XML-RPC as an integration strategy makes sense if
>  you're inter-process but, I think, might be overkill in the same process.
>  Integrating multiple event loops is nasty because they >all "help" by
>  abstracting (hiding) the underlying OS mechanisms which emit events to the
>  application layer.  If we can get around this (as I did with the Qt reactor)
>  we're better off.
>
>  In fact xml-rpc should be used when guys want to play with GUI in Ipython
>  shell widget whithout burry my WXapp.
>  If it not a required thing than, xml-rpc is unusefull sure :), but I also
>  don't want to split my code...
>
>
>  >Furthermore, I wonder if your strategy will work in general even with
>  threads... it all depends on what the other event loop does with the
>  foundational wrapping of the event sources.... so, if you try, >for example,
>  to add your own socket and try and block on select, will that interfere with
>  Wx?
>
>  Well, for what I've tried on the current code, it works ;) I mean the
>  ipython instance with no GUI support and a wx loop controlling it (my
>  wxConsole widget)
>
>
>  >  If you had Wx and Qt in the same process, where would the window events
>  go?  None of these core engines >acknowledge the existence of any other
>  engines... and would likely behave very badly with the introduction of such
>  a virus in their process space.

Yes, multiple GUIs in the same process is a bad idea.

>  That why I want to do the "xml-rpc/ipython0 in an other process" trick :)
>
>
>  >If you're in a different process, all this is an entirely different
>  discussion and is more about event IO... none of which requires threading...
>  Qt, Twisted, and Wx all provide very rich mechanisms to >register callbacks
>  for this kind of event handling.
>
>  Yep I discovered that, before my wx widget was doing some polling on on the
>  the ipython thread, now I'm using wx callback and no more polling is
>  necessary, but the thread is still necessary due to while 1: pass lockup. In
>  a different process n omore thread is necessary and that is what I want :)
>
>  Glenn, if I well understood, what you've done is QT support and twisted and
>  ipython inside the "same loop"?
>  It could be interesting to see if we could have some sort of unified class
>  to support the whole thing wx+twisted+ipython and qt+twisted+ipython.
>  We need to check also Brian warnings :)
>
>  Brian:
>  Do you have a better idea than twisted integration for my specific need
>  (that will less destroy ipython1)???

If you want to got this route (using twisted in ipython0) there is a good route:

1) Start the reactor in a thread

2) Use blockingCallFromThread to call Twisted using methods.
blockingCallFromThread allows you to block in one thread until a
deferred fires in the Twisted thread.

You can see how we are doing this by checking out ipython1 and looking in:

ipython1/kernel/twistedutil.py
ipython1/kernel/client.py
ipython1/kernel/multiengineclient.py


Cheers,

Brian

>  Laurent
>
>
>  No virus found in this outgoing message.
>  Checked by AVG.
>  Version: 7.5.519 / Virus Database: 269.21.7/1335 - Release Date: 19/03/2008
>  09:54
>
>
>
>
>  _______________________________________________
>  IPython-dev mailing list
>  IPython-dev at scipy.org
>  http://lists.ipython.scipy.org/mailman/listinfo/ipython-dev
>