[IPython-dev] Embedding threaded IPython without IPShellEmbed

[Ravikiran Rajagopal]

Hi,

Background:
  I am trying to use python as a scripting language for a C++ application. The 
application has a lot of arrays that are exposed to python (via 
Boost.Python). For debugging and extension code development, I would like to 
be able to open up a python interpreter, and interactively manipulate the 
data structures (using new scipy) and visualize them (using matplotlib).

Problem:
  After Py_Initialize, I load up scipy via PyRun_SimpleString. Then, from C++ 
code, I would like to call the interpreter. Without ipython, I could use 
PyRun_InteractiveLoop, which would return control to my application on EOF. I 
would like to be able to do the same with ipython. The trick is also very 
straightforward: create an IPShellEmbed instance via PyRun_SimpleString, and 
then call its __call__ method using PyRun_SimpleString whenever I needed an 
interpreter.
  However, IPShellEmbed does not support threading options, and that makes it 
a pain to use Matplotlib. As far as I can see, the only option with 
IPShellEmbed is to use a single threaded back end (such as Tk) for MPL. I 
would much prefer to use the GTKAgg back end for MPL, since I would be able 
to keep the plot windows responsive (in a separate thread) while my 
application goes about its number crunching.
  I realize that I only need one instance of the embedded shell. In this case, 
I would like to make ipython do the necessary thread setup options (see code 
later) which works perfectly fine when calling the mainloop directly. 
However, I need ipython to return control back to the application without 
exiting the GTK/Qt main loop. How do I do this? In other words, how do I get 
in and out of the ipython mainloop interactively?

Code for simply using embedded ipython with pylab GTK backend support:

const char *ipython_ini = "import sys\n"
                          "sys.argv = ['','-pylab']\n"
                          "import IPython.Shell\n"
                          "IPython.Shell.start().mainloop()\n";
PyRun_SimpleString( ipythonini );

Code for embedding IPShellEmbed (no pylab support):

const char *ipython_ini = "import sys\n"
                          "sys.argv = ['','-pylab']\n"
                          "import IPython.Shell\n"
                          "ish = IPython.Shell.IPShellEmbed(sys.argv)\n";
const char *ipython_run = "ish()\n"
PyRun_SimpleString( ipython_ini );
// Much later in the code
PyRun_SimpleString( ipython_run );

Code example of what I would like to have but does NOT work:

const char *ipython_ini = "import sys\n"
                          "sys.argv = ['','-pylab']\n"
                          "import IPython.Shell\n"
                          "ish = IPython.Shell.start()\n";
const char *ipython_run = "ish.embed_mainloop()\n"
PyRun_SimpleString( ipython_ini );
// Much later in the code
PyRun_SimpleString( ipython_run );


Any help is greatly appreciated.

Thanks,
Ravi