[IPython-dev] Embedding/Integrating IPython into an application

Joshua Adelman jla65 at pitt.edu
Thu Oct 10 15:05:04 EDT 2013


Hi IPython devs and community,

I was hoping to get some feedback/suggestions on potentially embedding/integrating IPython into a Qt (PySide) application. The basic use case is that I have a large dataset stored as a numpy array that fits in memory but takes a non-negligble amount of time to load and large enough that I want to avoid making copies of it.  I want that data to be persistent in memory and then execute analysis code against that data without modifying it. I thought that IPython might be well suited to this sort of scheme as I could embed or integrate a kernel and then execute code in the same namespace as the data. I guess it's sort of a multiple-program single data type scheme. The Qt application is just providing an interface for setting up the methods to run on the data and the different parameter sets, where the code is prepared externally, but is discoverable by the application. The output of the code is a set of .npy or .npz files that might be loaded subsequently to perform further analysis, although it would be desirable to optionally load those arrays into the namespace. The analysis is an iterative and interactive process in which analysis scripts might be modified subsequently and then re-run, so the data should remain loaded in a long-running app. 

I coded a simple prototype of such a scheme that launched an IPython instance and then attached another client to the kernel and then used the send_channel to execute an analysis script using the %run magic akin to what is done in vim-ipython. Looking through the code base and the Scipy conference - IPython in Depth kernel embedding examples, it looks like there are several levels at which one might approach integrating IPython into an app though (e.g. using kernel_embed, integration with a GUI event loop, etc). 

I'd be particularly interested in feedback about whether (1) using IPython in this manner seems reasonable given the use case (and if not, are there other recommended alternative approaches), and if so (2) what level of the API might be best suited for integrating IPython in this case?

Any suggestions/insight would be most appreciated.

Best wishes,
Josh 

--
Dr. Joshua L. Adelman
Department of Biological Sciences
University of Pittsburgh, Pittsburgh, PA


More information about the IPython-dev mailing list