[IPython-dev] Building an integrated measurement system with IPython

Ronen Abravanel ronena at gmail.com
Mon Feb 17 17:18:56 EST 2014

This is awesome.   I would very much like to see something like that
working (at least as a proof of concept. And also in order to help and try
to 'cell' python \ IPython to the experimentalists in my department.

Regarding your first question, try looking at IPython 2.0 and the new
widget mechanism,
For some examples, see

On Mon, Feb 17, 2014 at 8:27 PM, Zahari Dim <zaharid at gmail.com> wrote:

> Hi,
> My name is Zahari Dimitrov, and I am a last year Physics Student. My final
> year project consists on developing a system on top of IPython which is
> able to integrate the configuration of laboratory measurements with the
> analysis of the data of those measurements (which could be made in remote
> clusters). I would appreciate it very much if you could tell me what do you
> think on the overall design, and answer some (I'm afraid too many) specific
> questions I have. Apologies for this long message.
> Some information on this project can be found on zigzah.com (but it's not
> really up to date...)
> I have already done the part dedicated to talking with instruments (more
> or less), and now I would like to do the following:
> - I'd like to represent the computation process as a polytree (
> http://en.wikipedia.org/wiki/Polytree ) where the nodes (which I call
> IObjects) would be functions that run on some ipengine. These functions
> have some parameters (inputs) and return a dict of outputs. The outputs of
> one node can be connected to the inputs of the next, and to execute a child
> node, all the parents must be executed and have some results.
> -This structures are saved on a MongoDb Database (seems really easy to use
> for now) and managed with the mongoengine ORM,
> -I'd like to be able to transform the inputs of the graph that are not
> connected to an output (or have a constant value set) in an html form (the
> inputs declare some types like str, int or range which are mapped to
> different JS input widgets where possible). The free outputs are turned
> into some display widgets as well.
> -When you type ipython notebook --iograph=mygraph, the form appears on the
> top of your ipython notebook, and you can set the parameters and execute
> different parts of the graph. The results are accessible as variables in
> the notebook.
> -Things like the instrument commands (ie, ask for the frequency in an
> oscilloscope) are just things that extend the IObject class and run on a
> dedicated engine with id __instruments in the computer where the instrument
> is connected (could be different for different instruments).
> -Other more complicated classes like RangedExperiment (measure something
> over a given range which is an input) are also IObjects. This classes would
> have the capability of reporting the measurements as they are produced and
> fill sequentially a plot with that.
> -Other IObjects can take the results from the experiment and process them
> in a remote cluster.
> -For a given IObject, all results (outputs) it produces are logged by
> default in the mongo database. You can always set the value of the outputs
> to a previous result and execute its children with this input.
> -There will be a decorator that converts any function in a IObject (and
> possibly use python 3 annotations for that)
> As said, I'd be very happy if I could complete this. But here are the
> questions:
> -At what level do I hack the IPython notebook to add  the forms? I haven't
> found an extension mechanism capable of doing what I need, so I thing I
> have to fork the project. Is there any place that explains the conventions
> used for the templates and the JavaScript code in the notebook?
> -I believe the Ipython controller has some log system. Could it be
> integrated with the one I want for the IObjects?
> -I am thinking about using mongodb to deploy a shared filesystem in the
> computers where Ipython is running, so it's possible load data from
> anywhere. Is this a good solution, particularly to save things like
> text-pickled python objects?
> -How do I avoid moving around large amounts of data when the output and
> input IObjects are in the same computer, different from the controller.
> -Which is the best way to obtain partial results from the engines, like
> the ones needed for the RangedExperiment example (ie, display the
> measurement after it is produced and then do the next)?
> -Which is the current best way to display interactive plots in IPython?
> Thinking again about the Ranged Experiment.
> -I understand each Ipengine will see the hardware of the computer it runs
> in. Is this right?
> -Is there any problem for a node to itself run a paralell algorithm on
> other engines?
> Thank you again for your patience,
> Best regards,
> Zahari.
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20140218/d7b88b61/attachment.html>

More information about the IPython-dev mailing list