Zdeněk Hurák wrote:

Coming from the field of control engineering, I don't think that at this moment there is any replacement for their graphical interface to solvers for nonlinear differential/difference equations called Simulink.

This is correct. This is the one thing that Python needs improvements in. Many of my colleagues use simulink to design and model a digital signal processing element which they can then download to an FPGA using a third-party tool from XiLINX. There is no way they could replace that with Python/SciPy at this point (although of course it "could" be done).

I would love to see some good contributions in the area of Simulink-like work. There are several things out there that are good starts.

But what makes Matlab difficult to be replaced is that lots of other projects (commercial: Mathematica, Maple, ... and free: octave, maxima, scipy, ...) only offer computation and visualization, while engineers in my field also need INTERACTION OF THE SYSTEM WITH EXTERNAL WORLD. That is, compatibility with a real-time operating system and MOST available input-output (AD-DA) cards.

The only way to solve this is to get more users interested in making these kinds of things happen. Or to start a company that does this and charges for a special build of Python compatible with your favorite hardware.

Being able to acquire measurement data from an external industrial system, process them computationally (for instance, solving some Riccati matrix differential equations), visualize the data and put the computed results back to the real system, this is what we need.

This is doable in many respects already (look at what Andrew Straw has done for example), but it of course could be made "easier" to do. But, I'm not sure it will happen without the work of a company. It would be great if hardware manufacturers used Python and made sure their stuff worked right with it, but this requires many more users.

I am absolutely sure that Python (and Scipy and Numpy) project has a potential to fulfill also these needs (and by starting using these and sharing my own code I would like to contribute), but it is not the case at the moment. Without being negative and discouraging, I think it is fair to say that currently for some people it would be very difficult to switch completely to Python libraries.

Yes, this is fair to say. As you have indicated, the situation is improving.

-Travis

On Thu, Apr 26, 2007 at 12:06:56PM +0200, Zdeněk Hurák wrote:

But what makes Matlab difficult to be replaced is that lots of other projects (commercial: Mathematica, Maple, ... and free: octave, maxima, scipy, ...) only offer computation and visualization, while engineers in my field also need INTERACTION OF THE SYSTEM WITH EXTERNAL WORLD. That is, compatibility with a real-time operating system and MOST available input-output (AD-DA) cards. Being able to acquire measurement data from an external industrial system, process them computationally (for instance, solving some Riccati matrix differential equations), visualize the data and put the computed results back to the real system, this is what we need.

I am very surprised that you think Matlab is more suited for such a task. I have had the opposite experience.

I work in an experimental lab where we do rather heavy experimental physics (Bose Einstein condensation). I am building an experiment from scratch and had to build a control framework for the experiment. It is fully automated and with need some reasonably sophisticated logics to scan parameters, do on the fly processing of the data, store it on the disk, display all this in a useful interface for the user, and feed all this back to the experiment.

Due to legacy reasons I built the software in Matlab. It was a bad experience.

First of all linking to C libraries to control hardware is really a pain. Writing Mex files is not that much fun and the memory management of MatLab is flawed. I have had segfaults for no obvious reasons, and the problems where neither in my code nor in the drivers (I worked with the engineer who wrote the driver to diagnose this). Matlab is single threaded so all your calls to the hardware are blocking (unless you are willing to add threads in you mex file, but this is very dangerous as you are playing with a loop-hole in matlab's mex loader which keeps the mex in the memory after it is executed). Finally the lack of proper object oriented programming and separated namespaces make it hard to write good code to control instruments.

The lack of object oriented programming, passing by reference, threads, and proper control of the GUI event-loop also makes it very hard to write a complex interactive GUI.

Python provides all this. What it does not provide are pre-linked libraries to control hardware, but if you are trying to control exotic hardware, all you have is a C or C++ SDK. It is even worse when the hardware is homemade, as you have to write the library yourself, and trust me, I'd much better write a hardware control library that speak to my custom made electronic over a bus in Python than in Matlab (GPIB, ethernet, serial, Python has bindings for all that).

I have had a second choice to build the computer control framework of a similar experiment. I took my chance to build it in Python (it was a different lab, and I was replacing an existing C software that had become impossible to maintain). I never regretted it. Python has really been very good at it due to all the different libraries available, the ease to link to C, and the possibility to do proper OOP.

I have written an article about this, that I submitted a month ago to CiSE. I have no news from them. The article is very poorly written (I had no hindsight about these matters when I started writing it, submitted it because I was tired to see it dragging along, and now I wish I had reworked it a bit), but I think I raise some points about what you need to build a control framework for an experiment, and how python can address those needs.

Anyway, IANAL, and I am not to sure if releasing a preprint on a mailing list renders the article ineligible for CiSE or not but I just put a version on http://gael-varoquaux.info/computers/agile_computer_control_of_an_experiment... . I hope it can help understanding what is needed to control an experiment from Python, what can be improved, and what already exists and is so incredibly useful.

I am interested in comments. Keep in mind that I am working in a field where nobody is interested in computing and I sure could use some advice. I have written quite a few softwares to control experiments in different labs with different languages, and I think I have acquired some experience, and made a lot of progress, but I also have had to reinvent the wheel more than once. We all know that good software engineering books or article readable by some one who nows little about computing are hard to find. Let us address this issue !

Gaël

On 4/26/2007 2:19 PM, Steve Lianoglou wrote:

...

Beside proper programing paradigm Python easily scales to large- scale number crunching: You can run large-matrices calculations with about 1/2 to 1/4 of memory consumption comparing to Matlab.

Is that really true? (The large-matrix number crunching, not the proper programming paradigm ;-)

By no scientific means of evaluation, I was under the impression that the opposite was true to a smaller degree.

Matlab have pass-by-value semantics, so you have to copy your data in and copy your data out for every function call. You can achieve the same result in Python by pickling and unpickling arguments and return values, e.g. using this function decorator:

import cPickle as pickle

def Matlab_Semantics(f):

''' Emulates Matlab's pass-by-value semantics, objects are serialized in and serialized out.

Example: @Matlab_Semantics def foo(bar): pass '''

func = f return wrapper

def wrapper(*args,**kwargs): args_in = pickle.loads(pickle.dumps(args)) kwargs_in = {} for k in kwargs: kwargs_in[k] = pickle.loads(pickle.dumps(kwargs[k])) args_out = func(*args_in,**kwargs_in) args_out = pickle.loads(pickle.dumps(args_out)) return args_out

Imagine using this horrible semantics in several layers of function calls. That is exactly what Matlab does. Granted, Matlab optimizes function calls by using copy-on-write, so it will be efficient in some cases, excessive cycles og copy-in and copy-out is usually what you get.

Sturla Molden

On 4/26/2007 2:42 PM, David Cournapeau wrote:

You are true for the semantics, but wrong for the consequences on

copying, as matlab is using COW, and this works well in matlab.

It works well only if you don't change your input arguments. Never try to write to a matrix received as an argument in a function call. If you do, the memory expenditure may grow very rapidly.

But as long a s NumPy does not use lazy evaluation the difference is not as striking as it could be.

S.M.

On 4/26/2007 2:47 PM, Neal Becker wrote:

That's interesting. How did you find this information?

What information?

Matlab's pass-by-value semantics is well known to anyone who has ever used Matlab.

The Mathwork's employees have numerous times stated that Matlab uses copy-on-write to optimize function calls. I first learned of it in the usenet group comp.soft-sys.matlab.

Sturla Molden