[IPython-dev] !commands in the zmq model
gael.varoquaux at normalesup.org
Sat Aug 28 08:51:56 EDT 2010
On Sat, Aug 28, 2010 at 12:29:53AM -0700, Fernando Perez wrote:
> We can interrupt subprocesses fine with our current
> (subprocess.Popen-based) implementation, but we can't get their output
> cleanly as it happens. So in the usual:
> for i in range(200):
> print i,
> print 'done!'
> We only see *all* the numbers at the very end. I haven't found any way
> around this with subprocess, and neither did our old ipythonx
> implementation done by Gael; as best as I understand popen it simply
> can't be done: in pipe mode, the C stdio library does block-buffered
> io on all process execution and there's no clean way I can find, to
> read from a pipe with a timeout.
Hum, I thought that I had it working. I can't test right now, but I seem
to remember I had spend quite a lot of time on that.
I am not claiming that my code was terribly good, ( :> "I write code, I
look at it a year later, I vomit", Michael A., CACR) but if I remember
correctly, there might be a few tricks to try and keep.
* For pure Python code, as your example above, the trick was to use what
I have called the 'RedirectorOutputTrap' that registers a callback on
writing to the sys.stdout/sys.stderr. It is implemented in
* For subprocesses, I had to resort to 2 threads, one executing the
process, the other polling its stdout (ugly). This logic can be found
in IPython.frontend.process.pipedprocess. It also adds a bit of logic
to be able to kill the process under Windows, which you might want to
keep in mind.
I am not sure if this code can be of some use to you or not, but just in
case. I believe that it might be one of the few useful things that came
out of my work. It actually even seems to have tests :$
More information about the IPython-dev