running a pipe of commands from python

[Harald Kirsch]

"Fredrik Lundh" <fredrik at effbot.org> writes:

> Harald Kirsch wrote:
> > In Tcl I just write
> >
> >    exec doThis | doThat |toMore | finishUp >output <input 2>errors
> >
> > In the Python reference manual I find os.exec(), os.fork(), popen2()
> > etc, but none of these is as easy to use as Tcl's exec. What is
> > Python's equivalent (and I am sure there must be somehting)?
> 
> you don't say what output, input and errors are, so it's
> not obvious what you're looking after.
> 
> if they're filenames, use:
> 
>     os.system("doThis | doThat >output ...")

Buuuh. I don't like the overhead of the call to the shell. In addition
the shell has to parse the given command line which means that I
really have to take care of escaping special characters and the like.

> 
> if they're file handles, use os.popen3:
> 
>     input, output, errors = os.popen3("doThis | doThat")

This also seems to call a shell in between.

> for a more general (but unix-only) solution, see:
> http://www.python.org/doc/current/lib/module-pipes.html

This also uses /bin/sh. And all this although python has all the
necessary ingredients like fork/exec/wait/signal to do the plumbing
itself. Strange.

I think I'll stick to my own hack, the doc of which currently reads
like this:

def execpipe(cmds):
  """
  runs the commands given in the list cmds in a pipe with their stdout
  and stdin plumbed together. Every element of cmds must itself be a
  list giving the name of the command as the first element and the
  rest of the list being its arguments.

  If any of the elements of cmds is a pair, then the first must be the
  command to run and the 2nd must be a file descriptor for output
  which is then connected to the commands stderr. By default, the
  stderr is the same as that of the calling script.
  """

The implementation seems to work for most cases. Not all signals which
cause the parent to die are distributed properly to the childs.

Any comments?

  Harald Kirsch

-- 
----------------+------------------------------------------------------
Harald Kirsch   | kirschh at lionbioscience.com | "How old is the epsilon?"
LION Bioscience | +49 6221 4038 172          |        -- Paul Erdös
       *** Please do not send me copies of your posts. ***