Getting stdout and stderr from popen

[Andrew MacIntyre]

On 6 Mar 2002, Stefan Heimann wrote:

> >|> If I write
> >|>
> >|> pin, pout, perr = os.popen3('cvs status')
> >|>
> >|> I get the output of stderr and stdout but not in the right order.
> >|
> >| What do you mean in the right order? Do you mean that it is returning in,
> >| err, out or that err and out are spitting out stuff simultaneously?
>
> the latter one.

os.popen4() should give you a combined stdout/stderr pipe.  Wouldn't rely
on it to get the order correct though, especially on Windows.

> >
> > Probably not simultaneous, because the C I/O library probably has
> > thread interlocks that prevent it, but if you want a single time
> > ordered stream, you'd need time stamps to reassemble it out of
> > two.  For example, suppose you record the output of "make", which
> > will be both output and error streams.  If you read them separately,
> > you won't be able to tell which compiler errors belong to which
> > make commands.
>
> That's what I need! Is it really so complicated? I think it should be
> common that someone needs to read stdout and stderr ordered by time.

Its somewhat more complicated than appears on the surface for sure,
because you get a whole heap of interactions involving:-
- scheduling of the processes at either end of the pipe(s);
- buffering (if any) of pipe data by the OS;
- buffering (if any) of pipe data by the C runtime library,
to mention some obvious issues.

Because Unix was designed with the intent of using pipes to implement
complex processes from simpler building blocks, using pipes on Unix
usually works in a relatively sane way, although ptys are probably more
appropriate/effective when bidirectional data transfer is required.

The concept of pipes is definitely an afterthought in the Windows world
(yes I know [MP]C-DOS 2 supported them, but it did so by writing actual
temporary files, and I don't think its improved much).

I am fairly sure you won't find a satisfactory way of getting
stdout/stderr output synchronised to your liking on Windows - be nice to
be proved wrong though.

--
Andrew I MacIntyre                     "These thoughts are mine alone..."
E-mail: andymac at bullseye.apana.org.au  | Snail: PO Box 370
        andymac at pcug.org.au            |        Belconnen  ACT  2616
Web:    http://www.andymac.org/        |        Australia