Getting stdout and stderr from popen

[Jonathan Gardner]

Stefan Heimann scribbled with his keyboard:
> I want to launch an external process and have stdout and stderr
> redirected to one output stream,

You are on Unix, right?

> something like
> 
> f = os.popen('cvs status 2>&1')
> 
> does the right thing, but does not work on all platforms.
> 
> If I write
> 
> pin, pout, perr = os.popen3('cvs status')
> 
> I get the output of stderr and stdout but not in the right order.

What do you mean in the right order? Do you mean that it is returning in, 
err, out or that err and out are spitting out stuff simultaneously?

> 
> I tried also
> 
> sys.stderr = sys.stdout
> f = os.popen('cvs status')
> f.read()
> 
> but still I get only the output from stdout (what cvs outputs on
> stderr is written directly to the console).
> 

Okay, let's talk about what is really happening deep underneath the hood.

These popen functions are a shorthand for a very common thing in the world 
of Unix. It goes something like this:

1) make three pipes (in, out, err)
2) fork.

3) If you are the child (pid == 0) then close the writing part of in, and 
the reading part of out and err.
4) Now, make in be my stdin, out my stdout, err my stderr. (this has to be 
done at a lower level - you need to use the dup type functions.)
5) Now, execute the program I wanted to execute ("cvs status") and exit 
when it is finished. "cvs status" should be reading from and writing to the 
pipes.

3) If you are the parent (pid != 0, pid is the child's pid) then close the 
reading part of in, the writing part of out and err. If you want to send 
something to the child's stdin, write to in. If you want to see what he is 
saying, read from out. If you want to hear what he is crying about, read 
from err. (Yes, I meant to write (3).)
4) go on with your parental role and discipline your child if he misbehaves.

That is what is happening under the hood. There are four things the parent 
gets when he makes a child like this - in, out, err, and the pid.

os.popen plays a funny trick. He only gives us back one file that we can 
either read from or write to. You have to choose one. If you choose the 
write to file, then you are going to be writing to the child's stdin. If 
you choose the read from one, then you will be reading from his stdout. You 
can't  read from stderr from this one. It will use stderr from the parent. 
(Note: this may be different from sys.stderr.)

os.popen2, 3, and 4 do something similar, but they will give you back 
multiple pipes. Read the documentation closely.

My favorite, for when I am a control freak, is the Popen2, 3, or 4 class in 
popen2 module. Take a look at that for how that works.

More questions about pipes and child processes? Take it to a Unix 
programmer. Windows does a horrible job with it, and Windows programmers 
tend to only know about it, but are not able to do much with it.

Jonathan