subprocess.Popen and ordering writes to stdout and stderr

[exarkun at twistedmatrix.com]

On 09:56 pm, chris at simplistix.co.uk wrote:
>exarkun at twistedmatrix.com wrote:
>>>How can I get this to be the case?
>>
>>You probably just need to flush stdout and stderr after each write. 
>>You set them up to go to the same underlying file descriptor, but they 
>>still each have independent buffering on top of that.
>
>Okay, but if I do:
>
>os.system(sys.executable+' '+path)
>
>...with test.py as-is, I get things in the correct order.

libc is probably giving you line buffering when you use os.system 
(because the child process inherits the parent's stdio, and the parent's 
stdio is probably a pty, and that's the policy libc implements).

When you use subprocess.Popen, the child process gets a pipe (ie, not a 
pty) for its stdout/err, and libc gives you block buffering instead.

This makes the difference, since your test writes all end with \n - 
flushing the libc buffer when it's in line buffered mode, but not 
otherwise.

Try the os.system version with the parent process's stdio not attached 
to a pty (say, 'cat | program | cat') or try giving the subprocess.Popen 
version a pty of its own (I'm not sure how you do this with 
subprocess.Popen, though).  You should be able to observe the behavior 
change based on this.

Jean-Paul