On Mon, Jul 27, 2009 at 3:04 PM, Paul Moore <p.f.moore@gmail.com> wrote:
I like MRAB's idea of using a (non-standard) "e" flag to include
stderr. So "r" reads from stdout, "re" reads from stdout+stderr.

Anything more complicated probably should just use "raw" Popen
objects. Don't overcomplicate the interface.

In my opinion, mangling stderr and stdout together is already an overcomplication.  It shouldn't be implemented.

It seems like a good idea, until you realize that subtle changes to your OS, environment, or buffering behavior may result in arbitrary, unparseable output.

For example, let's say you've got a program whose output is a list of lines, each one containing a number.  Sometimes it tries to import gtk, and fails to open its display.

That's fine, and you can still deal with it, as long as the interleaved output looks like this:

Gtk-WARNING **: cannot open display:

but of course the output might (although unlikely with such small chunks of output) end up looking like this, instead:

2Gtk-WAR0NING0 **:
can30not 0open display:


this is the sort of thing which is much more likely to happen once you start dealing with large volumes of data, where there are more page-boundaries for your buffers to get confused around, and you are playing with buffering options to improve performance.  In other words, it's something that fails only at scale or under load, and is therefore extremely difficult to debug.

This option might be okay if it were allowed only on subprocesses opened in a text mode, and if the buffering logic involved forced stderr and stdout to be line-delimited, and interleave only lines, rather than arbitrary chunks of bytes.  Of course then if you use this flag with a program that outputs binary data with no newlines it will buffer forever and crash your program with a MemoryError, but at least that's easy to debug when it happens.