My motivation came from an instance when I was using subprocess.Popen for a Linux / Windows cross platform program. In part of the program, I was writing and reading to a cron like object. On Windows, it was a text file and on Linux it would be the crontab executable. Had I been able to substitute the "open()" function with my wrapper, it would have been the only change I had to make for cross platform compatibility; instead of having to change numerous lines because Linux would need Popen and Windows would need a regular file open(), I could simply make it so that if the platform was Linux, my wrapper is used in place of that. Just another example would be having an external program decrypt a file that can be in plain text or encrypted that might go something like this:<br>
<br>if encryptionEnabled:<br> fileobj = subprocess.ProcessIOWrapper("gpg --decrypt supersecret.html.gpg")<br>else:<br> fileobj = open("notsosecret.html")<br><br>From there, the functions would not have to be modified despite using a running process versus a file object.<br>
<br><div class="gmail_quote">On Tue, Jul 28, 2009 at 01:53, Gregory P. Smith <span dir="ltr"><<a href="mailto:greg@krypto.org" target="_blank">greg@krypto.org</a>></span> wrote:<br><blockquote class="gmail_quote" style="border-left: 1px solid rgb(204, 204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;">
<div>On Mon, Jul 27, 2009 at 5:32 PM, Glyph Lefkowitz<<a href="mailto:glyph@twistedmatrix.com" target="_blank">glyph@twistedmatrix.com</a>> wrote:<br>
> On Mon, Jul 27, 2009 at 3:04 PM, Paul Moore <<a href="mailto:p.f.moore@gmail.com" target="_blank">p.f.moore@gmail.com</a>> wrote:<br>
>><br>
>> I like MRAB's idea of using a (non-standard) "e" flag to include<br>
>> stderr. So "r" reads from stdout, "re" reads from stdout+stderr.<br>
>><br>
>> Anything more complicated probably should just use "raw" Popen<br>
>> objects. Don't overcomplicate the interface.<br>
><br>
> In my opinion, mangling stderr and stdout together is already an<br>
> overcomplication. It shouldn't be implemented.<br>
><br>
> It seems like a good idea, until you realize that subtle changes to your OS,<br>
> environment, or buffering behavior may result in arbitrary, unparseable<br>
> output.<br>
<br>
</div>Agreed. Leave stderr support out of this. People who need stderr<br>
should use the full subprocess.Popen interface. Quick hack unixy<br>
types will just run their process using a shell (which already seems<br>
to be the default based on the "ls -l" example) and add 2>&1. This<br>
functionality is basically the equivalent of adding the | symbol on<br>
either or both ends of a filename given to open() in perl. (but<br>
without being so gross).<br>
<br>
I do wonder why you're trying to make it behave exactly like open()<br>
including the mode= syntax.<br>
<br>
Why not just define several names based on the behavior?<br>
<br>
ProcessReadWrapper()<br>
ProcessWriteWrapper()<br>
ProcessReadWriteWrapper()<br>
<br>
-gps<br>
<div><div></div><div><br>
><br>
> For example, let's say you've got a program whose output is a list of lines,<br>
> each one containing a number. Sometimes it tries to import gtk, and fails<br>
> to open its display.<br>
><br>
> That's fine, and you can still deal with it, as long as the interleaved<br>
> output looks like this:<br>
><br>
> 100<br>
> 200<br>
> Gtk-WARNING **: cannot open display:<br>
> 300<br>
> 400<br>
><br>
> but of course the output might (although unlikely with such small chunks of<br>
> output) end up looking like this, instead:<br>
><br>
> 100<br>
> 2Gtk-WAR0NING0 **:<br>
> can30not 0open display:<br>
><br>
> 400<br>
><br>
> this is the sort of thing which is much more likely to happen once you start<br>
> dealing with large volumes of data, where there are more page-boundaries for<br>
> your buffers to get confused around, and you are playing with buffering<br>
> options to improve performance. In other words, it's something that fails<br>
> only at scale or under load, and is therefore extremely difficult to debug.<br>
><br>
> This option might be okay if it were allowed only on subprocesses opened in<br>
> a text mode, and if the buffering logic involved forced stderr and stdout to<br>
> be line-delimited, and interleave only lines, rather than arbitrary chunks<br>
> of bytes. Of course then if you use this flag with a program that outputs<br>
> binary data with no newlines it will buffer forever and crash your program<br>
> with a MemoryError, but at least that's easy to debug when it happens.<br>
><br>
</div></div>> _______________________________________________<br>
> Python-Dev mailing list<br>
> <a href="mailto:Python-Dev@python.org" target="_blank">Python-Dev@python.org</a><br>
> <a href="http://mail.python.org/mailman/listinfo/python-dev" target="_blank">http://mail.python.org/mailman/listinfo/python-dev</a><br>
> Unsubscribe:<br>
> <a href="http://mail.python.org/mailman/options/python-dev/greg%40krypto.org" target="_blank">http://mail.python.org/mailman/options/python-dev/greg%40krypto.org</a><br>
><br>
><br>
</blockquote></div><br>