Need feedback on subprocess-using function
rylesny at gmail.com
Mon Oct 5 11:29:38 CEST 2009
On Oct 4, 9:46 pm, Nobody <nob... at nowhere.com> wrote:
> On Sat, 03 Oct 2009 13:21:00 +0000, gb345 wrote:
> > I'm relatively new to Python, and I'm trying to get the hang of
> > using Python's subprocess module. As an exercise, I wrote the Tac
> > class below, which can prints output to a file "in reverse order",
> > by piping it through the Unix tac utility. (The idea is to delegate
> > the problem of managing the memory for an arbitrarily large task
> > to tac.)
> > self.pipe = subprocess.Popen(['tac'], stdout=out,
> > stdin=subprocess.PIPE,
> > stderr=subprocess.PIPE)
> > This works OK, as far as I can tell, but I'm not sure that I've
> > dotted all the i's and crossed all the t's... E.g., I had to add
> > the line "p.stdin.close()" to the close method when I when I ran
> > into sporadic deadlock at the p.stderr.read() statement. Are there
> > other similar problems lurking in this code?
> Yep. If the process writes more than a buffer-full of data to stderr, it
> will deadlock. tac will block trying to write to stderr, and won't be
> reading its stdin, so your program will block trying to write to tac.
> This is why the POSIX popen() call only lets you attach a pipe to stdin or
> stdout but not both.
> If you want a "double-ended" slave process, you need to use polling or
> non-blocking I/O or asynchronous I/O or multiple threads. I'm not aware of
> any single solution which works on all platforms.
> The easiest way around this problem is to redirect stderr to a temporary
> file and read in the file's contents in the close() method.
There is also Popen.communicate():
More information about the Python-list