Help! calling tar from Python on Linux

[Donn Cave]

Quoth Preston Landers <prestonlanders at my-deja.com>:
| I'm trying to untar a large file from within a Python script.
| I'm calling tar like this:
|
| filename = "/tmp/data.tar.gz"
| status, result = commands.getstatusoutput("tar xzvf %s" % filename)
| if status:
|    print "error! tar said:\n%s" % result
| else:
|    # continue
|
| The problem is that every time I get an error status from tar, though
| tar seems to completely extract all files.  I get this as the last 2
| lines of output:
|
| gzip: stdout: Broken pipe
| tar: Child returned status 1
|
| If I try to perform this same operation from the command line, it works
| fine.  But if I try to do it in Python, I get this error status.  Has
| anybody seen anything like this?

I think there may be a bug in commands.getstatusoutput.  Here's what I
see in commands.py:

def getstatusoutput(cmd):
    """Return (status, output) of executing cmd in a shell."""
    import os
    pipe = os.popen('{ ' + cmd + '; } 2>&1', 'r')
    text = pipe.read()
    sts = pipe.close()
    if sts == None: sts = 0
    if text[-1:] == '\n': text = text[:-1]
    return sts, text
 
I suspect your problem may go away if you rewrite that function, changing
"text = pipe.read()"  to something like this:

    text = ''
    while 1:
        frag = pipe.read()
        if frag == '':
            break
        text = text + frag

Basically, you need this if your command is going to write more than once
to output.  The way it's written now, that read() catches the first output
and then closes the pipe, so the command will get a SIGPIPE the next time
it writes output.

I can't really explain why "gzip" is getting the SIGPIPE instead of tar,
unless maybe tar catches SIGPIPE.  GNU tar spawns a gzip command to
decompress its input, so if tar survives, it can do to gzip what you
just did to it.

	Donn Cave, University Computing Services, University of Washington
	donn at u.washington.edu