Issue combining gzip and subprocess

[Piet van Oostrum]

>>>>> Iwan Vosloo <iwan at reahl.org> (IV) wrote:

>IV> Hi there,
>IV> We tried to gzip the output of a shell command, but this results in a
>IV> strange error: the resulting file seems to be the concatenation of the
>IV> plaintext file with the zipped content.

>IV> For example:

>IV> f = gzip.open(filename, 'w')
>IV> subprocess.check_call(['ls','-la'], stdout=f)
>IV> f.close()

>IV> Using a normal file works as expected, but a GzipFile results in a file
>IV> containing what looks like the unzipped data, followed by the zipped
>IV> data.

>IV> I suspect this may have something to do with limitations of GzipFile
>IV> such as it not implementing truncate().

>IV> Does anyone have an explanation / can you suggest a nice solution for
>IV> doing what we are trying to do?

stdout (and the others) must be None, PIPE or a real file object or file
descriptor, not a file like object. In your case the solution would be
to use PIPE, and then read the output and write in to the GzipFile
yourself. 

f = gzip.open(filename, 'w')
proc = subprocess.Popen(['ls','-la'], stdout=subprocess.PIPE)
while True:
    line = proc.stdout.readline()
    if not line: break
    f.write(line)
f.close()

-- 
Piet van Oostrum <piet at cs.uu.nl>
URL: http://pietvanoostrum.com [PGP 8DAE142BE17999C4]
Private email: piet at vanoostrum.org