Python, subprocess, dump, gzip and Cron

[Aidan]

Sebastian "lunar" Wiesner wrote:
>  Aidan <aweraw at gmail.com> at Dienstag 10 Juni 2008 07:21:
> 
>> TT wrote:
>>> On Jun 10, 2:37 pm, Aidan <awe... at gmail.com> wrote:
>>>> Hi,
>>>>
>>>> I'm having a bit of trouble with a python script I wrote, though I'm not
>>>> sure if it's related directly to python, or one of the other software
>>>> packages...
>>>>
>>>> The situation is that I'm trying to create a system backup script that
>>>> creates an image of the system, filters the output though gzip, and then
>>>> uploads the data (via ftp) to a remote site.
>>>>
>>>> The problem is that when I run the script from the command line, it
>>>> works as I expect it, but when it is run by cron I only get a 20 byte
>>>> file where the compressed image should be...  does anyone have any idea
>>>> as to why this might be happening?  Code follows
>>>>
>>>> <code>
>>>>
>>>> #!/usr/bin/python
>>>>
>>>> from subprocess import PIPE, Popen
>>>> from ftplib import FTP
>>>>
>>>> host = 'box'
>>>>
>>>> filename = '%s.img.gz' % host
>>>> ftp_host = '192.168.1.250'
>>>> ftpuser, ftppass = 'admin', 'admin'
>>>> dest_dir = '/share/%s' % host
>>>>
>>>> dump = Popen('dump 0uaf - /',shell=True,stdout=PIPE)
> You should avoid the use of ``shell=True`` here and use a argument list
> instead:
> 
> dump = Popen(['dump', '0uaf', '-', '/'], stdout=PIPE)
> 
> This results in an exception thrown if the executable doesn't exist.  This
> exception can be caught and handle for instance with the logging module.
> 

thanks.  That exception certainly would have helped me...

>>>> gzip = Popen('gzip',shell=True,stdin=dump.stdout,stdout=PIPE)
> 
> Same here, but why don't you use the gzip functionality from the standard
> library?

is there a way I can create a gzip file-like object which can read the 
output from the dump subprocess, and has a read method which outputs the 
compressed data, which will not write to disk first?  With the above 
code python doesn't have to write the system image data to disk at all, 
which helps when there is not enough disk space to hold an intermediate 
image (at least, that is my understanding of it...).

I had a look at the gzip module, but eventually just fell back to using 
the stdin and stdout of a gzip subprocess.  I'd be interested to know 
how it could be done using the python standard lib though.