Help replacing os.system call with subprocess call

David Pratt fairwinds at
Mon Apr 7 15:38:47 CEST 2008

Hi David and Matt. I appreciate your help which has got me moving 
forward again so many thanks for your reply. I have been using 
subprocess.Popen a fair bit but this was the first time I had to use 
subprocess to capture large file output. The trouble I was having was 
with the process would just hang. Chunking was the solution. I guess I 
assumed this would be taken care of in the internals.

Overall, I wish subprocess had some better documentation since it is 
definitely not a drop in replacement for os.system. In other 
circumstances I am using for simple calls which works 

The speed of this solution is slower than os.system. Would a queue of 
some kind be needed to speed this up? Has anyone implemented something 
like this? Many thanks.


Matt Nordhoff wrote:
> David Pratt wrote:
>> Hi. I am trying to replace a system call with a subprocess call. I have 
>> tried subprocess.Popen and with but have not been 
>> successful. The command line would be:
>> svnadmin dump /my/repository > svndump.db
>> This is what I am using currently:
>> os.system('svnadmin dump %s > %s' % (svn_dir,
>>      os.path.join(backup_dir, 'svndump.db')))
>> Many thanks.
> Try this:
> import os.path
> import subprocess
> p = subprocess.Popen(
>     ['svnadmin', 'dump', svndir],
>     stdout=subprocess.PIPE,
> )
> fh = open(os.path.join(backup_dir, 'svndump.db'), 'wb')
> while True:
>     chunk =**20) # 1 MB
>     if not chunk:
>         break
>     fh.write(chunk)
> fh.close()
> It reads svnadmin's stdout in 1 MB chunks, in case it's large enough
> that reading the whole thing into RAM at once would be a bad idea.
> No error handling. For one, you might want to add a try...finally to
> ensure that fh will get closed. (Or if you have Python 2.5, use a with
> statement! :-) ) Also, Popen will raise an OSError if svnadmin can't be
> found or something. And this isn't even considering svnadmin erroring out...
> svnadmin's stderr will go to your stderr.
> I didn't test it, but I'm pretty sure it will work. (I spotted a syntax
> error while writing that though.) I don't have much experience with
> Popen's stdio objects, so it's possible you'd need to do something like
> call p.wait() to wait for it to exit before being able to read its stdout.
> It could be slower than the os.system version, since now Python is doing
> all of the I/O, instead of your shell, but I doubt that'll be a big problem.
> (Also, insert suggestion about using a good VCS. ;-) )

More information about the Python-list mailing list