How to read large amounts of output via popen

Gabriel Genellina gagsl-py2 at
Fri Aug 6 12:02:07 CEST 2010

En Fri, 06 Aug 2010 06:06:29 -0300, loial <jldunn2000 at> escribió:

> I need to read a large amount of data that is being returned in
> standard output by a shell script I am calling.
> (I think the script should really be writing to a file but I have no
> control over that)
> Currently I have the following code. It seeems to work, however I
> suspect this may not work with large amounts of standard output.
> What is the best way to read a large amount of data from standard
> output and write to a file?
> Here is my code.
> process=subprocess.Popen(['myscript', 'param1'],
> shell=False,stdout=subprocess.PIPE,stderr=subprocess.PIPE)
> cmdoutput=process.communicate()
> myfile = open('/home/john/myoutputfile','w')
> myfile.write(cmdoutput[0])
> myfile.close()

If all you do with the process' output is to write it to the output file,  
you can avoid the intermediate step:

myfile = open('/home/john/myoutputfile','w')
myerror = open('/home/john/myerrorfile','w')
process=subprocess.Popen(['myscript', 'param1'],


Gabriel Genellina

More information about the Python-list mailing list