Most efficient way to write data out to a text file?

Emile van Sebille emile at
Thu Jun 27 10:54:42 EDT 2002

candiazoo at
> I am a newbie... this is my first Python "project" so I am probably
> horribly inefficient code...  I have never used the profiler but I'll
try it!
> I am not opening/closing the file each time.  I am extracting 700000
rows from a
> mysql database, extracting additional data from our primary, Oracle
database per
> row, then stuffing each piece of data into a class which preformats
the data (I
> need to output the data into a fixed format string/record for another
> application which reads them) and returns a single string... which I
write out
> to the file.

This is not the most efficient way, but for comparison purposes this
wrote almost 3 million records using ~400Mb in a minute:

>>> rec = 'this is a test'*10
>>> import time
>>> def test(rec):
...     global count
...     t = time.time()+60
...     while time.time() < t:
...             outfile.write(rec)
...             count += 1
>>> count = 0
>>> outfile = open(r'f:\test.out', 'w')
>>> test(rec)
>>> print count


Emile van Sebille
emile at


More information about the Python-list mailing list