Most efficient way to write data out to a text file?

[Emile van Sebille]

candiazoo at attbi.com
> I am a newbie... this is my first Python "project" so I am probably
writing
> horribly inefficient code...  I have never used the profiler but I'll
try it!
>
> I am not opening/closing the file each time.  I am extracting 700000
rows from a
> mysql database, extracting additional data from our primary, Oracle
database per
> row, then stuffing each piece of data into a class which preformats
the data (I
> need to output the data into a fixed format string/record for another
> application which reads them) and returns a single string... which I
write out
> to the file.
>

This is not the most efficient way, but for comparison purposes this
wrote almost 3 million records using ~400Mb in a minute:

>>> rec = 'this is a test'*10
>>> import time
>>> def test(rec):
...     global count
...     t = time.time()+60
...     while time.time() < t:
...             outfile.write(rec)
...             count += 1
...
>>> count = 0
>>> outfile = open(r'f:\test.out', 'w')
>>> test(rec)
>>> print count
2982620


--

Emile van Sebille
emile at fenx.com

---------