Most efficient way to write data out to a text file?

candiazoo at candiazoo at
Thu Jun 27 10:09:09 EDT 2002

I am a newbie... this is my first Python "project" so I am probably writing
horribly inefficient code...  I have never used the profiler but I'll try it!

I am not opening/closing the file each time.  I am extracting 700000 rows from a
mysql database, extracting additional data from our primary, Oracle database per
row, then stuffing each piece of data into a class which preformats the data (I
need to output the data into a fixed format string/record for another
application which reads them) and returns a single string... which I write out
to the file.


On Wed, 26 Jun 2002 23:56:09 -0400, Peter Hansen <peter at> wrote:

>candiazoo at wrote:
>> I assume some sort of block i/o or preallocating a large block of file space
>> would be the best way to do it?  I am outputting about 700,000 records to a text
>> file and my program only spits out about 20 records per second.  This is using
>> standard file open and write...
>You're not opening and closing the file for each record that is written, are you?
>Can you post more detail on the amount of data in a record, and the format?
>Why not profile the code using the module and actually see
>where the problem is?

More information about the Python-list mailing list