Most efficient way to write data out to a text file?

[candiazoo at attbi.com]

Basically, Erv, what I did was to try optimize any code referenced in the major
loop of the function that was taking so long.  I ended up getting that one piece
down from 8 hours to 5 hours of processing time.  Still forever, but not any
longer than a VB app that I am replacing took to run...  in fact it is a bit
quicker.  Total run-time of the program is now 7 hours 30 minutes down from much
longer (the original VB program ran for 24 hours, but it also generated a
fulcrum database along with the text files and also pulled back a few hundred
thousand more records for another product we market).  Part of that is replacing
a MS Access database with mySQL.  Part of that is my aforementined
optimizations...

1) "import sys" became "import exit from sys"
2) "import os" became "import makedires from os"
3) "import time" bacame "import asctime from time" (this one is used frequently)
4) I localized constants I was using in boolean tests (there were declared at
the beginning of the module, I moved them to the function call just before the
loop).
5) I localized self.__source_id...  source_id = self.__source_id before using it
in every loop iteration.
6) I localized self.fix_date_range()... fix_date = self.fix_date_range before
using it in every loop iteration/
7) I changed my method that built the output string by changing...

return self.val_a + self.value_b + etc..

...to...

return "%-10.10s%-200.200s (etc)" % (self.val_a, self.val_b, etc.)

...and those were basically the changes I made!  They made a huge difference!

Mike J.