Most efficient way to write data out to a text file?
BPettersen at NAREX.com
Sat Jun 29 21:06:01 CEST 2002
> From: Dave Kuhlman [mailto:dkuhlman at rexx.com]
> candiazoo at attbi.com wrote:
> Thank you for posting this. Very helpful.
> Could we have a bit of evaluation and explanation please. I'm
> suspicious about whether some of these changes actually made a
> difference. I'd like a little understanding before I start
> changing my Python coding style.
[snip optimization involving removing free vars, and string concat]
Looking up a name in globals or builtin is very expensive compared to
looking it up in local scope (use the dis module to see the opcodes
involved -- anything that has LOAD_GLOBAL in it is potentially slow).
Most experienced Python programmers will therefore use something like:
def foo(arg1, arg2, func=MyModule.func):
after they have profiled the program and found foo to be a bottleneck.
Similarly, looking up an attribute of an object involves searching its
__dict__. If you save the returned value in a local, you basically get a
fast "load from array" operation instead.
When it comes to string concatenation it is an O(n**2) operation, so
it's usually better to do:
result = 
append = result.append
for i in xrange(100000):
append(`i`) # `i` is faster than repr(i)
I would say the string concat optimization is always a good idea, the
global->local optimization is ok as long as it doesn't interfer with
readability -- other people will surely disagree <wink>.
More information about the Python-list