[Python-Dev] optimization required: .format() is much slower than %

[Nick Coghlan]

Following Terry's suggestion, I repeated Christian's tests with the 
lookup of the format method taken outside the loop (since one advantage 
of the new approach is to make that micro-optimisation easier), and also 
tried a few additional examples.

First, there seems to be about a 0.25 microsecond hit (on my machine) 
which reflects the impact of the lookup of the "format" attribute itself.

Secondly, the string % operator appears to have an explicit optimisation 
for the 'just return str(self)' case. This optimisation is missing from 
the new string format method.

The only optimisation idea I came up with (other than the addition of a 
tp_format slot) is for string objects to be able to cache their 
(lookup:subformat) pairs rather than having to parse themselves every 
time. That has obvious memory consumption implications though.

Cheers,
Nick.

Test results:

# No formatting at all, just the method invocation:
$ ./python -m timeit "''.format()"
1000000 loops, best of 3: 0.635 usec per loop
$ ./python -m timeit -s "fmt = '%s'.format" "fmt()"
1000000 loops, best of 3: 0.48 usec per loop
$ ./python -m timeit "'' % ()"
1000000 loops, best of 3: 0.389 usec per loop

# Format one argument by position
$ ./python -m timeit "'{0}'.format('nothing')"
1000000 loops, best of 3: 1.56 usec per loop
$ ./python -m timeit -s "fmt = '{0}'.format" "fmt('nothing')"
1000000 loops, best of 3: 1.44 usec per loop
$ ./python -m timeit "'%s' % 'nothing'"
10000000 loops, best of 3: 0.0736 usec per loop

# Format one non-string argument by position
$ ./python -m timeit "'{0}'.format(0)"
100000 loops, best of 3: 1.74 usec per loop
$ ./python -m timeit -s "fmt = '{0}'.format" "fmt(0)"
1000000 loops, best of 3: 1.51 usec per loop
$ ./python -m timeit "'%s' % 0"
10000000 loops, best of 3: 0.0987 usec per loop

# Format one argument by position with additional text
$ ./python -m timeit "'some text with {0}'.format('nothing')"
1000000 loops, best of 3: 1.7 usec per loop
$ ./python -m timeit -s "fmt = 'some text with {0}'.format" "fmt('nothing')"
1000000 loops, best of 3: 1.45 usec per loop
$ ./python -m timeit "'some text with %s' % 'nothing'"
1000000 loops, best of 3: 0.612 usec per loop

# Format two arguments by position with additional text
$ ./python -m timeit "'some text with {0} {1}'.format('nothing', 'more')"
100000 loops, best of 3: 2.63 usec per loop
$ ./python -m timeit -s "fmt = 'some text with {0} {1}'.format" 
"fmt('nothing', 'more')"
100000 loops, best of 3: 2.31 usec per loop
$ ./python -m timeit "'some text with %s %s' % ('nothing', 'more')"
1000000 loops, best of 3: 0.804 usec per loop

# Format two non-string arguments by position with additional text
$ ./python -m timeit "'some text with {0} {1}'.format(0, 1)"
100000 loops, best of 3: 2.81 usec per loop
$ ./python -m timeit -s "fmt = 'some text with {0} {1}'.format" "fmt(0, 1)"
100000 loops, best of 3: 2.46 usec per loop
$ ./python -m timeit "'some text with %s %s' % (0, 1)"
1000000 loops, best of 3: 0.851 usec per loop

# Format two non-string arguments by name with additional text
$ ./python -m timeit -s "d = dict(a=0, b=1)" "'some text with {a} 
{b}'.format(**d)"
100000 loops, best of 3: 2.81 usec per loop
$ ./python -m timeit -s "d = dict(a=0, b=1)" -s "fmt = 'some text with 
{a} {b}'.format" "fmt(**d)"
100000 loops, best of 3: 2.58 usec per loop
$ ./python -m timeit -s "d = dict(a=0, b=1)" "'some text with %(a)s 
%(b)s' % (d)"
1000000 loops, best of 3: 1 usec per loop

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------
             http://www.boredomandlaziness.org