[Python-Dev] optimization required: .format() is much slowerthan %

[Terry Reedy]

"Gregory P. Smith" <greg at krypto.org> wrote in message 
news:52dc1c820805261717t3842df91ubc44111872d19b98 at mail.gmail.com...
| On Mon, May 26, 2008 at 3:43 PM, Christian Heimes <lists at cheimes.de> 
wrote:
| > Dear fellow Python developers!
| >
| > Ten minutes ago I raised a concern about speed differences between the
| > old style % formatting and the new .format() code. Some quick
| > benchmarking from Benjamin and me showed, that it's even worse than I
| > expected.
| >
| > $ ./python -m timeit "'%s'.format('nothing')"
| > 100000 loops, best of 3: 2.63 usec per loop
| > $ ./python -m timeit "'%s' % 'nothing'"
| > 10000000 loops, best of 3: 0.188 usec per loop
| >
| > $ ./python -m timeit "'some text with {0}'.format('nothing')"
| > 100000 loops, best of 3: 4.34 usec per loop
| > $ ./python -m timeit "'some text with %s' % 'nothing'"
| > 100000 loops, best of 3: 2.04 usec per loop
| >
| > $ ./python -m timeit "'some text with {0} {1}'.format('nothing', 
'more')"
| > 100000 loops, best of 3: 6.77 usec per loop
| > $ ./python -m timeit "'some text with %s %s' % ('nothing', 'more')"
| > 100000 loops, best of 3: 2.22 usec per loop
| >
| > As you can clearly see the new .format() code is *much* slower than the
| > old style % code. I recommend we spend some time on optimizing common
| > code paths of the new .format() code.
| >
| > As first step I propose the move the __format__ method to a new type
| > slot. __format__() is called for every object. My gut feeling says that
| > a slot method is going to speed up the most common usage
| > "{0}".format(some_string).
|
| My gut feels the same way...  How about seeing the profile data of the
| new vs old string formatting functions first as a comparison.  If
| those disagree vs the above then the ".format()" name lookup

In real usages, the format string is typically a constant.  To mirror this,
the .format name lookup and creation of the bound method object could be 
factored out of the testing loop and put in a setup string.  IOW, I write 
code like

emsg = 'Bad input {0}'.format
for i in inputs:
  if squiggle(i): emsg(i)
  else: realwork(i)

However, if I have done the following correctly (my first use of timeit), 
this seems not to be the slowdown.

>>> t=timeit.Timer(stmt="'{0}'.format(None)")
>>> t.timeit()
3.769231005616632
>>> t2=timeit.Timer(setup="m='{0}'.format", stmt='m(None)')
>>> t2.timeit()
3.7023311622007213

tjr