[Python-Dev] 2.0 Optimization & speed

[Vladimir Marangozov]

Continuing my impressions on the user's feedback to date: Donn Cave
& MAL are at least two voices I've heard about an overall slowdown
of the 2.0b1 release compared to 1.5.2. Frankly, I have no idea where
this slowdown comes from and I believe that we have only vague guesses
about the possible causes: unicode database, more opcodes in ceval, etc.

I wonder whether we are in a position to try improving Python's
performance with some `wise quickies' in a next beta. But this raises
a more fundamental question on what is our margin for manoeuvres at this
point. This in turn implies that we need some classification of the
proposed optimizations to date.

Perhaps it would be good to create a dedicated Web page for this, but
in the meantime, let's try to build a list/table of the ideas that have
been proposed so far. This would be useful anyway, and the list would be
filled as time goes.

Trying to push this initiative one step further, here's a very rough start
on the top of my head:

Category 1: Algorithmic Changes

These are the most promising, since they don't relate to pure technicalities
but imply potential improvements with some evidence.
I'd put in this category:

- the dynamic dictionary/string specialization by Fred Drake
  (this is already in). Can this be applied in other areas? If so, where?

- the Python-specific mallocs. Actually, I'm pretty sure that a lot of
  `overhead' is due to the standard mallocs which happen to be expensive
  for Python in both space and time. Python is very malloc-intensive.
  The only reason I've postponed my obmalloc patch is that I still haven't
  provided an interface which allows evaluating it's impact on the
  mem size consumption. It gives noticeable speedup on all machines, so
  it accounts as a good candidate w.r.t. performance.

- ??? (maybe some parts of MAL's optimizations could go here)

Category 2: Technical / Code optimizations

This category includes all (more or less) controversial proposals, like

- my latest lookdict optimizations (a typical controversial `quickie')

- opcode folding & reordering. Actually, I'm unclear on why Guido
  postponed the reordering idea; it has received positive feedback
  and all theoretical reasoning and practical experiments showed that
  this "could" help, although without any guarantees. Nobody reported
  slowdowns, though. This is typically a change without real dangers.

- kill the async / pending calls logic. (Tim, what happened with this
  proposal?)

- compact the unicodedata database, which is expected to reduce the
  mem footprint, maybe improve startup time, etc. (ongoing)

- proposal about optimizing the "file hits" on startup.

- others?

If there are potential `wise quickies', meybe it's good to refresh
them now and experiment a bit more before the final release?

-- 
       Vladimir MARANGOZOV          | Vladimir.Marangozov@inrialpes.fr
http://sirac.inrialpes.fr/~marangoz | tel:(+33-4)76615277 fax:76615252