[Python-Dev] A word from the author (was "pymalloc", was "fun ", was "2.1 slowe r than 2.0")

[Vladimir Marangozov]

[Tim]
> 
> Help us out a little more, briefly.  The last time you 
> mentioned obmalloc on
> Python-Dev was:
> 
>     Date: Fri, 8 Sep 2000 18:23:13 +0200 (CEST)
>     Subject: [Python-Dev] 2.0 Optimization & speed
>     > ...
>     > The only reason I've postponed my obmalloc patch is that I
>     > still haven't provided an interface which allows evaluating
>     > it's impact on the mem size consumption.
> 
> Still a problem in your eyes?

Not really. I think obmalloc is a win w.r.t. both space & speed.
I was aiming at evaluating precisely how much we win with the help
of the profiler, then tune the allocator even more, but this is
OS dependant anyway and most people don't dig so deep. I think
they don't need to either, but it's our job to have a good
understanding of what's going on.

In short, you can go for it, opt-in, without fear.

Not opt-out, though, because of custom object's thread safety.

Thread safety is a problem. Current extensions implement custom
object constructors & destructors safely, because they use (at the
end of the macro chain, today) the system allocator which is
thread safe. Switching to a thread unsafe allocator by default is
risky because this may inject bugs in existing working extensions.
Although the core objects won't be affected by this change because
of the interpreter lock protection, we have no provisions so far
for custom object's thread safety.

> 
> I expect we'll have to do the dance of evaluating it with and 
> without locks regardless

See above -- it's not about speed, it's about safety.

> BTW, obmalloc has some competition.  Hans Boehm popped up on 
> c.l.py last week, transparently trying to seduce Neil Schemenauer
> into  devoting his life to making the BDW collector make Python's
> refcounting look like a cheap Dutch trick <wink>:
> 
>    http://www.deja.com/getdoc.xp?AN=722453837&fmt=text

Yes, I saw that. Hans is speaking from experience, but a non-Python
one <wink>. I can hardly agree with the idea of dropping RC (which
is the best strategy towards expliciteness and fine-grain control of
the object's life-cycles) and replacing it with some collector beast
(whatever its nature). We'll loose control for unknown benefits.
We're already dealing with the major pb of RC (cycle garbage) in
an elegant way which is complementary to RC.

Saying that we're probably dirtying more cache lines than we should
in concurrent scenarios is ... an opinion. My opinion is that this
is not really our problem <wink>.

If Hans were really right, Microsoft would have already plugged his
collector in Windows, instead of using RC. And we all know that MS
is unbeatable in providing efficient, specialized implementations
for Windows, tuned for the processors Windows in running on <wink>.

On a personal note, after 3 months in Munich, I am still intrigued
by the many cheap Dutch tricks I see every day on my way, like
the latest Mercs, BMWs, Porsches or Audis, to name a few <wink>.

can't-impress-them-with-my-Ford-<wink>'ly y'rs

Vladimir