[pypy-dev] 1.4.1 threading

[Carl Friedrich Bolz]

On 12/27/2010 07:00 AM, Dima Tisnek wrote:
> (dan) random binary trees: O(log2(n)) is 7.2 for builtins, though the
> constant factor for locks, etc, might make it worthwhile
>
> (paolo) non blocking hash maps: memory barriers can be quite costly too
>
> different options will have to be implemented and tested when we get there,
> speaking on which, is there a test dict load?
> does anyone have some profiling data, what portion of runtime is spent
> reading and writing attributes and globals anyways?
>
> while we are on the subject, is there a plan to provide different
> levels of sparseness for different levels of name lookup?
> for example globals vs builtins, first needs to be quite sparce so
> that builtins show through well, second hardly, because there's
> nowhere else to look if builtins don't have the name.
> then the tradeoff could be more dynamic, that is frequently accessed
> dicts could be e.g. more sparse and rarely accessed more compact?
> (obviousely more sparse is not always better, e.g. in terms of cpu cache)
> of course "frequently accessed" is not as easy as frequently ran code,
> e.g. here "class A: ...; while 1: A().func()", func lookup occurs in
> different objects, yet it is same logical operation.
>
> come to think of it, is there any point in polymorphic dicts, e.g.
> attribute access could be imeplemented as a near-perfect compact hash
> map if attribute names change rarely, while regular dict()s are
> expected to change keys often.

All these thoughts go into the wrong direction, imo. The JIT removes 
nearly all dictionary accesses to global dicts, instance and class 
dicts. Even without the JIT, purely interpreting things, there are 
caches that bypass most dict lookups.

The only case where improving dicts would help is for user-defined 
dictionaries, i.e. when you write dicts with curly braces. that is 
basically very hard, because the usage patterns vary widely and a lot of 
work would be necessary to find common uses.

Cheers,

Carl Friedrich