[Python-Dev] Looking for master thesis ideas involving Python

[Michael Hudson]

Armin Rigo <arigo at tunes.org> writes:

> Hello Martin,
>
> On Sun, Nov 02, 2003 at 08:05:55PM +0100, Martin v. L?wis wrote:
>> > > More than that in the good cases.  Something I forgot was that you'd
>> > > probably have to knock variable length types on the head.
>> > 
>> > Why?
>> 
>> Assuming "to knock on the head" means "to put an end to":
>> 
>> If you put all objects of the same type into a pool, you really want
>> all objects to have the same side, inside a pool. With that
>> assumption, garbage objects can be reallocated without causing
>> fragmentation. If objects in a pool have different sizes, it is not
>> possible to have an efficient reallocation strategy.
>
> "Not easy" would have been more appropriate.  It is still basically what
> malloc() does.

Well, yeah, but as Tim said pymalloc gets its wins from assuming that
each allocation is the same size.  You could combine my idea with some
other allocation scheme, certainly, but given the relative paucity of
variable length types and the reduction in allocator overhead using
something like pymalloc gives us, I think it might just be easier to
not do them any more.  Of course, I don't see myself having any time
to play with this idea any time soon, and it's probably not really
beefy enough to get a masters thesis from, so maybe we'll never know.

> One way would be to use Python's current memory allocator, by
> adapting it to sort objects into pools not only according to size
> but also according to type.

That's pretty much what I was suggesting.

> What seems to me like a good solution would be to use one relatively
> large "arena" per type and Python's memory allocator to subdivide
> each arena.  If each arena starts at a pointer address which is
> properly aligned, then *(p&MASK) gives you the type of any object,
> and possibly even without much cache-miss overhead because there are
> not so many arenas in total (probably only 1-2 per type in common
> cases, and arenas can be large).

Hmm, maybe.  I'm not going to make guesses about that one :-)

Cheers,
mwh

-- 
  ... Windows proponents tell you that it will solve things that
  your Unix system people keep telling you are hard.  The Unix 
  people are right: they are hard, and Windows does not solve 
  them, ...                            -- Tim Bradshaw, comp.lang.lisp