[Python-Dev] gc ideas -- sparse memory

[Dima Tisnek]

Oh my bad, I must've confused python with some research paper.
Unique id is not so hard to make without an address.

While on this topic, what is the real need for unique ids?
Also I reckon not all objects need a unique id like this, e.g.
interned strings, simple data types and hashable and comparable
objects could perhaps survive without unique id?

On 3 December 2010 16:04, Terry Reedy <tjreedy at udel.edu> wrote:
> On 12/3/2010 5:55 PM, Dima Tisnek wrote:
>>
>> How hard or reasonable would it be to free memory pages on OS level?
>>
>> [pcmiiw] Gabage collection within a generation involves moving live
>> objects to compact the generation storage. This separates the memory
>> region into 2 parts "live" and "cleared", the pointer to the beginning
>> of the "cleared" part is where next allocation is going to happen.
>>
>> When this is done, does Python gc move the objects preserving order or
>> does it try to populate garbaged slot with some live object
>> disregarding order? Earlier case is more applicable, but latter case
>> is a target for below too.
>>
>> If we were to look at memory regions from the OS point of view, they
>> are allocated as pages (or possibly as hugetlb pages). So if we are to
>> compact something like this [LL__][_L__][____][L_LL][LFFF]  where []
>> is a page, L is live object and _ is garbage and F is free memory,
>> would it not make more sense to tell OS that [____] is not needed
>> anymore, and not move some of the consequtive [L_LL][LFFF] at all, or
>> at least not move those objects as far down the memory region?
>>
>> This would of course have a certain overhead of tracking which pages
>> are given back to OS and mapping them back when needed, but at the
>> same time, be beneficial because fewer objets are moved and also
>> possibly improve cpu cache performance because objects won't be moved
>> so far out.
>>
>> p.s. if someone has an athoritative link to modern python gc design,
>> please let me know.
>
> gc is implementation specific. CPython uses ref counting + cycle gc. A
> constraint on all implementations is that objects have a fixed, unique id
> during their lifetime. CPython uses the address as the id, so it cannot move
> objects. Other implementations do differently. Compacting gc requires an id
> to current address table or something.
>
> --
> Terry Jan Reedy
>
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> http://mail.python.org/mailman/options/python-dev/dimaqq%40gmail.com
>