Python 2.2 __slots__ Bug?

Jonathan Hogg jonathan at onegoodidea.com
Thu Jun 27 08:19:58 EDT 2002


On 27/6/2002 1:24, in article afdlv6$bu6$0 at 216.39.172.122, "Bengt Richter"
<bokr at oz.net> wrote:

> Wondering if there's any 12*n byte chunks involved, fitting
> into some 2**n (-overhead) allocation space? Just noticing
> 
>>>> divmod(2**19, 43551)
> (12, 1676)
>>>> divmod(2**16, 5430)
> (12, 376)
> 
> Sometimes these things mean something, sometimes they don't ;-)
> Maybe it will jog someone's thought. E.g., a last-item index/limit
> problem?

It appears that GjR is on the case ;-) but I did some digging into the
source out of curiosity and it's definitely a stack overrun problem. If
anyone is interested, here's my analysis:

At the moment the deallocators for the builtin container types pull a neat
trick to stop the stack getting too deep. As they're recursively
deallocating objects they keep a count of how deep deallocation has gotten
so far. If this gets over a set limit then instead of deallocating the
object it jams it onto a list and returns immediately.

When the stack has unwound back to the top a new piece of magic is invoked
that checks to see if anything was put onto the "delete later" list. If
something was then it grabs the first thing off the list and starts
recursively deleting at that point again.

A hack is used to make this "delete later" list: The object's type is
checked and, since this trick is only used by a few builtin object types, a
numeric type code is stored in the reference count slot of the object (which
is no longer interesting since it must be 0). The type object pointer slot
is then re-used as a pointer to the next item in the "delete later" list.
When the item is pulled off the list and actually deallocated the reverse
trick is pulled to correct the type object pointer and reset the reference
count to 0.

This is all fine except that this trick is not pulled for __slots__
variables as these are not stored in a dict that can be added to the "delete
later" list (the point of slots variables). In the case of Glyph's example,
because the chain is made up entirely of slots pointers, there is never a
dict, tuple, or list - the only* types that can take part in this trick -
that can be added to the "delete later" list allowing the stack to be
unwound, so the deallocation simply recurses until it runs out of stack.

I can't see an easy way to fix this without changing the way that the
"delete later" list is constructed.

Chewy problem.

Jonathan


* OK, I lied slightly there. Actually frames and traceback objects take part
  in this crime as well, but they're not interesting here.




More information about the Python-list mailing list