[Python-Dev] tp_dictoffset calculation in 2.2.2

[Guido van Rossum]

Hi David and Kevin,

I'd like to hear from you if a particular last-minute change to 2.2.2
would affect your code in any way.  If you don't have time to read
this whole message, can you please check if this patch to Python
2.2.2b1 (in CVS on the release22-maint branch) has any bad effects on
your code?

diff -r2.126.4.24 typeobject.c
2090c2090
< 		COPYSLOT(tp_dictoffset);
---
> 		/*COPYSLOT(tp_dictoffset);*/


Background
----------

In Python 2.2, type objects have a tp_dictoffset which tells us where
the instance dict pointer is in the instance object layout.  This
pointer is calculated in three ways.  In precedence:

  1. (Line 1113) For dynamically created types (i.e. created by class
     statements), if the new type has no __slots__, the dominant base
     class has a zero tp_dictoffset, and the dominant base doesn't
     have a custom tp_getattro, a __dict__ will be added to the
     instance layout, and its tp_dictoffset is calculated by taking
     the end of the base class instance struct (or in a more
     complicated way if tp_itemsize is nonzero).

  2. (Line 1941) If the dominant base has a nonzero tp_dictoffset,
     copy the tp_dictoffset from the dominant base.

  3. (Line 2090) The tp_dictoffset of the first non-dominant base
     class that has a nonzero tp_dictoffset is copied.

Jeremy & I just went through a debugging session of some code he's
trying to add to Zope3 where we found that rule #3 is evil.
Simplified, we had the following situation:

  from some_extension import P
  class C(object): pass
  class D(C, P): pass

Type P in has some unique properties: its tp_dictoffset is zero, and
it has a custom tp_setattro.  This means that neither rule #1 fires
(custom tp_setattro) nor rule #2 (the dominant base has no
tp_dictoffset).  So the tp_dictoffset from class C is simply copied
into class D.  This offset is 12, but at that offset lives some
unrelated field in the P instance layout!

I believe that in cases where rule #3 triggers, it is *always* wrong:
when rule #2 hasn't triggered, the dict offset in the dominant base
class is different from the dict offset in the other base classes, but
the instance layout is inherited only from the dominant base.

But I can't prove this to myself with 100% satisfaction, so I'd like
to hear some test results, before I check this in as part of 2.2.2
final.

(The other COPYSLOT calls in the vicinity of the patch are also a bit
suspect, but much less so, since they are function pointers rather
than object offsets.)

--Guido van Rossum (home page: http://www.python.org/~guido/)