[ python-Bugs-963246 ] Multiple Metaclass inheritance limitation

[SourceForge.net]

Bugs item #963246, was opened at 2004-05-30 19:52
Message generated for change (Comment added) made by pje
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=963246&group_id=5470

>Category: Type/class unification
Group: Python 2.3
>Status: Closed
>Resolution: Wont Fix
Priority: 5
Submitted By: Greg Chapman (glchapman)
Assigned to: Nobody/Anonymous (nobody)
Summary: Multiple Metaclass inheritance limitation

Initial Comment:
I'm not sure if this constitutes a bug or a limitation 
(which should be documented?), but the following 
doesn't work with Python 2.3.4.

Assume CMeta is a type defined in C, with
  tp_base = PyType_Type
  tp_new = some C function which uses the mro to 
      call the inherited tp_new
  tp_flags includes Py_TPFLAGS_BASETYPE

class PyMeta(type):
    def __new__(meta, name, bases, attrs):
        return super(PyMeta, meta).__new__(meta,
                       name, bases, attrs)

class MetaTest(CMeta, PyMeta):
    pass

class Test:
    __metaclass__ = MetaTest

The attempt to define Test generates a 
TypeError: "type.__new__(MetaTest) is not safe, use 
CMeta.__new__()".  

The above error is generated (in tp_new_wrapper) by 
the super call in PyMeta.__new__, but this is only 
reached as a result of an initial call to CMeta.tp_new 
(which, using the mro to find the next "__new__" 
method, finds and calls PyMeta.__new__).  

It may be there is no good way to allow the above 
scenario, but I just thought I'd point it out in case 
someone can think of a workaround.



----------------------------------------------------------------------

>Comment By: Phillip J. Eby (pje)
Date: 2004-06-05 20:19

Message:
Logged In: YES 
user_id=56214

There are two things that can cause the error message you
got.  One is calling a Python __new__ from a C type, such as
your code is doing.  The only workaround is "don't do that".
 A C type must always call only C __new__ methods.

You can avoid this in your example by moving CMeta *after*
PyMeta in the __bases__, but it will fail if CMeta is ever
placed anywhere but the end of the list.  The alternative is
to change CMeta to call its tp_base->tp_new instead of using
the mro to find the next base.  This will silently ignore
any Python __new__ methods in the mro, instead of causing a
TypeError.

Another multiple inheritance situation that can cause the
same error is if you define a C type which subtypes another
C type and does not increase its tp_basicsize, *and* the C
type is placed anywhere but first in the __bases__ of a
Python subclass.  You can work around that by ensuring that
its tp_basicsize is larger than that of its base C type, so
that Python will always pick it as the __base__ (aka
tp_base) even if it is not listed first in __bases__.

I have written the following additions to the Extending and
Embedding manual for future reference:

\note{If you want your type to be subclassable from Python,
and your
type has the same \member{tp_basicsize} as its base type,
you may   
have problems with multiple inheritance.  A Python subclass
of your  
type will have to list your type first in its
\member{__bases__}, or
else it will not be able to call your type's
\method{__new__} method
without getting an error.  You can avoid this problem by
ensuring
that your type has a larger value for \member{tp_basicsize} than
its base type does.  Most of the time, this will be true anyway,
because either your base type will be \class{object}, or
else you will
be adding data members to your base type, and therefore
increasing its
size.}

and...

\note{If you are creating a co-operative \member{tp_new}
(one that  
calls a base type's \member{tp_new} or \method{__new__}), you
must \emph{not} try to determine what method to call using
method resolution order at runtime.  Always statically determine
what type you are going to call, and call its \member{tp_new}
directly, or via \code{type->tp_base->tp_new}.  If you do
not do this, Python subclasses of your type that also inherit   
from other Python-defined classes may not work correctly.
(Specifically, you may not be able to create instances of
such subclasses without getting a \exception{TypeError}.)}

For more discussion on this, you can also see:

A thread on Python-Dev that touches on these issues:
http://mail.python.org/pipermail/python-dev/2003-April/034633.html

Some notes on ZODB4 running afoul of the same issues:
http://collector.zope.org/Zope3-dev/86





----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=963246&group_id=5470