[Cython] [cython-users] which __new__ to use?

[Stefan Behnel]

Stefan Behnel, 11.01.2014 06:46:
> Nils Bruin, 11.01.2014 00:38:
>> In Objects/typeobject.c:2895 I find:
>>
>> static PyObject *
>> object_new(PyTypeObject *type, PyObject *args, PyObject *kwds)
>> {
>>     int err = 0;
>>     if (excess_args(args, kwds)) {
>>         if (type->tp_new != object_new &&
>>             type->tp_init != object_init)
>>         {
>>             err = PyErr_WarnEx(PyExc_DeprecationWarning,
>>                        "object() takes no parameters",
>>                        1);
>>         }
>>         else if (type->tp_new != object_new ||
>>                  type->tp_init == object_init)
>>         {
>>             PyErr_SetString(PyExc_TypeError,
>>                 "object() takes no parameters");
>>             err = -1;
>>         }
>>     }
>>     if (err < 0)
>>         return NULL;
>> 
>> so it would seem that the objects in sage have both tp_new and tp_init 
>> overridden (and warnings are enabled). I'm puzzled how `object_new` can 
>> even end up being called then.
> 
> Because of this change, I guess:
> 
> http://thread.gmane.org/gmane.comp.python.cython.devel/15140/focus=15145

The corresponding Py3 code looks like this:

    if (excess_args(args, kwds) &&
        (type->tp_init == object_init || type->tp_new != object_new)) {
        PyErr_SetString(PyExc_TypeError, "object() takes no parameters");
        return NULL;
    }

My change would set "type->tp_new = object_new" if possible, which means
that these checks no longer hit in Py2, unless the __init__ code calls up
into object.__init__() with non-empty arguments, which has a similar check.
That kind of code would be broken, though.

But they would hit in Py3 if __init__() is not implemented. This might be
the case for code that uses only the fast "X.__new__(X, some, args)" way of
instantiation for some internal type that does not implement an __init__(),
because it knows that it's not being used anyway.

One could argue that such code is incorrect, even though it currently
works. One could also argue that such code that would be broken by my
change is most likely not common at all, because the change applies only to
types that have no attributes or C methods themselves. Instantiating those
by passing in arguments can't be a very common case.

However, considering that subtypes "innocently" call up their supertype
tp_new() chain, it's not obvious to me what other harmful combinations
there may be.

All in all, I think this has a potential to break stuff, while, at the same
time, it doesn't fix a regression, nor does it fix a serious problem that
users can't work around. So my change will not go into 0.20 (and I've
reverted it in master already), and it will then need some convincing
argumentation to get reapplied in the future, including a good series of
test cases that cover the safety checks in CPython.

Stefan