[Python-ideas] Implicit submodule imports

Nathaniel Smith njs at pobox.com
Fri Sep 26 03:02:01 CEST 2014


On Thu, Sep 25, 2014 at 11:31 PM, Greg Ewing
<greg.ewing at canterbury.ac.nz> wrote:
> Nathaniel Smith wrote:
>>
>> They are really really hard
>> to do cleanly, and you risk all kinds of breakage in edge-cases (e.g.
>> try reload()'ing a module that's been replaced by an object).
>
> One small thing that might help is to allow the
> __class__ of a module to be reassigned to a
> subclass of the module type. That would allow
> a module to be given custom behaviours, while
> remaining a real module object so that reload()
> etc. continue to work.

Heh, I was actually just pondering whether it would be opening too big
a can of worms to suggest this myself. This is the best design I
managed to come up with last time I looked at it, though in existing
versions of python it requires ctypes hackitude to accomplish the
__class__ reassignment. (The advantages of this approach are that (1)
you get to use the full class machinery to define your "metamodule",
(2) any existing references to the module get transformed in-place, so
you don't have to worry about ending up with a mixture of old and new
instances existing in the same program, (3) by subclassing and
avoiding copying you automatically support all the subtleties and
internal fields of actual module objects in a forward- and
backward-compatible way.)

This would work today, and would solve all these problems, except for
the following code in Objects/typeobject.c:object_set_class:

    if (!(newto->tp_flags & Py_TPFLAGS_HEAPTYPE) ||
        !(oldto->tp_flags & Py_TPFLAGS_HEAPTYPE))
    {
        PyErr_Format(PyExc_TypeError,
                     "__class__ assignment: only for heap types");
        return -1;
    }
    if (compatible_for_assignment(oldto, newto, "__class__")) {
        Py_INCREF(newto);
        Py_TYPE(self) = newto;
        Py_DECREF(oldto);
        return 0;
    }

The builtin "module" type is not a HEAPTYPE, so if we try to do
mymodule.__class__ = mysubclass, then the !(oldto->tp_flags &
Py_TPFLAGS_HEAPTYPE) check gets triggered and the assignment fails.

This code has been around forever, but I don't know why. AFAIK we
could replace the above with

    if (compatible_for_assignment(oldto, newto, "__class__")) {
        if (newto->tp_flags & Py_TPFLAGS_HEAPTYPE) {
            Py_INCREF(newto);
        }
        Py_TYPE(self) = newto;
        if (oldto->tp_flags & Py_TPFLAGS_HEAPTYPE) {
           Py_DECREF(oldto);
        }
        return 0;
    }

and everything would just work, but I could well be missing something?
Is there some dragon lurking inside Python's memory management or is
this just an ancient overabundance of caution?

-n

-- 
Nathaniel J. Smith
Postdoctoral researcher - Informatics - University of Edinburgh
http://vorpus.org


More information about the Python-ideas mailing list