[Python-Dev] Possible rough edges in Python 3 metaclasses (was Re: Language reference updated for metaclasses)

[PJ Eby]

On Tue, Jun 5, 2012 at 3:53 AM, Nick Coghlan <ncoghlan at gmail.com> wrote:

> Please don't try to coerce everyone else into supporting such an ugly
> hack by abusing an implementation detail.


Whoa, whoa there.  Again with the FUD.

Sorry if I gave the impression that I'm about to unleash the monkeypatching
hordes tomorrow or something.  I'm unlikely to begin serious Python 3
porting of the relevant libraries before 3.3 is released; the reason I'm
talking about this now is because there's currently Python-Dev discussion
regarding metaclasses, so it seemed like a good time to bring the subject
up, to see if there were any *good* solutions.

Just because my three existing options are, well, the only options if I
started porting today, doesn't mean I think they're the *only* options.  As
I said, an option 4 or 5 would be fantastic, and your new PEP 422 is
wonderful in that regard.  Thank you VERY much for putting that together, I
appreciate that very much.

(I just wish you hadn't felt forced or coerced to do it -- that was not my
intention *at all*!)

Frankly, a big part of my leaning towards the __build_class__ option was
that it's the *least work by Python implementors* (at least in terms of
coding) to get my use case met.  The reason I didn't write a PEP myself is
because I didn't want to load up Python with yet another protocol, when my
use case could be met by stuff that's already implemented in CPython.

IOW, my motivation for saying, "hey, can't I just use this nice hook here"
was to avoid asking for a *new* feature, if there weren't enough other
people interested in a decoration protocol of this sort.

That is, I was trying to NOT make anybody do a bunch of work on my behalf.
(Clearly, I wasn't successful in the attempt, but I should at least get
credit for trying.  ;-)



> Now, one minor annoyance with current class decorators is that they're
> *not* inherited. This is sometimes what you want, but sometimes you
> would prefer to automatically decorate all subclasses as well.
> Currently, that means writing a custom metaclass to automatically
> apply the decorators. This has all the problems you have noted with
> composability.
>
> It seems then, that a potentially clean solution may be found by
> adding a *dynamic* class decoration hook. As a quick sketch of such a
> scheme, add the following step to the class creation process (between
> the current class creation process, but before the execution of
> lexical decorators):
>
>    for mro_cls in cls.mro():
>        decorators = mro_cls.__dict__.get("__decorators__", ())
>        for deco in reversed(decorators):
>            cls = deco(cls)
>
> Would such a dynamic class decoration hook meet your use case? Such a
> hook has use cases (specifically involving decorator inheritance) that
> *don't* require the use of sys._getframes(), so is far more likely to
> achieve the necessary level of consensus.
>

Absolutely.  The main challenge with it is that I would need stateful
decorators, so that they do nothing when called more than once; the
motivating use cases for me do not require actual decorator inheritance.
But I do have *other* stuff I'm currently using metaclasses for in 2.x,
that could probably be eliminated with PEP 422.  (For example, the #1
metaclass I use in 2.x is one that basically lets classes have
__class_new__ and __class_init__ methods: a rough equivalent to in-line
metaclasses or inheritable decorators!)

In general, implementing what's effectively inherited decoration is what
*most* metaclasses actually get used for, so PEP 422 is a big step forward
in that.

Sketching something to get a feel for the PEP...

def inheritable(*decos):
    """Wrap a class with inheritable decorators"""
    def decorate(cls):
        cls.__decorators__ =
list(decos)+list(cls.__dict__.get('__decorators__',()))
        for deco in reversed(decos):
            cls = deco(cls)
        return cls

Hm.  There are a few interesting consequences of the PEP as written.
In-body decorators affect the __class__ closure (and thus super()), but
out-of-body decorators don't.  By me this is a good thing, but it is a bit
of complexity that needs mentioning.  Likewise, the need for inheritable
decorators to be idempotent, in case two base classes list the same
decorator.  (For my own use attribute/method use cases, I can just have
them remove themselves from the class's __decorators__ upon execution.)



 You complain that metaclasses are hard to compose, and your
> "solution" is to monkeypatch a deliberately undocumented builtin?
>

To be clear, what I specifically proposed (as I mentioned in an earlier
thread) was simply to patch __build_class__ in order to restore the missing
__metaclass__ hook.  (Which, incidentally, would make ALL code using
__metaclass__ cross-version compatible between 2.x and 3.x: a potentially
valuable thing in and of itself!)

As for metaclasses being hard to compose, PEP 422 is definitely a step in
the right direction.  (Automatic metaclass combining is about the only
thing that would improve it any further.)
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20120605/e5b2608a/attachment.html>