[Python-Dev] PEP 487 vs 422 (dynamic class decoration)

[PJ Eby]

On Thu, Apr 2, 2015 at 6:24 PM, Martin Teichmann
<lkb.teichmann at gmail.com> wrote:
> The whole point of PEP 487 was to reduce PEP 422 so much that
> it can be written in python and back-ported.

As I said earlier, it's a fine feature and should be in the stdlib for
Python 3.  (But it should have a `noconflict` feature added, and it
doesn't need a language change.)

However, since my specific use case was the one PEP 422 was originally
written to solve, and PEP 487 does not address that use case, it is
not a suitable substitute *for PEP 422*.

This is also not your fault; you didn't force Nick to withdraw it,
after all.  ;-)

My main concern in this thread, however, is ensuring that either the
use case behind PEP 422 doesn't get dropped, or that Nick is now okay
with me implementing that feature by monkeypatching __build_class__.
Since he practically begged me not to do that in 2012, and IIRC
*specifically created* PEP 422 to provide an alternative way for me to
accomplish this *specific* use case, I wanted to see what his current
take was.  (That is, did he forget the history of the PEP, or does he
no longer care about userspace code hooking __build_class__?  Is there
some other proposal that would be a viable alternative? etc.)


> Now you want to be able to write decorators whose details
> are filled in at class creation time.

Not "now"; it's been possible to do this in Python 2 for over a
decade, and code that does so is in current use by other packages.
The package providing this feature (DecoratorTools) was downloaded 145
times today, and 3274 times in the past month, so there is active,
current use of it by other Python 2 packages.  (Though I don't know
how many of them depend directly or indirectly upon this particular
feature.)

Currently, however, it is not possible to port this feature of
DecoratorTools (or any other package that uses that feature,
recursively) to Python 3, due to the removal of __metaclass__ and the
lack of any suitable substitute hook.


> Your point is that you want to be able to use your decorators
> without having to ask users to also inherit a specific class.
> I personally don't think that's desirable.  Many frameworks out
> there have such kind of decorators and mandatory base classes
> and that works fine.

The intended use case is for generic method decorators that have
nothing to do with the base class per se, so inheriting from a
specific base-class is an anti-feature in this case.


> The only problem remains once you need to
> inherit more than one of those classes, as their metaclasses
> most likely clash. This is what PEP 487 fixes.

No, it addresses the issue for certain *specific* metaclass use cases.
It does not solve the problem of metaclass conflict in general; for
that you need something like the sample `noconflict` code I posted,
which works for Python 3.1+ and doesn't require a language change.


> So my opinion is that it is not too hard a requirement to ask
> a user to inherit a specific mixin class for the sake of using
> a decorator.

If this logic were applied to PEP 487 as it currently stands, the PEP
should be rejected, since its use case is even *more* easily
accomplished by inheriting from a specific mixin class.  (Since the
feature only works on subclasses anyway!)

Further, if the claim is that metaclass conflict potential makes PEP
487 worthy of a language change, then by the same logic method
decorators are just as worthy of a language change, since any mixin
required to use a method decorator would be *just as susceptible* to
metaclass conflicts as SubclassInit.  (Notably, the stdlib's ABCMeta
is a common cause of metaclass conflicts in Python 2.6+ -- if you mix
in anything that implements an ABC by subclassing it, you will get a
metaclass conflict.)

Finally, I of course disagree with the conclusion that it's okay to
require mixins in order for method decorators to access the containing
class, since it is not a requirement in Python 2, due to the
availability of the __metaclass__ hook.  Further, PEP 422 was
previously approved to fix this problem, and has a patch in progress,
so I'm understandably upset by its sudden withdrawal and lack of
suitable replacement.

So personally, I think that PEP 422 should be un-withdrawn (or
replaced with something else), and PEP 487 should be retargeted
towards defining a `metaclass` module for the stdlib, including a
`noconflict` implementation to address metaclass conflict issues.
(Mine or someone else's, as long as it works.)  PEP 487 should not be
a proposal to change the language, as the provided features don't
require it.  (And it definitely shouldn't pre-empt a separately useful
feature that *does* require a language change.)

At this point, though, I mostly just want to get some kind of closure.
After three years, I'd like to know if this is a yea or nay, so I can
port the thing and move on, whether it's through a standardized
mechanism or ugly monkeypatching.  Honestly, the only reason I'm even
discussing this in the first place is because 1) Nick pleaded with me
three years ago not to hold off porting until a standardized way of
doing this could be added to the language, and 2) I've had some recent
inquiries from users about porting PEAK-Rules (which uses this
particular feature of DecoratorTools) to Python 3.  So I went to check
on PEP 422's status, and here we are.