[Python-ideas] A way out of Meta-hell (was: A (meta)class algebra)

[Petr Viktorin]

On Tue, Feb 17, 2015 at 11:10 AM, Martin Teichmann
<lkb.teichmann at gmail.com> wrote:
> Hi Petr, Hi Andrew, Hi list,
>
> (responding to my own post, as I cannot respond to Petr's and Andrew's
> at the same time)
>
>> - is it desirable to have a programmatic way to combine metaclasses?
>
> there seems to be general agreement that there should be no automatic
> combination of metaclasses, while many think that explicit combination
> should be possible.
>
> The only way to do that currently is by subclassing existing metaclasses.
> This is typically done by the user of the metaclass, not its author.
> It would be a nice thing if an author could do that. Petr argues s/he
> already can, by subclassing. Unfortunately, this is not so easy as this
> means that every user of such a class then has to use all inherited
> metaclasses.
>
> As an example: imagine you have a metaclass that mixes well with
> PyQt's metaclass. Then you could simply inherit from it. But this
> would mean that every user of you class also has to install PyQt,
> no matter if the code has anything to do with it.

But if you did have a merging mechanism, would it solve this problem?
To work properly, the merge mechanism in the new metaclass would have
to check that it's dealing with the PyQt metaclass, wouldn't it? How
whould that check work if the PyQt metaclass is not available?

> Now Thomas argues that this should simply not be done at all.
> I understand his argument well that multiple inheritance across
> project boundaries is probably a bad idea in most cases.
>
> But I have an example where it does make a lot of sense: ABCs.
> It would just be nice if one could implement and ABC which is a
> QObject. That's what ABCs are for. Sure, you can do this by
> registering the class, but this means you loose a lot like already
> predefined methods of an ABC. But maybe we can just
> special-case ABCs.

I don't quite get the example. Why would you want an ABC that is a QObject?

[...]
> Once we have one of my ideas implemented, I think we can
> implement PEP 422 in the standard library. Petr claims that this
> is just doing simple things with complex code. I disagree, I think
> that the current metaclass code in the python interpreter is already
> incredibly context, and just throwing more code at it is actually
> a very complex solution. Sure, on the python side all looks pretty,
> but only because we swept all complexity under the C carpet.
> But I won't fight for that point. What I am fighting for is to modify
> PEP 422 in a way that it can be backported to older versions of
> python and be put on PyPI. This means that __init_class__ should
> become __init_subclass__, meaning that only subclasses are
> initialized. (__init_class__ doesn't work properly, as super() won't
> work. This problem already exists in the zope version).

I've warmed up to this somewhat, once you mentioned the "registration
metaclass", where all subclasses of Registrar *except Registrar
itself* are collected somehow. Since the class name is not bound when
__init_class__ is called, it would not be straightforward to filter
out the base class.
I'm not sure how common this is, however – use cases I can come up
with would work either way.

An option is that __init_class__ only being called on subclasses could
be a documented limitation of just the backport on PyPI.

> There is also a completely different way we could solve all those
> problems: abandon metaclasses for good, and replace them
> eventually with a PEP 422 like approach. This would mean that
> on the long run the following should be added:
>
> - a __new_subclass__ as a replacement for the metaclass __new__.
> This shows that it is necessary to operate on subclasses, otherwise
> one will have a hard time to create the class itself...

Well, super() cooperative inheritance won't help if you really need
different types of objects, rather than differently initialized
objects. This pretty much needs the full power of metaclasses.

> - __subclasscheck__ and __instancecheck__ must be called on the
> class, not the metaclass
>
> - sometimes metaclasses are used to add methods to the class only,
> that cannot be reached by an instance. A new @classonlymethod
> decorator should do that.

Such a decorator isn't terribly hard to write today.

> - I'm sure I forgot something.
>
> A pure python version of this proposal (it would go into C, for sure)
> can be found here:
> https://github.com/tecki/metaclasses
>
> Note that this is much more complicated as it would need to be if
> only the original PEP422 ideas were implemented.
>
> This was a long post.
>
> Enjoy.
>
> Greetings