[Python-ideas] A way out of Meta-hell (was: A (meta)class algebra)

[Petr Viktorin]

On Sat, Feb 14, 2015 at 10:34 AM, Martin Teichmann
<lkb.teichmann at gmail.com> wrote:
> Hi everyone,
>
> there seems to be a general agreement that metaclasses,
> in the current state, are very problematic. As to what the
> actual problem is there seems to be no agreement, but
> that there IS a problem seems to be accepted.
>
> Where do we go from here? One great idea is PEP 422.
> It just replaces the idea of metaclasses with something simpler.
> One point of critique is that it will take ages until people can
> actually use it. In order to avoid this, I wrote a pure python
> implementation of PEP 422 that works all the way down to
> python 2.7. If everyone simply started using this, we could have
> a rather smooth transition.

"Everyone" here means authors of every library that uses metaclasses, right?
I think you'll have a better chance convincing Python developers.
They're a subset :)

> I think that implementing PEP 422 as part of the language only
> makes sense if we once would be able to drop metaclasses
> altogether. I thought about adding a __new_class__ to PEP 422
> that would simulate the __new__ in metaclasses, thinking that
> this is the only way metaclasses are used.

Well, if you want Python to drop metaclasses, the way starts with PEP
422. You have to introduce the alternative first, and then wait a
*really* long time until you drop a feature.
I think __new_class__ can be added after PEP 422 is in, if it turns
out to be necessary.

> I was wrong. Looking at PyQt (to be precise: sip), I realized that
> it uses more: it overwrites get/setattr. That's actually a good
> usecase that cannot be replaced by PEP 422.
> (Sadly, in the case of sip, this is technically not a good usecase:
> it is apparently used to give the possibility to write mixin classes
> to the right of the mixed-in class in the inheritance chain. Could
> someone please convince Phil of PyQt that mixins go to the
> left of other base classes?)
>
> The upshot is: my solution sketched above simply does not work
> for C extensions that use metaclasses.
>
> This brings me to a completely different option: why don't we
> implement PEP 422 in pure python, and put it in the standard
> library? Then everyone writing some simple class initializer
> would just use that standard metaclass, and so you can use
> multiple inheritance and initializers, as all bases will use the same
> standard metaclass. Someone writing more complicated
> metaclass would inherit from said standard metaclass, if it
> makes sense to also have class initializers.

Just putting it in the standard library doesn't make sense, since it
would still only be available there from Python 3.5 on (or whenever it
gets in).
It really makes more sense to put this into the *real* standard
metaclass (i.e. `type`).
The Python implementation can be on PyPI for projects needing the
backwards compatibility.

> For C extensions:
> is it possible as a C metaclass to inherit from a python base
> class?

Not really, but you can rewrite the metaclass as a C extension (maybe
after the Python variant is ironed out).

> I added my pure python implementation of PEP 422. It is
> written to be backwards compatible, so a standard library
> version could be simplified. A class wanting to have initializers
> should simply inherit from WithInit in my code.
>
> Greetings
>
> Martin
>
> class Meta(type):
>     @classmethod
>     def __prepare__(cls, name, bases, namespace=None, **kwds):
>         if namespace is not None:
>             cls.__namespace__ = namespace
>         if hasattr(cls, '__namespace__'):
>             return cls.__namespace__()
>         else:
>             return super().__prepare__(name, bases, **kwds)
>
>     def __new__(cls, name, bases, dict, **kwds):
>         if '__init_class__' in dict:
>             dict['__init_class__'] = classmethod(dict['__init_class__'])
>         return super(Meta, cls).__new__(cls, name, bases, dict)
>
>     def __init__(self, name, bases, dict, **kwds):
>         self.__init_class__(**kwds)
>
>
> def __init_class__(cls, **kwds):
>     pass
>
> WithInit = Meta("WithInit", (object,), dict(__init_class__=__init_class__))
>
> # black magic: assure everyone is using the same WithInit
> import sys
> if hasattr(sys, "WithInit"):
>     WithInit = sys.WithInit
> else:
>     sys.WithInit = WithInit

I haven't studied the PEP in detail, but I'm not sure this adheres to
it; see https://www.python.org/dev/peps/pep-0422/#calling-init-class-from-type-init
I suggest adapting the tests from http://bugs.python.org/issue17044 for this.