For the record - the 3rd process that is currently un-customizable when
creating a class, i.e. things that happen in an opaque way inside `type.__new__`, 
is the ABC class machinery. I could not recall it when
writing the previous e-mail.

Still - I think this might be very little disruptive, and yet provide
metaclasses back with full customization power, including
being able to address the problem brought up by Etham.


   js
 -><-

On Fri, 25 Dec 2020 at 01:40, Joao S. O. Bueno <jsbueno@python.org.br> wrote:
Actually, there are a few steps that `type.__new__`  perform that are not
customizable in metaclasses.

I had sometimes thought about mailing this here, or Python ideas, but could not 
come up with a "real world" use case where the customization of those
would be meaningful.

Let'me see if I recall all cases - two of them are the calls to
`__init_subclass__` and the descriptors `__set_name__` as you put it,
I think there is a third behavior that can't be separated from `type.__new__` - but
I can't remember it now


Anyway, the "thing to do" that always occurred to me about it is to add "soft" method slots
to `type` itself - so that `type.__new__` would call those  on the corresponding initialization phases.

Since these are to be run only when classes are created, their impact should be negligible.

In other words, have `type` implement methods like `__run_init_subclass__`, `__run_descriptor_setname__`,
(and one for the other task I can't remember now). So, all metaclass code written up to today remains valid,
and these behaviors become properly customizable. 

Adding keyword parameters to `type.__new__`, IMHO, besides a little bit fishy as we are talking of
arguments to change the behavior of the method, would themselves compete and have to be
filtered out, or otherwise special-cased in the `__init_subclass__` method itself.
I mean - let's suppose we add `__suppress_init_subclass__` as an named parameter to
`type.__new__` - what would happen with this argument in `__init_subclass__` ? Would it show
up in the kwargs? Otherwise it would be the _only_  kwarg popped out and not passed
to __init_subclass__, being an inconvenient exception. 

Having an overridable, separate, method in type to run __init_subclass__ and __set_name__
bypass these downsides.

In time, Happy holidays everyone!

   js
 -><-

On Fri, 25 Dec 2020 at 00:38, Ethan Furman <ethan@stoneleaf.us> wrote:
PEP 487 introduced __init_subclass__ and __set_name__, and both of those were wins for the common cases of metaclass usage.

Unfortunately, the implementation of PEP 487 with regards to __init_subclass__ has made the writing of correct
metaclasses significantly harder, if not impossible.

The cause is that when a metaclass calls type.__new__ to actually create the class, type.__new__ calls the
__init_subclass__ methods of the new class' parents, passing it the newly created, but incomplete, class.  In code:

```
class Meta(type):
     #
     def __new__(mcls, name, bases, namespace, **kwds):
         # create new class, which will call __init_subclass__ and __set_name__
         new_class = type.__new__(mcls, name, bases, namespace, **kwds)
         # finish setting up class
         new_class.some_attr = 9
```

As you can deduce, when the parent __init_subclass__ is called with the new class, `some_attr` has not been added yet --
the new class is incomplete.

For Enum, this means that __init_subclass__ doesn't have access to the new Enum's members (they haven't beet added yet).

For ABC, this means that __init_subclass__ doesn't have access to __abstract_methods__ (it hasn't been created yet).

Because Enum is pure Python code I was able to work around it:
- remove new __init_subclass__ (if it exists)
- insert dummy class with a no-op __init_subclass__
- call type.__new__
- save any actual __init_subclass__
- add back any new __init_subclass__
- rewrite the new class' __bases__, removing the no-op dummy class
- finish creating the class
- call the parent __init_subclass__ with the now complete Enum class

I have not been able to work around the problem for ABC.

Two possible solutions I can think of:

- pass a keyword argument to type.__new__ that suppresses the call to __init_subclass__; and
- provide a way to invoke new class' parent's __init_subclass__ before returning it

or

- instead of type.__new__ doing that work, have type.__init__ do it.

Thoughts?

--
~Ethan~
_______________________________________________
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-leave@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/ZMRRNSFSLJZDGGZ66CFCYQBINU62CDNX/
Code of Conduct: http://python.org/psf/codeofconduct/