[Python-3000] Metaclasses in Py3K

[Josiah Carlson]

Talin <talin at acm.org> wrote:
> 
> 
> Josiah Carlson wrote:
> > Maybe I'm strange, but I don't like the precreation/double calling
> > semantic. It just seems...sloppy?  It requires modification of all
> > current metaclasses to update to the new metaclass= syntax, just to
> > possibly support a single operation, which, according to the discussions
> > so far have but a single use-case: taking advantage of operation
> > ordering of assignments in the class namespace.
> 
> I don't see why any existing metaclasses would need to be re-written - 
> see below.






> > I'd rather not see class dictionary overloading than a metaclass
> > double-call semantic.
> 
> I see it as more of a "begin / end" kind of operation. One function gets 
> called to begin the creation of a new class, the second gets called to 
> finish it. I think of it as compilation - one operation creates a new 
> template object, and the second one compiles it.

I understand what you are trying for, but I don't see it as necessary or
useful (in a simialr way how you don't see splitting the metaclass and
dictionary as being useful).

I like Greg's idea; leave the syntax as-is and always use an ordered
dictionary.


> >>> Of course, then we still have __slots__.  Will the madness never end?
> >> I believe that with this new system, we'll eventually be able to 
> >> eliminate the need for __slots__. The metaclass can examine the 
> >> dictionary and create a '__slots__' member based on the decorators or 
> >> wrappers of the various values within the dict.
> > 
> > At the point of class creation, the only place where attributes of the
> > final instance are defined is within __init__, or any other callable
> > from __init__.  The removal of __slots__ by attempting to introspect on
> > the class namespace is either *really hard* or impossible.
> > 
> > Note that __slots__ removal, at least as they currently exist in the
> > Python cookbook, basically all rely on introspecting on the argument
> > list to __init__.  One looks for leading underscores __init__(self, _a,
> > _b), others use other semantics.  Many of them attempt to generalize on
> > problems such as...
> > 
> >     class point(object):
> >         __slots__ = ['x', 'y', 'z']
> >         def __init__(self, x, y, z):
> >             self.x = x
> >             self.y = y
> >             self.z = z
> > 
> > What makes it not generalizable is that things like the above is for one
> > fairly small set of use case that is not covered by any of the autoslots
> > implementations.
> 
> Slots can be done today, with the existing __metaclass__ syntax:
> 
>      class Slot:
>          pass
> 
>      def slot_holder(name, bases, cdict):
>          slots = []
>          newdict = dict(__slots__=slots)
>          for key, value in cdict.iteritems():
>              if value is Slot:
>                  slots.append( key )
>              else:
>                  newdict[ key ] = value
> 
>          return type(name, bases, newdict)
> 
>      class X:
>          __metaclass__ = slot_holder
>          x = y = z = Slot
> 
>          def __init__(self, x, y, z):
>              self.x = x
>              self.y = y
>              self.z = z
> 
>      a = X( 1, 2, 3 )

The above *still* has the ugliness of needing to state the name of the
slots attributes as many times as in the original version, which was one
of the complaints about __slots__ use.  That is to say, the same issue
that existed with function decorators, exists today with __slots__. 
There are mechanisms to get around it (using one of the autoslots and
attribute initializers in the Python cookbook)...

    class X(object):
        __metaclass__ = AutoSlots
        def __init__(self, _x, _y, _z):
            InitAttrs(self, locals())

However, I've not been convinced that 


> >>>> 4) Backwards compatibility
> >>>>
> >>>> It might be possible to retain backwards compatibility if desired. In 
> >>>> the proposed scheme, the metaclass is invoked twice - once to create the 
> >>>> dictionary, and once to 'finish' the class. Note that the second step is 
> >>>> identical to the operation of the current __metaclass__ feature.
> >>> Any syntax-based scheme is, by definition, not fully backwards
> >>> compatible.  The best we can hope for is for new functionality to not
> >>> work as intended in previous Pythons, but still be runnable.
> >> No, I'm talking about allowing the old __metaclass__ syntax to still 
> >> continue work in the new versions of Python. It won't have all the 
> >> functionality of the new syntax, but it will still work as it did before 
> >> - at least, until we decide to remove it.
> > 
> > That not backwards compatability of new features, it is forwards
> > compatability of old features.
> 
> Huh? I don't understand. In my lexicon, the term "backwards 
> compatibility" generally means that existing code will continue to run 
> on a new version of the interpreter and/or standard libraries.

I'm thinking of it in the other direction.  That the code I write for
Py3k with the ability to use new features can still be run (perhaps with
slightly different semantics).  In this particular case, not changing
syntax, and just changing semantics, there is nothing barring us from
using a __future__ import to choose between using a standard dictionary
and an ordered one to have the functionality in Python 2.6 (if such is
desireable).

Also, if the point of this new syntax is to remove the __metaclass__
definition from the class namespace, why are we even talking about
keeping it for backwards compatibility?  Especially when it would offer
two ways of defining what the metaclass for a class is.


> As I see it, any class that uses the "__metaclass__ = foo" syntax would 
> continue to work exactly like it does today - assuming we want it to. 
> I'm basically saying that the new proposal and the existing system do 
> not interfere with each other, and share some common implementation 
> elements, that's all.
> 
> I don't see why you feel that all existing metaclasses would need to be 
> re-written.

Say I have a metaclass X.  Say that I've been using metaclass X since
Python 2.2 days.  One day I hear that shortly after Py3k, __metaclass__
is going away and I need to start using some other syntax (say the
metaclass= syntax for arguments sake).  So what do I do?

    class Foo(A, B, metaclass=X):
        ...

And I think that I have fixed my implementation for Py3k+, nevermind
that it doesn't work in Python 2.x anymore, I'm a forward thinker and I
like the new syntax. X implemented a class registry, so when X is called
twice (once for setup, and once for finalization), now the class gets
registered twice, or perhaps I get a crash because my metaclass doesn't
have a default dict argument that discriminates between the two cases
(using Steven's semantics). Ok, so now I need to adjust my metaclass to
check to see whether it is doing the setup or finalization step.  That's
fine, but now I've had to modify my metaclass, something that I was told
I wouldn't need to do, and with __metaclass__ going away (presumably,
otherwise would we create a new syntax?), I'm going to need to rewrite
*all* of my metaclasses when I switch syntaxes.

But I thought you said I wouldn't need to rewrite all of my metaclasses?

My arguments against a double-calling semantic with a new syntax are:
1) Given a double-calling semantic, for people moving to the new syntax,
they need to update their metaclasses, whether they want or need the new
functionality or not.
2) Having a new syntax, in addition to the current __metaclass__ syntax,
violates TOOWTDI.
3) Forcing rewrites, just to offer a single defined use case, seems to
be a waste.


Using Greg's suggestion of keeping the old syntax, but making the class
dictionary always be ordered, has none of these problems.  For people
who don't care about ordering, etc., their metaclasses keep working. For
those who want to care about ordering, they write new metaclasses and
get the features. If they want to use their code in both Py3k and 2.x,
they can check by version, or even check to see if the dictionary
argument is a dict or not (type (dct) is dict).

I would also mention that Greg's suggestion of keeping the old syntax
requires the smallest amount of work to make happen.

 - Josiah