[Python-3000] PEP for Metaclasses in Python 3000

Brett Cannon brett at python.org
Fri Mar 9 22:31:02 CET 2007 

On 3/9/07, Talin <talin at acm.org> wrote:
> I had a conversation with Guido last night at the Python user's group
> meeting, and we hashed out some of the details of how metaclasses should
> work. I've gone ahead and written up a PEP, which I present for your review.
> --------------------------------------------
> PEP: xxx
> Title: Metaclasses in Python 3000
> Version: $Revision$
> Last-Modified: $Date$
> Author: Talin <talin at acm.org>
> Status: Draft
> Type: Standards
> Content-Type: text/plain
> Created: 07-Mar-2007
> Python-Version: 3.0
> Post-History:
>
> Abstract
>
>      This PEP proposes changing the syntax for declaring metaclasses,
>      and alters the semantics for how classes with metaclasses are
>      constructed.
>
>
> Rationale
>
>      There are two rationales for this PEP, both of which are somewhat
>      subtle.
>
>      The primary reason for changing the way metaclasses work, is that
>      there are a number of interesting use cases that require the
>      metaclass to get involved earlier in the class construction process
>      than is currently possible. Currently, the metaclass mechanism is
>      essentially a post-processing step. With the advent of class
>      decorators, much of these post-processing chores can be taken over
>      by the decorator mechanism.
>
>      In particular, there is an important body of use cases where it
>      would be useful to preserve the order in which a class members are
>      declared. Ordinary Python objects store their members in a
>      dictionary, in which ordering is unimportant, and members are
>      accessed strictly by name. However, Python is often used to
>      interface with external systems in which the members are organized
>      according to an implicit ordering. Examples include declaration of C
>      structs; COM objects; Automatic translation of Python classes into
>      IDL or database schemas, such as used in an ORM; and so on.
>
>      In such cases, it would be useful for a Python programmer to specify
>      such ordering directly using the declaration order of class members.
>      Currently, such orderings must be specified explicitly, using some
>      other mechanism (see the ctypes module for an example.)
>
>      Unfortunately, the current method for declaring a metaclass does
>      not allow for this, since the ordering information has already been
>      lost by the time the metaclass comes into play. By allowing the
>      metaclass to get involved in the class construction process earlier,
>      the new system allows the ordering or other early artifacts of
>      construction to be preserved and examined.
>
>      The other, weaker, rationale is purely cosmetic: The current method
>      for specifying a metaclass is by assignment to the special variable
>      __metaclass__, which is considered by some to be aesthetically less
>      than ideal. Others disagree strongly with that opinion. This PEP
>      will not address this issue, other than to note it, since aesthetic
>      debates cannot be resolved via logically proofs.

I think you mean "via logical proofs" or "logically via proofs".

>
>
> Specification
>
>      In the new model, the syntax for specifying a metaclass is via a
>      keyword argument in the list of base classes:
>
>        class Foo(base1, base2, metaclass=mymeta):
>          ...
>
>      Additional keywords will also be allowed here, and will be passed to
>      the metaclass, as in the following example:
>
>        class Foo(base1, base2, metaclass=mymeta, private=True):
>          ...
>
>      Note that this PEP makes no attempt to define what these other
>      keywords might be - that is up to metaclass implementors to
>      determine.
>

Do the keywords have to follow the metaclass keyword, or is order
irrelevant?  While order makes sense, it would be a new precedent for
keyword arguments to have an important order.

> Invoking the Metaclass
>
>      In the current metaclass system, the metaclass object can be any
>      callable type. This does not change, however in order to fully
>      exploit all of the new features, the metaclass will need to have an
>      extra attribute which is used during class pre-construction.
>

That last sentence felt a little clumsy.  I think if you ditch that
last comma it reads more easily.

>      This attribute is a method named __metacreate__, which is invoked
>      before the evaluation of the class body, and which has the
>      following form:
>
>       classdict = metaclass.__metacreate__(name, bases, keywords)
>
>      Where:
>
>        'name' is the name of the class being created.
>        'bases' is the list of base classes.
>        'keywords' is the dictionary of keywords in the base class list.
>        'classdict' is a custom dictionary object which is created by the
>            metaclass, and which is used to store the class members as
>            they are declared.
>
>      Note that the Python interpreter will check to insure that the
>      __metacreate__ attribute exists before calling it. This preserves
>      backwards compatibility with existing metaclasses.
>
>      The 'classdict' object can be a regular dictionary or a custom
>      mapping type. It does not need to implement the full dictionary
>      interface; only the ability to insert items and retrieve them are
>      required. (Note: double check that this is true). When the body of
>      the class is evaluated, the dictionary will be used as the
>      'locals()' dict for that evaluation.
>
>      Once the class body has finished evaluating, the metaclass will be
>      called (as a callable) with the class dictionary, which is no
>      different from the current metaclass mechanism.
>
>      Typically, a metaclass will create a custom dictionary - either a
>      subclass of dict, or a wrapper around it - that will contain
>      additional properties that are set either before or during the
>      evaluation of the class body. Then in the second phase, the
>      metaclass can use these additional properties to further customize
>      the class.
>
>      An example would be a metaclass that uses information about the
>      ordering of member declarations to create a C struct. The metaclass
>      would provide a custom dictionary that simply keeps a record of the
>      order of insertions. This does not need to be a full 'ordered dict'
>      implementation, but rather just a Python list of (key,value) pairs
>      that is appended to for each insertion.
>

Does the language spec guarantee that the body of a class will be
executed in definition order?  Or is that considered implicit by the
fact that the class body is executed as code?

>      Note that in such a case, the metaclass would be required to deal
>      with the possibility of duplicate keys, but in most cases that is
>      trivial. The metaclass can use the first declaration, the last,
>      combine them in some fashion, or simply throw an exception. It's up
>      to the metaclass to decide how it wants to handle that case.
>
>
> Alternate Proposals
>
>      Josiah Carlson proposed using the name 'type' instead of
>      'metaclass', on the theory that what is really being specified is
>      the type of the type. While this is technically correct, it is also
>      confusing from the point of view of a programmer creating a new
>      class. From the application programmer's point of view, the 'type'
>      that they are interested in is the class that they are writing; the
>      type of that type is the metaclass.
>
>      There were some objections in the discussion to the 'two-phase'
>      creation process, where the metaclass is invoked twice, once to
>      create the class dictionary and once to 'finish' the class. Some
>      people felt that these two phases should be completely separate, in
>      that there ought to be separate syntax for specifying the custom
>      dict as for specifying the metaclass. However, in most cases, the
>      two will be intimately tied together, and the metaclass will most
>      likely have an intimate knowledge of the internal details of the
>      class dict. Requiring the programmer to insure that the correct dict
>      type and the correct metaclass type are used together creates an
>      additional and unneeded burden on the programmer.
>
>      Another good suggestion was to simply use an ordered dict for all
>      classes, and skip the whole 'custom dict' mechanism. This was based
>      on the observation that most use cases for a custom dict were for
>      the purposes of preserving order information. However, this idea has
>      two drawbacks, first because it means that an ordered dict
>      implementation would have to be added to the set of built-in types
>      in Python, and second because it would impose a slight speed (and
>      complexity) penalty on all class declarations.
>
>
> Backwards Compatibility
>
>      It would be possible to leave the existing __metaclass__ syntax in
>      place. Alternatively, it would not be too difficult to modify the
>      syntax rules of the Py3K translation tool to convert from the old to
>      the new syntax.
>
>
> References
>
>      [1] [Python-3000] Metaclasses in Py3K (original proposal)
>
> http://mail.python.org/pipermail/python-3000/2006-December/005030.html
>
>      [2] [Python-3000] Metaclasses in Py3K (Guido's suggested syntax)
>
> http://mail.python.org/pipermail/python-3000/2006-December/005033.html
>
>      [3] [Python-3000] Metaclasses in Py3K (Objections to two-phase init)
>
> http://mail.python.org/pipermail/python-3000/2006-December/005108.html
>
>      [4] [Python-3000] Metaclasses in Py3K (Always use an ordered dict)
>
> http://mail.python.org/pipermail/python-3000/2006-December/005118.html
>
>      [5] PEP 359: The 'make' statement -
>          http://www.python.org/dev/peps/pep-0359/
>
> Copyright
>
>      This document has been placed in the public domain.


Seems good, although I hardly ever use metaclasses so that doesn't say
a whole lot.  =)

-Brett

More information about the Python-3000 mailing list