metaclasses, how you might actually use 'em

[Jack Diederich]

__metaclass__ has been given a bad name.  They aren't as hard as you think and
they're a good deal more handy in your day to day life.  This is just a brief
exposition on metaclasses.  I just finished refactoring a bunch of code and
dropped the linecount by 15% while making the source more concise.  But it is
only more concise if people understand metaclasses, hence this evangelism.
Standard disclaimer, this is how I've come to know and love metaclasses.
Corrections encouraged, pedantry especially.

Brief overview of metaclasses

metaclasses just allow you to tweak a class definition on the fly.  Mainly 
adding or removing attributes just before the class becomes 'real'.  You could
get the same benefits from using eval() with a lot of pain and suffering. I 
use them for the Register pattern, and the Named Prototype pattern[1].

1) The Register pattern.
  Not included in GoF's 'Design Patterns' but one I've used quite frequently.
  Every time you define a subclass of X, store a pointer to it in a central 
  location, usually a factory but handy for generating any list of subclasses
  of X.

class Register(type):
  """A tiny metaclass to help classes register themselves automagically"""

  def __init__(cls, name, bases, dict):
    if ('register' in dict): # initial parent class
      setattr(cls, 'register', staticmethod(dict['register']))
    elif (getattr(object, 'DO_NOT_REGISTER', None)): # don't register this guy
      delattr(cls, 'DO_NOT_REGISTER') # this attribute won't be inherited
    else: # the usual case, register the class
      cls.register(cls)
    return

Here is an example of using the generic Register metaclass

class Factory(object):
  """A relatively useless factory"""
  string_name_to_class = {}
  def make_one(string_classname):
    """if string_classname == 'Foo', return Foo()"""
    return Factory.string_name_to_class[string_classname]()
  make_one = staticmethod(make_one) # make this method static

class Base(object):
  __metaclass__ = Register
  def register(klass):
    """tell Factory about klass"""
    Factory.string_name_to_class[klass.__name__] = klass
    return

class Foo(Base): pass
class Bar(Base): pass

Now let's create a Foo object by calling factory with the string name

>>> print Factory.string_name_to_class
{'Foo': <class 'meta.Foo'>, 'Bar': <class 'meta.Bar'>}
>>> foo = Factory.make_one('Foo')
>>> foo
<Foo object at 0x8153bdc>
>>> 

I used to have a list of lines at the bottom of a module that looked like
register_REST('my_URI_name1', SomeClass1)
register_REST('my_URI_name2', SomeClass2)
.. [lots more lines here]

These registered classes to do handle particular request when my mod_python 
program got a URI like 'http://REST/my_URI_name1'

I've also used it to count instances of subclasses in a graphing program,
there could be multiple lines are bars on a graph.  If there was N bars on
the graph the width of the bar is divided by N and offset from the starting
point.  I didn't want to know how many primitives there were because only
bars have this property. It counts the number of items of a class (lines or
bars) and tells each class you are the first, second, etc out of N for your
class.

[Speculation, the reason this wasn't included in the GoF is because C++
 makes no guarantees about order of class initialization.  So it isn't used.]

2) Named Prototype pattern.
  This one is in 'Design Patterns' by the GoF.  I'm from a C++ background so
  when I started with python I still did this the C++ way.  The prototype
  pattern is when you get a copy of an instance and then call some methods
  on the copy to change its behavior.  Specialization, but at runtime instead
  of by inheritance.

  This pattern is driven by laziness.  I can type quickly but verbosity hurts.
  It also allows for some compile time checking.

  My runtime definition of specialized prototypes looked like

  newob = ob.copy()
  newob.define(param1 = None,
               param2 = 'Sally',
               ..
               paramN = 2 * 8,
              )
  otherob = ob.copy()
  otherob.define(another_long_list_of_parameters)

  The parameters that define the behavior of the new object look like line
  noise.  The define() method would check the arguments, but this didn't happen
  until the code actually ran. Enter the Named Prototype.

class NamedPrototype(type):
  def __init__(cls, name, bases, dict):
    params = {}
    for (param) in my_list_of_important_params:
      params[param] = getattr(cls, param) # might raise an error, that is OK
      delattr(cls, param)
    validate_parameters(params) # might raise an error, that is OK
    
    setattr(cls, 'final_params', params)

class Base(object):
  """Virtual base class for our classes"""
  __metaclass__ = NamedPrototype

class A(Base):
  param1 = None
  param2 = 'Sally'
  ..
  paramN = 2 * 8

All the param* values will be removed from class A, validated, and stored in
the final class definition in the 'final_params' hash

>>> a = A()
>>> a.final_params
{ 'param1' : None, 'param2' : 'Sally' ... }

All this is only done once at the time the class is defined.
If you have every used PLY or SPARK (both lex/yacc rewritten in python)
they use the names of functions and even the doc strings to define the
behavior of the resulting classes.  A lexer in a modified PLY[2] looks like[3]

class MyLexer(BaseLexer):
  t_ignore = ' \t'
  
  def t_newline(self, t):
    r'[\n\r]'
    self.code += "\n"
    return t
  
  t_opencurly = r'\{'
  t_closecurly = r'\}'
  
  def t_leading_white(self, t):
    r'^\s+'
    if (not self.in_block):
      self.whitespace = ''
      self.whitespace = t.value
    return t

all the names starting with t_ are special and define rules for tokens.
Plain tokens are just regular expression strings, fancier tokens will call
a function and are matched by the function's doc string.  The BaseLexer class
has a metaclass that will check all the attributes, delattr() the members and
store them in a more regular structure and validate the results.

All the benefits of the Named Prototype pattern could be achieved through
straight inheritance in many instances, and regular Prototyping in the rest.
All except for easier finger typing.  The coder/user has to know that these 
methods will be treated specially, but he doesn't have to know they would be 
any more special because the metaclass cares or the plain inheritance super
class cares.  Either way they have to have read the documentation.


Ending disclaimer.  I don't read the snippets on ASPN, just c.l.py.  If this 
is known stuff or there is a better standard way just mark this thread as
ignored and enjoy your Sunday.  If people show interest I can provide
more deatils of my actual usage, but this post is already too long.

-jackdied

[1] Register pattern and Named Prototype pattern are TM me, right now, just 
    thought em up, couldn't think of anything better, suggestions welcome.

[2] For my own purposes I bastardized PLY.  An out of date version can
be found as lexer.py at http://jackdied.com/cgi-bin/viewcvs.cgi/pymud/
This code is completely unsupported. Use PLY or SPARK, they have 
good documentation.

[3] This is part of a psp (python server pages) lexer I posted to the
    mod_python development list a couple weeks ago.  Unfortunately, I don't
    think it is archived.