[Python-Dev] ABCs and MRO

[P.J. Eby]

At 10:41 AM 3/2/2009 +0000, Paul Moore wrote:
>2009/3/2 Benjamin Peterson <benjamin at python.org>:
> > 2009/3/1 Paul Moore <p.f.moore at gmail.com>:
> >>
> >> Is it worth getting simplegeneric exposed in 3.1
> >> (http://bugs.python.org/issue5135)? If it's going to be in 2.7, I'd
> >> like to see it hit 3.1. The patch is against trunk (for 2.7) at the
> >> moment, I'm not sure what the process would be for forward-porting it
> >> (do I generate a new patch against the py3k branch, or should it be
> >> applied to trunk and merged in?)

By the way guys, are you aware of:

   http://pypi.python.org/pypi/simplegeneric

There might be a bit of name confusion by exposing pkgutils' internal 
simplegeneric there.  Perhaps it should be called "trivialgeneric", 
as it's even tinier than simplegeneric.  ;-)


>The key problem with the patch is that ABCs do not play well with the
>type of introspection required to implement a generic function -
>namely enumeration of the superclasses of a class. The MRO of the
>class is fine for normal inheritance, but for ABCs it is possible to
>register classes which don't inherit from the ABC, so that you have a
>situation where issubclass (C, MyABC) can be true without MyABC being
>in C.__mro__:
>
> >>> import abc
> >>> class MyABC(object):
>...     __metaclass__ = abc.ABCMeta
>...
> >>> class C(object):
>...     pass
>...
> >>> MyABC.register(C)
> >>> issubclass(C, MyABC)
>True
> >>> C.__mro__
>(<class '__main__.C'>, <type 'object'>)
> >>>
>
>More generally, there is NO WAY to determine the list of classes for
>which issubclass(C, x) is true.
>
>This could be considered a limitation of, or a bug in, ABCs, I don't
>have a particular opinion on that, but it does mean that no code which
>relies on being able to traverse the class inheritance graph will see
>ABCs. One particular case of this is (any implementation I can think
>of, of) generic functions.
>
>In my view, this implies one of the following:
>
>1) It should be a documented limitation of such code that it doesn't
>work with ABCs (and conversely, this limitation of ABCs should be
>documented in the ABC documentation)
>2) Generic functions, and any other code requiring this type of
>introspection, is essentially useless unless it can support ABCs, and
>should not be used in the light of this limitation.
>3) This is a bug in ABCs and should be fixed.
>4) Something else I didn't think of :-)
>
>In my view, (2) is an unreasonable position to take, given the fact
>that (as I understand it) ABCs are supposed to be largely optional and
>shouldn't affect code that doesn't care about them...
>
>It's not clear to me how (3) should be addressed. Adding a slot to all
>classes to hold a list of ABCs they are registered against seems to be
>a large overhead for a relatively rarely used feature. I guess having
>a global registry of ABC registrations could work, but it seems
>clumsy. Any other suggestions?

This isn't really a new problem; if you base your generic function 
methods off of interfaces implemented by a type or instance, you have 
the same basic issues.

For systems that use a cache based on object type (like Guido's 
tuple-dispatch prototype, and my enhanced version in PEAK-Rules), the 
actual lookup is not a big deal.  You have a type-based test and you 
cache the result for the type.

PEAK-Rules' predicate dispatching is a bit more complex, because you 
need a rather more complex type test; the tree generator has to look 
at whether a type test is an ABC, and effectively translate it to 
"oldstyleisinstance(arg, ABC) or not oldstyleisinstance(arg, ABC) and 
ABC.__instancecheck__(arg)".  (Where oldstyleisinstance represents an 
__instancecheck__-free version of isinstance.)

This isn't a major problem either, just a bit of a bore/pain to implement.

The hairier issue for these types of systems is method precedence, 
though.  Since __mro__'s have to be consistently ordered, you can 
straightforwardly determine whether one class is "more specific" than 
another in a static way.  But with dynamic registration, the question 
could be more complex.

Personally, I'd like to see some way to subscribe to changes in ABC 
registration, so that generic functions or other tools can update 
their caches.  With that feature, you might even be able to implement 
full ABC support for simplegeneric, by treating ABC registrations as 
equivalent to mass registration of the ABC's registrants.

That is, if "AnABC.register(X)" and "afunc.register(AnABC, meth)" 
then "afunc.register(X, meth)".  So each time AnABC gets a new 
registrant, you automatically register the ABC method for the new 
registrant, as long as there's not already a method registered for 
that specific type.  That would probably be sufficient for what 
simplegeneric is doing.