At 10:41 AM 3/2/2009 +0000, Paul Moore wrote:
2009/3/2 Benjamin Peterson
: 2009/3/1 Paul Moore
: Is it worth getting simplegeneric exposed in 3.1 (http://bugs.python.org/issue5135)? If it's going to be in 2.7, I'd like to see it hit 3.1. The patch is against trunk (for 2.7) at the moment, I'm not sure what the process would be for forward-porting it (do I generate a new patch against the py3k branch, or should it be applied to trunk and merged in?)
By the way guys, are you aware of: http://pypi.python.org/pypi/simplegeneric There might be a bit of name confusion by exposing pkgutils' internal simplegeneric there. Perhaps it should be called "trivialgeneric", as it's even tinier than simplegeneric. ;-)
The key problem with the patch is that ABCs do not play well with the type of introspection required to implement a generic function - namely enumeration of the superclasses of a class. The MRO of the class is fine for normal inheritance, but for ABCs it is possible to register classes which don't inherit from the ABC, so that you have a situation where issubclass (C, MyABC) can be true without MyABC being in C.__mro__:
import abc class MyABC(object): ... __metaclass__ = abc.ABCMeta ... class C(object): ... pass ... MyABC.register(C) issubclass(C, MyABC) True C.__mro__ (
, ) More generally, there is NO WAY to determine the list of classes for which issubclass(C, x) is true.
This could be considered a limitation of, or a bug in, ABCs, I don't have a particular opinion on that, but it does mean that no code which relies on being able to traverse the class inheritance graph will see ABCs. One particular case of this is (any implementation I can think of, of) generic functions.
In my view, this implies one of the following:
1) It should be a documented limitation of such code that it doesn't work with ABCs (and conversely, this limitation of ABCs should be documented in the ABC documentation) 2) Generic functions, and any other code requiring this type of introspection, is essentially useless unless it can support ABCs, and should not be used in the light of this limitation. 3) This is a bug in ABCs and should be fixed. 4) Something else I didn't think of :-)
In my view, (2) is an unreasonable position to take, given the fact that (as I understand it) ABCs are supposed to be largely optional and shouldn't affect code that doesn't care about them...
It's not clear to me how (3) should be addressed. Adding a slot to all classes to hold a list of ABCs they are registered against seems to be a large overhead for a relatively rarely used feature. I guess having a global registry of ABC registrations could work, but it seems clumsy. Any other suggestions?
This isn't really a new problem; if you base your generic function methods off of interfaces implemented by a type or instance, you have the same basic issues. For systems that use a cache based on object type (like Guido's tuple-dispatch prototype, and my enhanced version in PEAK-Rules), the actual lookup is not a big deal. You have a type-based test and you cache the result for the type. PEAK-Rules' predicate dispatching is a bit more complex, because you need a rather more complex type test; the tree generator has to look at whether a type test is an ABC, and effectively translate it to "oldstyleisinstance(arg, ABC) or not oldstyleisinstance(arg, ABC) and ABC.__instancecheck__(arg)". (Where oldstyleisinstance represents an __instancecheck__-free version of isinstance.) This isn't a major problem either, just a bit of a bore/pain to implement. The hairier issue for these types of systems is method precedence, though. Since __mro__'s have to be consistently ordered, you can straightforwardly determine whether one class is "more specific" than another in a static way. But with dynamic registration, the question could be more complex. Personally, I'd like to see some way to subscribe to changes in ABC registration, so that generic functions or other tools can update their caches. With that feature, you might even be able to implement full ABC support for simplegeneric, by treating ABC registrations as equivalent to mass registration of the ABC's registrants. That is, if "AnABC.register(X)" and "afunc.register(AnABC, meth)" then "afunc.register(X, meth)". So each time AnABC gets a new registrant, you automatically register the ABC method for the new registrant, as long as there's not already a method registered for that specific type. That would probably be sufficient for what simplegeneric is doing.
2009/3/2 P.J. Eby
By the way guys, are you aware of:
Yes. It has been mentioned, and I am certainly aware of both it and RuleDispatch.
There might be a bit of name confusion by exposing pkgutils' internal simplegeneric there. Perhaps it should be called "trivialgeneric", as it's even tinier than simplegeneric. ;-)
:-) The problem is that any discussion of more than the bare minimum functionality regularly results in people's heads exploding. Most specifically, Guido usually ends up hating the whole idea when it gets too complex. I'd rather stick with the most basic (and frankly, useful) aspects and avoid the headaches and messy explosions :-)
This isn't really a new problem; if you base your generic function methods off of interfaces implemented by a type or instance, you have the same basic issues. [...]
See? You just made my head hurt :-)
Personally, I'd like to see some way to subscribe to changes in ABC registration, so that generic functions or other tools can update their caches. With that feature, you might even be able to implement full ABC support for simplegeneric, by treating ABC registrations as equivalent to mass registration of the ABC's registrants.
But the issue is that the __subclasshook__ method allows things to be virtual subclasses of an ABC without either being a real subclass, *or* being explicitly registered. There simply is no way to answer the question "which ABCs does this class implement" and get such ABCs reported in the answer. Consider: class MyABC: __metaclass__ = ABCMeta @classmethod def __subclasshook__(cls, C): if cls is MyABC: if any("foo" in B.__dict__ for B in C.__mro__): return True return NotImplemented class C(object): def foo(self): pass c = C() Here, isinstance(c, MyABC) is True. But if f is a generic function with a registered implementation for MyABC, the only way I can see of ensuring that f(c) calls that implementation, would be to test isinstance(c, ImlCls) for all ImplCls for which there is a registered implementation. There'd need to be a cache of results of some form, to avoid a serious performance penalty, and it's not entirely obvious what the definition of "best fit" would be. That may be what you just said. :-) But regardless, this is getting beyond what I'd describe as "simple", and certainly is beyond any remit of tweaking pkgutil.simplegeneric to make it public. So either pkgutil.simplegeneric is useful with current (ignoring ABC) semantics, or it isn't suitable for exposure. If the latter, someone will have to propose a more complete implementation for the stdlib, which is something that traditionally is doomed to failure :-( So I don't intend to extend the functionality of pkgutil.simplegeneric, and I await a decision on whether or not the patch as it stands can go in. If the decision is that the current implementation isn't powerful enough to be of any use, I'd suggest that you propose your simplegeneric implementation for the stdlib (enhancing it to handle the ABC issue, if it doesn't already). Or we continue managing without generic functions in the stdlib, and the pprint rewrite which started all this off, goes ahead with an adhoc approach to extensibility. Paul.
participants (2)
-
P.J. Eby
-
Paul Moore