[Python-3000] ABC's, Roles, etc

[Phillip J. Eby]

At 03:52 PM 5/8/2007 -0600, Jeff Shell wrote:
>I have a lengthy post that dissects a major issue that I have with
>ABCs and the Interface definition that I saw in PEP 3124:: it all
>seems rigidly class and class-instance based.

Hi Jeff; I read your post a few days ago, but your blog doesn't 
support comments, so I've been "getting around to" writing a 
counterpoint on my blog.  But, now I can do it here instead.  :)


>The cardinal sin I saw
>in the Interface definition in PEP 3124 (at least, at the time I last
>viewed it) was the inclusion of 'self' in a method spec.

That's because you're confusing a generic function and a "method 
spec".  PEP 3124 interfaces are not *specifications*; they're 
namespaces for generic functions.  They are much closer in nature to 
ABCs than they are zope.interface-style Interfaces.  The principal 
thing they have in common with zope.interface (aside from the name) 
is the support for "IFoo(ob)"-style adaptation.

Very few of zope.interface's design goals are shared by PEP 
3124.  Notably, they are not particularly good for type checking or 
verification.  In PEP 3124, for example, IFoo(ob) *always* returns an 
object with the specified attributes; the only way to know whether 
they are actually implemented is to try using them.

Notice that this is diametrically opposed to what zope.interface 
wants to do in such situations -- which is why the PEP makes such a 
big deal about it being possible to use zope.interface *instead*.

That is, my own observation is that different frameworks sometimes 
need different kinds of interfaces.  For example, someone might 
create a framework that wants to verify preconditions and 
postconditions of methods in an interface, rather than merely 
specifying their names and arguments!  Using zope.interface as an 
exclusive basis for interface definition and type annotations would 
block innovation in this area.

PEP 3124 interfaces are therefore explicitly intended to be merely 
one *possible* kind of interface, rather than a be-all end-all 
interface system.  They have many differences from zope.interface, 
which, depending on your goals, may be a plus or minus.  But you 
certainly aren't obligated to *use* them.  PEP 3124 merely proposes a 
framework for how to use interfaces for method overloading, generic 
functions, and AOP.

 From the PEP itself:

"""For example, it should be possible
to use a ``zope.interface`` interface object to specify the desired
type of a function argument, as long as the ``zope.interface`` package
registered itself correctly (or a third party did the registration).

In this way, the proposed API simply offers a uniform way of accessing
the functionality within its scope, rather than prescribing a single
implementation to be used for all libraries, frameworks, and
applications."""


>... 'self' is an internal detail of class-instance implementations.

Again - this is because you're assuming the purpose of a PEP 3124 
interface is to *specify* an interface, when in fact it's much more 
like an ABC, which may also *implement* the interface.  The 
specification and implementation are intentionally unified here.

Of course, again, you will be able to use zope.interfaces as argument 
annotations to @overload-ed functions and methods, which is the point 
of the PEP.  Its "Interface" class is merely a suggested default, 
leaving the fancier tricks to established packages, in the same way 
that its generic function implementation will not do everything that 
RuleDispatch or PEAK-Rules can do.  It's supposed to be a core 
framework for such add-on packages, not a replacement for them.


>It seems to me that Abstract Base Classes and even PEP 3124 are
>primarily focused on classes. But in Python, "everything is an
>object", but not everything is class-based.
>...
>The rest of this post focuses on what `zope.interface` already
>provides - a system for specifying behavior and declaring support at
>both the class and object level - and 'object' really means 'object',
>which includes modules.

Right -- and *neither* "specifying behavior" nor "declaring support" 
are goals of PEP 3124; they're entirely out of its scope.  The 
Interface object's purpose is to support uniform access to, and 
implementation of, individual operations.  These are somewhat 
parallel concepts, but very different in thrust; zope.interface is 
LBYL, while PEP 3124 is EAFP all the way.

As a result, PEP 3124 chooses to punt on the issue of individual 
objects.  It's quite possible within the framework to allow 
instance-level checks during dispatching, but it's not going to be in 
the default engine (which is based on type-tuple dispatching; see 
Guido's Py3K overloading prototype).

zope.interface (and zope.component, IIRC) pay a high price in 
complexity for allowing interfaces to be per-instance rather than 
type-defined.  I implemented the same feature in PyProtocols, but 
over the years I rarely found it useful.  My understanding of its 
usefulness in Zope is that it:

1. supports specification and testing of module-level Zope APIs
2. allows views and other wrapping operations to be selected on a dynamic basis

Since #1 falls outside of PEP 3124's goals (i.e., it's not about 
specification or testing), that leaves use case #2.  In my 
experience, it has been more than sufficient to simply give these 
object some *other* interface, such as an IViewTags interface with a 
method to query these dynamic "tag" interfaces.  In other words, my 
experience and opinion supports the view that use case #2 is actually 
a coincidental abuse of interfaces for convenience, rather than the 
"one obvious way" to handle the use case.

To put it another way, if you define getView() as a generic function, 
you can always define a dynamic implementation of it for any type 
that you wish to have dynamic view selection capability.  Then, only 
those cases that require a complex solution have to pay for the complexity.

So, that's my rationale for why PEP 3124 doesn't provide any 
instance-based features out of the box; outside of API specs for 
singleton objects, the need for them is mostly an illusion created by 
Zope 3's dynamic view selection hack.


>My main focus is on determining what Abstract Base Classes and/or PEP
>3124's Interfaces do better than `zope.interface` (if anyone else is
>familiar with that package).

I at least am quite familiar with it, having helped to define some of 
its terminology and API, as well as being the original author of its 
class-decorator emulation for Python versions 2.2 and up.  :)  I also 
argued for its adoption of PEP 246, and wrote PyProtocols to unify 
Twisted and Zope interfaces in a PEP 246-based adaptation framework.

And what PEP 3124 does much better than zope.interface or even PyProtocols is:

1. Adaptation, especially incomplete adaptation.  You can implement 
only the methods that are actually needed for your use case.  If the 
interface includes generic implementations that are defined in terms 
of other methods in the interface, you need not reimplement 
them.  (Note: I'm well aware that here my definition of "better" 
would be considered "worse" by Jim Fulton, since zope.interface is 
LBYL-oriented.  However, for many users and use cases, EAFP *is* 
better, even if it's not for Zope.)

2. Interface recombination.  AFAIK, zope.interface doesn't support 
subset interfaces like PyProtocols does.  Neither zope.interface nor 
PyProtocols support method renaming, where two interfaces have a 
method with the same specification but different method names.

3. Low mental overhead.  PEP 3124 doesn't even *need* interfaces; 
simple use cases can just use overloaded functions and be on about 
their business.  Use cases that would require an interface and half a 
dozen adapter classes in zope.interface can be met by simply creating 
an overloaded function and adding methods.  And the resulting code 
reads like code in other languages that support overloading or 
generic functions, rather than reading like Java.


>In my blog post, I also show a dynamically constructed object
>providing an interface's specified behavior. An instance of an empty
>class is made, and then methods and other supporting attributes are
>attached to this specific instance only. Real world examples of this
>include Zope 2, where a folder may have "Python Scripts" or other
>callable members that, in effect, make for a totally custom object. It
>can also provide this same behavior (in fact, I was able to take
>advantage of this on some old old old Zope 2 projects that started in
>the web environment and transitioned to regular Python
>modules/classes).

And how often does this happen outside of Zope?  As I said, I rarely 
found it to be the case anywhere else.  I replicated the ability in 
PyProtocols because I was biased by my prior Zope experience, but 
once I got outside of Zope it almost entirely ceased to be useful.

Meanwhile, as I said, PEP 3124 is not closed to extension.  It's 
specifically intended that zope.interface (and any other interface 
packages that might arise in future) should be able to play as 
first-class citizens in the proposed API.  However, depending on the 
specific features desired, those packages might have some additional 
integration work to do.

(Note, by the way, that zope.interface is explicitly mentioned three 
times in the PEP, as an example of how other interface types should 
be able to be used for overloading, as long as they register 
appropriate methods with the provided framework.)