[Python-3000] ABC's, Roles, etc

[Phillip J. Eby]

At 09:57 PM 5/8/2007 -0600, Jeff Shell wrote:
>I must admit that I didn't read PEP 3124 in depth - most of it was
>fascinating, some of it went way over my head in complexity, and then
>suddenly I saw an Interface. It seemed quite out of place, actually,

Yep; it's really more like a "adapting abstract base class".  In a 
GF-based language like Dylan, it would probably just be called a 
"module".  In Dylan, the generic functions you export from a module 
define an interface, in precisely the same way as PEP 3124 Interfaces 
work, except they have no IFoo(bar).baz(), only IFoo.baz(bar).  (And 
with different syntax, of course.)


>[snip lots of stuff about zope.interface's specification features]

If you don't count in-house "enterprise" operations and shops like 
Google, Yahoo, et al., the development of Zope is certainly one of 
the largest (if not the very largest) Python project.  It's 
understandable that LBYL is desirable in that environment.  On the 
other hand, such large projects in Python are pretty darn rare.

Meanwhile, the subject of typing systems for Python (into which 
category zope.interface most assuredly falls) is still an open 
research area.  I've watched zope.interface and its predecessors and 
spin-offs for almost a decade now, and it's *still* not particularly 
settled.  Look, for example, at Guido's blog posts about type 
expressions and type parameterization.  Look at how quickly the 
efforts to define standard ABCs for Py3K turned back from grand 
vision to, "oh heck that doesn't quite do what we wanted".

So, my thought is that LBYL type systems for Python are still "here 
there be dragons" territory.  PEP 3124 simply doesn't try to go 
there, but neither does it block the passage of other explorers.  It 
happily coexists with other type systems, and if you want to use 
something called "roles" or "traits" as argument annotations, it will 
be OK with that.

The specifics, which I haven't spelled out in the "extension API" 
section yet, are mainly that in order to work with the default 
dispatching engine, you must register methods for the "implies()" 
generic function, such that the engine can tell whether a class 
implies the annotation, the annotation implies a class, or the 
annotation implies any other annotation.

Of course, there will also be a generic function you can register 
with in order to use a different dispatch engine when your 
annotations are encountered.  This would be the hook you'd need to 
use in order to have instance-specific checks.  Bear in mind, of 
course, that the such checks will necessarily be slow, and the 
slowdown may apply to every invocation of the function.


>'Look Before You Leap'
>brings to mind the concept of "don't dive into an empty pool" or
>"don't do a backwards flip onto the pointy rocks"; where as 'Easier to
>Ask Forgiveness Than Permission' brings to bind the concept of "sorry
>i dove head first into your empty pool and cracked my skull open Mr.
>Johnson. If I had asked I'm sure you would have said no! In any case,
>even though it's your pool and there was a fence and everything and
>you did not give me permission, my parents are going to sue" (OK,
>maybe that last bit is the result of a healthy american upbringing...
>but still!)

Well, if you run a program whose effects are that important without 
having tested it first, perhaps you *should* be sued.  :)


>Huh? I'll try to look at that. types, classes, instances... That does
>it, I'm switching to Io. (Honestly - I've recently seen the light
>about prototype based object oriented programming; in light of types
>of types and classes of classes and classes and instances and oh my,
>languages that believe in "there are only objects, and they are only
>instances" are sounding sweeter every day)

Why stop at Io?  Cecil is a prototype-based language with generic 
functions, including full predicate dispatch and even "dynamic types" 
(i.e., an object can be of several types at the same time, depending 
on its current state).  :)

For Python, though, I really don't see the need to create such 
ultra-dynamic objects.  It's so easy to just dynamically create a 
class whenever you need one, that it doesn't seem worth the bother to 
munge instances.  Zope, of course, is always a special case in this 
respect, since *persisting* dynamic classes is a PITA, compared to 
dynamic instances.  But the stdlib ain't Zope, and most Python code 
doesn't need to have its classes stored in a database.


>>Since #1 falls outside of PEP 3124's goals (i.e., it's not about
>>specification or testing), that leaves use case #2.  In my
>>experience, it has been more than sufficient to simply give these
>>object some *other* interface, such as an IViewTags interface with a
>>method to query these dynamic "tag" interfaces.  In other words, my
>>experience and opinion supports the view that use case #2 is actually
>>a coincidental abuse of interfaces for convenience, rather than the
>>"one obvious way" to handle the use case.
>
>Ugh. Yeah, there are 'marker' interfaces, but.. ugh. dynamic "tag"
>interfaces. Yuck.
>
>My experience has been otherwise.

Now you're confusing me.  How is having a way to ask for markers 
different from marker interfaces?  Both are equally "yuck", except 
that one isn't abusing interfaces to use them as markers.


>But I'm sorry that I confused that section of PEP 3124 to be about
>specification and testing. I do, however, think that is a better use
>case.

Well, write another PEP, then.  :)


>We have a dynamic data transformation framework that exists outside of
>Zope (Zope is basically used for UI). Objects are being dynamically
>composed, wrapped, decomposed, rewrapped, filtered, and split -
>constantly. Objects, not types. It's all composed of rules. I'm
>itching to be able to add rules to apply zope.interface specifications
>to the generated objects; if only to then make it much easier to add
>other filtering rules later on.

Maybe I'm getting to be like Guido in my old age, but maybe you 
should just write the program first and extract the framework second.  :)

In truth, though, I'd almost bet some serious cash that proper use of 
generic functions would evaporate your framework to virtual 
nothingness.  My observation has been that in languages with generic 
functions, the sort of thing that requires complex frameworks with 
lots of interfaces in Zope looks like a trivial little library.  In 
PEAK, I was able to cut out about 75% of the code in one 
sub-framework by switching it from interfaces+adaptation to generic 
functions, and in the process made it much more comprehensible.  With 
that kind of productivity enhancement, one can afford to write a lot 
more unit tests to do the LBYL-ing, and still come out ahead.  :)

Really, the problem of LBYL interfaces and adaptation is that they 
require you to laboriously figure out in advance where to allow 
flexibility in the framework.

Worse, they emphasize the *solution* domain rather than the problem 
domain.  They define what's required of parts that have to be plugged 
into a machine that then "solves" the problem.  So you have to design 
that machine and where various parts fit into it, which is largely a 
distraction from whatever you were trying to do in the first place!

In contrast, with generic functions, you focus simply on identifying 
the operations required by the problem domain, performing a 
functional decomposition rather than trying to create an entire 
network mesh of roles and responsibilities.  You code the *problem*, 
not the solution, so your code might even be comprehensible (or at 
least explainable) to non-programmer domain experts.  (i.e., even if 
they can't read the code, you can read it to them and confirm whether 
it matches the requirements.)

*Then*, after you have the functional decomposition (which is your 
real "specification" anyway), you can then decide what concrete 
object types you might need, and implement the lowest-level domain 
operations for those types.

And if you're following that process, interfaces really aren't 
anything but the documentation that explains what those 
problem-domain operations are supposed to do, so that if for some 
reason you need to implement new concrete types at a later time, you 
can add appropriate implementations.

IMO, that's a lot closer to being the One Obvious Way, because it 
doesn't *need* LBYL or anything like zope.interface, anywhere in that 
process.  See, if you want contract enforcement, you can just *build 
it right in* to the generic functions, using an appropriate method 
type.  See my comments here:

http://mail.python.org/pipermail/python-3000/2007-May/007444.html

Of course, that's not a *static* guarantee of correctness, but 
neither is most of what you're talking about.  Tests are still 
required, either way.