[Python-3000] ABC's, Roles, etc
Phillip J. Eby
pje at telecommunity.com
Wed May 9 19:28:18 CEST 2007
At 09:57 PM 5/8/2007 -0600, Jeff Shell wrote:
>I must admit that I didn't read PEP 3124 in depth - most of it was
>fascinating, some of it went way over my head in complexity, and then
>suddenly I saw an Interface. It seemed quite out of place, actually,
Yep; it's really more like a "adapting abstract base class". In a
GF-based language like Dylan, it would probably just be called a
"module". In Dylan, the generic functions you export from a module
define an interface, in precisely the same way as PEP 3124 Interfaces
work, except they have no IFoo(bar).baz(), only IFoo.baz(bar). (And
with different syntax, of course.)
>[snip lots of stuff about zope.interface's specification features]
If you don't count in-house "enterprise" operations and shops like
Google, Yahoo, et al., the development of Zope is certainly one of
the largest (if not the very largest) Python project. It's
understandable that LBYL is desirable in that environment. On the
other hand, such large projects in Python are pretty darn rare.
Meanwhile, the subject of typing systems for Python (into which
category zope.interface most assuredly falls) is still an open
research area. I've watched zope.interface and its predecessors and
spin-offs for almost a decade now, and it's *still* not particularly
settled. Look, for example, at Guido's blog posts about type
expressions and type parameterization. Look at how quickly the
efforts to define standard ABCs for Py3K turned back from grand
vision to, "oh heck that doesn't quite do what we wanted".
So, my thought is that LBYL type systems for Python are still "here
there be dragons" territory. PEP 3124 simply doesn't try to go
there, but neither does it block the passage of other explorers. It
happily coexists with other type systems, and if you want to use
something called "roles" or "traits" as argument annotations, it will
be OK with that.
The specifics, which I haven't spelled out in the "extension API"
section yet, are mainly that in order to work with the default
dispatching engine, you must register methods for the "implies()"
generic function, such that the engine can tell whether a class
implies the annotation, the annotation implies a class, or the
annotation implies any other annotation.
Of course, there will also be a generic function you can register
with in order to use a different dispatch engine when your
annotations are encountered. This would be the hook you'd need to
use in order to have instance-specific checks. Bear in mind, of
course, that the such checks will necessarily be slow, and the
slowdown may apply to every invocation of the function.
>'Look Before You Leap'
>brings to mind the concept of "don't dive into an empty pool" or
>"don't do a backwards flip onto the pointy rocks"; where as 'Easier to
>Ask Forgiveness Than Permission' brings to bind the concept of "sorry
>i dove head first into your empty pool and cracked my skull open Mr.
>Johnson. If I had asked I'm sure you would have said no! In any case,
>even though it's your pool and there was a fence and everything and
>you did not give me permission, my parents are going to sue" (OK,
>maybe that last bit is the result of a healthy american upbringing...
Well, if you run a program whose effects are that important without
having tested it first, perhaps you *should* be sued. :)
>Huh? I'll try to look at that. types, classes, instances... That does
>it, I'm switching to Io. (Honestly - I've recently seen the light
>about prototype based object oriented programming; in light of types
>of types and classes of classes and classes and instances and oh my,
>languages that believe in "there are only objects, and they are only
>instances" are sounding sweeter every day)
Why stop at Io? Cecil is a prototype-based language with generic
functions, including full predicate dispatch and even "dynamic types"
(i.e., an object can be of several types at the same time, depending
on its current state). :)
For Python, though, I really don't see the need to create such
ultra-dynamic objects. It's so easy to just dynamically create a
class whenever you need one, that it doesn't seem worth the bother to
munge instances. Zope, of course, is always a special case in this
respect, since *persisting* dynamic classes is a PITA, compared to
dynamic instances. But the stdlib ain't Zope, and most Python code
doesn't need to have its classes stored in a database.
>>Since #1 falls outside of PEP 3124's goals (i.e., it's not about
>>specification or testing), that leaves use case #2. In my
>>experience, it has been more than sufficient to simply give these
>>object some *other* interface, such as an IViewTags interface with a
>>method to query these dynamic "tag" interfaces. In other words, my
>>experience and opinion supports the view that use case #2 is actually
>>a coincidental abuse of interfaces for convenience, rather than the
>>"one obvious way" to handle the use case.
>Ugh. Yeah, there are 'marker' interfaces, but.. ugh. dynamic "tag"
>My experience has been otherwise.
Now you're confusing me. How is having a way to ask for markers
different from marker interfaces? Both are equally "yuck", except
that one isn't abusing interfaces to use them as markers.
>But I'm sorry that I confused that section of PEP 3124 to be about
>specification and testing. I do, however, think that is a better use
Well, write another PEP, then. :)
>We have a dynamic data transformation framework that exists outside of
>Zope (Zope is basically used for UI). Objects are being dynamically
>composed, wrapped, decomposed, rewrapped, filtered, and split -
>constantly. Objects, not types. It's all composed of rules. I'm
>itching to be able to add rules to apply zope.interface specifications
>to the generated objects; if only to then make it much easier to add
>other filtering rules later on.
Maybe I'm getting to be like Guido in my old age, but maybe you
should just write the program first and extract the framework second. :)
In truth, though, I'd almost bet some serious cash that proper use of
generic functions would evaporate your framework to virtual
nothingness. My observation has been that in languages with generic
functions, the sort of thing that requires complex frameworks with
lots of interfaces in Zope looks like a trivial little library. In
PEAK, I was able to cut out about 75% of the code in one
sub-framework by switching it from interfaces+adaptation to generic
functions, and in the process made it much more comprehensible. With
that kind of productivity enhancement, one can afford to write a lot
more unit tests to do the LBYL-ing, and still come out ahead. :)
Really, the problem of LBYL interfaces and adaptation is that they
require you to laboriously figure out in advance where to allow
flexibility in the framework.
Worse, they emphasize the *solution* domain rather than the problem
domain. They define what's required of parts that have to be plugged
into a machine that then "solves" the problem. So you have to design
that machine and where various parts fit into it, which is largely a
distraction from whatever you were trying to do in the first place!
In contrast, with generic functions, you focus simply on identifying
the operations required by the problem domain, performing a
functional decomposition rather than trying to create an entire
network mesh of roles and responsibilities. You code the *problem*,
not the solution, so your code might even be comprehensible (or at
least explainable) to non-programmer domain experts. (i.e., even if
they can't read the code, you can read it to them and confirm whether
it matches the requirements.)
*Then*, after you have the functional decomposition (which is your
real "specification" anyway), you can then decide what concrete
object types you might need, and implement the lowest-level domain
operations for those types.
And if you're following that process, interfaces really aren't
anything but the documentation that explains what those
problem-domain operations are supposed to do, so that if for some
reason you need to implement new concrete types at a later time, you
can add appropriate implementations.
IMO, that's a lot closer to being the One Obvious Way, because it
doesn't *need* LBYL or anything like zope.interface, anywhere in that
process. See, if you want contract enforcement, you can just *build
it right in* to the generic functions, using an appropriate method
type. See my comments here:
Of course, that's not a *static* guarantee of correctness, but
neither is most of what you're talking about. Tests are still
required, either way.
More information about the Python-3000