[Python-3000] iostack and Oh Oh
Phillip J. Eby
pje at telecommunity.com
Sat Dec 2 01:29:47 CET 2006
At 01:46 PM 12/1/2006 -0800, Guido van Rossum wrote:
>I'm not sure what you mean this. Are you proposing that a library
>function that might take a mapping would be rewritten as a generic
>library function with one implementation that takes a mapping? Or are
>you proposing that the library function, instead of documenting that
>it takes a mapping, documents which generic functions should be
>applied to it?
>Both sound like rather a big stretch from current practice.
Actually, to me the thing that's a stretch from current practice is the
attempt to spell out in detail what a "mapping" is.
Note that in the simplest case, where you're only using getitem/setitem
operations, you're already using generic functions, just ones that have
syntax sugar (i.e. the  operator). So in that sense, you can say that
"mapping" means "supported by operator.getitem and operator.setitem".
I'm not saying there is no such thing as "mapping", IOW, I am saying that
"mapping" is an informal shorthand for a collection of operations. If you
want to be specific, refer to operations. If you wish to be concise (but
vague) then refer to "mapping".
To put it another way, I'm against halfway measures. It bothers me that
people are trying to introduce rigid interfaces, in order to address
"quick-and-dirty" use cases. These things are at opposite ends of the
spectrum, IMO: if you want a quick-and-dirty type test, test on the bloody
concrete type! Testing some abstract interface type to soothe your OO
conscience doesn't actually make the code any less rigid or dirty, it just
hides the smell. I'd rather that such code continued to obviously smell,
rather than pretend it's as fresh as daisies because "interfaces are better".
Conversely, if you really want that bit of code to be reusable or
extensible, then use a generic function, and the code will be in *fact*
extensible, rather than simply pushing the compatibility problem to
somebody else to figure out.
Example issue: library X demands a "mapping", but really only uses
getitem. Its code inspects interfaces to decide how to act on parameters,
and behaves differently if it sees "a mapping". I have an object that
implements getitem, and I want the special behavior, so I declare that
object "a mapping".
But now, library Y, that I also pass the same object to, suddenly starts
behaving differently, because it sees, "ah, you're a mapping! So I'll use
setitem on you..." And now I'm screwed.
Things like this used to happen to me all the time in Zope 2, which would
introspect methods and attributes a lot so that Zope could "decide" what to
do with an object based on what it was. (And it was this experience that
made me realize that inspect-and-decide is absolutely the wrong way to
write composable code.)
Zope 3 at first replaced this attribute-inspection with interface
inspection -- with no better result! After all, using an interface as a
flag is no different in essence than using a hasattr() check as a flag. As
I said, it only *looks* prettier. It wasn't until interface adaptation
arrived in Zope 3 that things actually improved, because (like generic
functions) adaptation at least allows third-party registration.
However, for interface adaptation to work well, the interfaces need to be
highly context-specific, or else we just end up back at the problem where
"mapping" means different things in different contexts. The PyProtocols
approach to solving this was to say, "One use case = one interface", which
eventually led me to realize that this generally amounts to having one or
more generic functions specific to the thing you're actually trying to
do. It takes a lot less time to just *explicitly* add overloads to your
code for the things you want to be able to do with an object, than to have
to debug the stuff that just suddenly starts happening all over the place,
as can happen with interface inspection.
Note that quick-and-dirty checks based on *concrete* types are actually
*safer* than abstract checks or interface checks, as this generally
prevents them from being used as mere behavior flags that can lead to
conflicts of the type I've described. In contrast, both duck typing (ie.
hasattr) checks and interface checks have proven in practice (Zope 2 and 3
respectively) to produce significant unwanted side-effects.
The only way to reduce these side effects is to allow persons other than
the code author to decide what should happen in a specific
context. Adaptation and generic functions have this ability in common, but
mere inspection (regardless of *what* is inspected) does not.
> > In other words, I just want to use my object with some operations that a
> > library provides (or requires). An "interface" is excise: something I have
> > to mess with that doesn't directly relate to my goals for using the
>I don't understand the sentence "An "interface" is excise". What does
>excise mean in this context?
It's an HCI buzzword meaning "work that doesn't obviously advance your
goal, but that you have to do anyway because of the way the system was
>Regardless, I find it quite a big change from various ways of saying
>"this object must have these methods (including perhaps some for which
>special syntax exists, like __getattr__)" to saying "this object must
>be supported by these generic functions".
Well, that's why I proposed last week that we allow you to say
ISomething(foo).somemethod() to be able to use such things. And, that you
be allowed to define your arguments as being of type ISomething, in order
to have the correct namespace automatically apply.
That proposal doesn't stop you from sticking with duck typing. However, if
you *want* to be explicit and extensible and pure, it allows you to do so.
Meanwhile, I argue that inspection is inherently quick-and-dirty, just like
duck typing. It doesn't actually improve anything, but instead makes you
do more work for no new benefit, while keeping the same likelihood of
What's more inspection is no different from generic functions in terms of
being able to produce "spooky action at a distance". However, at least
generic functions have tables whose contents can be inspected to find out
all the behaviors that might result, whereas interface inspection can
happen anywhere! And generic functions are actually extensible by third
parties, whereas inspection is not. So:
* No way to find "spooky" actions
* No way to modify broken behaviors without changing the code or trying to
"trick" the inspector
* "Spooky" actions are all in a table that can be dumped out and read
* Broken or undesirable actions can be overridden
>A method is just an
>identifier in the object's attribute namespace. A generic function is
>an object that may hve to be imported from elsewhere.
But this is also true of interfaces, regardless of how they're
defined. You can't simultaneously have "safe" duck typing and avoid the
use of imports, unless you use some kind of global naming scheme, as in Java.
>I object to the suggestion that seems to be implied here that in the
>future we'll all be writing ducklib.quack(ob) instead of the much
No, I'm saying that in any case where current interface proposals would do
if implements(ob, IDuck):
I would argue that you are better off with *either of*:
And that most code that isn't trying to be explicitly reusable would in
fact use the second, "normal" syntax. (In all likelihood, the code using
ducklib.quack would only be code *in* ducklib, actually.) GF's really only
come into play when you want to make a library extensible or generic, like
for pickling, pretty-printing, AST-visiting, etc. etc. Or, if you have
some circumstance that requires you to add custom code (like the sendmail()
overload example for 'str').
>I really like that when I have an object in my
>hands, I don't need to import anything else in order to manipulate it,
>as long as the manipulation can be done through methods. I also don't
>think that the homonym problem that you (and CLOS) are trying to solve
>here is all that important in practice.
Homonyms aren't the problem that's being solved, it's context-specific
extensibility and library composability. Those are much bigger problems
than name collisions. For example, a major source of Zope 2's
architectural difficulties was the direct result of using the same type
of inspection that's being promoted here. (Here being the Py3K list.) I
think it would be wise to learn from that experience.
Yes, Zope 2 used hasattr() checks, not interface checks, but the effect is
the same: people fudging what they provide in order to trick Zope into
doing the right thing(s), instead of being able to just directly define the
desired behavior, as they can do with adapters or GF's.
Now, I suppose you could look at these things as being no worse than they
are in various other languages, and that's probably true. On the other
hand, you could look at how much more composable libraries are in languages
with generic functions, and observe that as a general rule, such languages
do not have massive, all-inclusive branded frameworks like Zope, Twisted,
PEAK, etc. And the reason for that, is that in GF-based languages,
*libraries are composable on a larger scale*. So, there isn't a need for
single integrators to pull together any huge "all-in-one" frameworks.
Of course, packaging is also a factor: Twisted, Zope, and PEAK are
all-inclusive in part because the historical cost of depending on other
Python packages is high. I created setuptools specifically to address that
But the other major factor is integration: all-inclusive frameworks work
around the difficulties inherent in wrapping other packages. Many was the
time in developing PEAK that I wanted to use some existing library (or even
stdlib module), but couldn't because there was no way to add the necessary
"glue" without awkward monkeypatching. I would thus end up rolling my own
libraries more often than I wanted to.
GF-based frameworks, on the other hand, don't seem like frameworks at
all. They're just libraries that can be combined with other
libraries. Interfaces don't give you this ability on their own, and even
adaptation only gets you part of the way (and requires writing more code to
do the same things).
I'd like to see a Python where this type of composability falls out
naturally as a side effect of using the language idiomatically. Being able
to add overloads to existing functions makes it straightforward to either
override behaviors or add adapters as appropriate, when gluing disparate
More information about the Python-3000