[Python-3000] iostack and Oh Oh

[Guido van Rossum]

Phillip,

I have only a little time while I'm sitting in a hotel lobby, but I
printed this email out and read it several times over on the plane. It
seems as if you purposely ignore a point that I've brought up several
times now, without *ever* getting any kind of response from you. I
think I first discussed this in the sets example. I mentioned it again
in the post to which you're responding below -- but you seem to have
cut out those parts and ignored them.

My point is that an interface can *document* (at least in English) a
"contract" about the invariants between operations. While I'm not into
enforcing or verifying such contracts, I'm very interested in
documenting them. For example, something that has "mapping" behavior
has a very different relationship between x[y] and "y in x" than
something that has "sequence" behavior. If you see interfaces as
*just* combinations of operations there is no way to put this
information. So this is why I keep coming back to interfaces (or ABCs;
the difference isn't too important to me, though it seems to be to
others and if their argument is convincing enough I'd be happy to
accommodate them).

Note that I'm not against GFs; I just want interfaces or ABCs too.

--Guido

On 12/1/06, Phillip J. Eby <pje at telecommunity.com> wrote:
> At 01:46 PM 12/1/2006 -0800, Guido van Rossum wrote:
> >I'm not sure what you mean this. Are you proposing that a library
> >function that might take a mapping would be rewritten as a generic
> >library function with one implementation that takes a mapping? Or are
> >you proposing that the library function, instead of documenting that
> >it takes a mapping, documents which generic functions should be
> >applied to it?
> >
> >Both sound like rather a big stretch from current practice.
>
> Actually, to me the thing that's a stretch from current practice is the
> attempt to spell out in detail what a "mapping" is.
>
> Note that in the simplest case, where you're only using getitem/setitem
> operations, you're already using generic functions, just ones that have
> syntax sugar (i.e. the [] operator).  So in that sense, you can say that
> "mapping" means "supported by operator.getitem and operator.setitem".
>
> I'm not saying there is no such thing as "mapping", IOW, I am saying that
> "mapping" is an informal shorthand for a collection of operations.  If you
> want to be specific, refer to operations.  If you wish to be concise (but
> vague) then refer to "mapping".
>
> To put it another way, I'm against halfway measures.  It bothers me that
> people are trying to introduce rigid interfaces, in order to address
> "quick-and-dirty" use cases.  These things are at opposite ends of the
> spectrum, IMO: if you want a quick-and-dirty type test, test on the bloody
> concrete type!  Testing some abstract interface type to soothe your OO
> conscience doesn't actually make the code any less rigid or dirty, it just
> hides the smell.  I'd rather that such code continued to obviously smell,
> rather than pretend it's as fresh as daisies because "interfaces are better".
>
> Conversely, if you really want that bit of code to be reusable or
> extensible, then use a generic function, and the code will be in *fact*
> extensible, rather than simply pushing the compatibility problem to
> somebody else to figure out.
>
> Example issue: library X demands a "mapping", but really only uses
> getitem.  Its code inspects interfaces to decide how to act on parameters,
> and behaves differently if it sees "a mapping".  I have an object that
> implements getitem, and I want the special behavior, so I declare that
> object "a mapping".
>
> But now, library Y, that I also pass the same object to, suddenly starts
> behaving differently, because it sees, "ah, you're a mapping!  So I'll use
> setitem on you..."  And now I'm screwed.
>
> Things like this used to happen to me all the time in Zope 2, which would
> introspect methods and attributes a lot so that Zope could "decide" what to
> do with an object based on what it was.  (And it was this experience that
> made me realize that inspect-and-decide is absolutely the wrong way to
> write composable code.)
>
> Zope 3 at first replaced this attribute-inspection with interface
> inspection -- with no better result!  After all, using an interface as a
> flag is no different in essence than using a hasattr() check as a flag.  As
> I said, it only *looks* prettier.  It wasn't until interface adaptation
> arrived in Zope 3 that things actually improved, because (like generic
> functions) adaptation at least allows third-party registration.
>
> However, for interface adaptation to work well, the interfaces need to be
> highly context-specific, or else we just end up back at the problem where
> "mapping" means different things in different contexts.  The PyProtocols
> approach to solving this was to say, "One use case = one interface", which
> eventually led me to realize that this generally amounts to having one or
> more generic functions specific to the thing you're actually trying to
> do.  It takes a lot less time to just *explicitly* add overloads to your
> code for the things you want to be able to do with an object, than to have
> to debug the stuff that just suddenly starts happening all over the place,
> as can happen with interface inspection.
>
> Note that quick-and-dirty checks based on *concrete* types are actually
> *safer* than abstract checks or interface checks, as this generally
> prevents them from being used as mere behavior flags that can lead to
> conflicts of the type I've described.  In contrast, both duck typing (ie.
> hasattr) checks and interface checks have proven in practice (Zope 2 and 3
> respectively) to produce significant unwanted side-effects.
>
> The only way to reduce these side effects is to allow persons other than
> the code author to decide what should happen in a specific
> context.  Adaptation and generic functions have this ability in common, but
> mere inspection (regardless of *what* is inspected) does not.
>
>
> > > In other words, I just want to use my object with some operations that a
> > > library provides (or requires).  An "interface" is excise: something I have
> > > to mess with that doesn't directly relate to my goals for using the
> > library.
> >
> >I don't understand the sentence "An "interface" is excise". What does
> >excise mean in this context?
>
> It's an HCI buzzword meaning "work that doesn't obviously advance your
> goal, but that you have to do anyway because of the way the system was
> designed/implemented".
>
>
> >Regardless, I find it quite a big change from various ways of saying
> >"this object must have these methods (including perhaps some for which
> >special syntax exists, like __getattr__)" to saying "this object must
> >be supported by these generic functions".
>
> Well, that's why I proposed last week that we allow you to say
> ISomething(foo).somemethod() to be able to use such things.  And, that you
> be allowed to define your arguments as being of type ISomething, in order
> to have the correct namespace automatically apply.
>
> That proposal doesn't stop you from sticking with duck typing.  However, if
> you *want* to be explicit and extensible and pure, it allows you to do so.
>
> Meanwhile, I argue that inspection is inherently quick-and-dirty, just like
> duck typing.  It doesn't actually improve anything, but instead makes you
> do more work for no new benefit, while keeping the same likelihood of
> unexpected behavior.
>
> What's more inspection is no different from generic functions in terms of
> being able to produce "spooky action at a distance".  However, at least
> generic functions have tables whose contents can be inspected to find out
> all the behaviors that might result, whereas interface inspection can
> happen anywhere!  And generic functions are actually extensible by third
> parties, whereas inspection is not.  So:
>
> Inspection:
> * No way to find "spooky" actions
> * No way to modify broken behaviors without changing the code or trying to
> "trick" the inspector
>
> Generics:
> * "Spooky" actions are all in a table that can be dumped out and read
> * Broken or undesirable actions can be overridden
>
>
> >A method is just an
> >identifier in the object's attribute namespace. A generic function is
> >an object that may hve to be imported from elsewhere.
>
> But this is also true of interfaces, regardless of how they're
> defined.  You can't simultaneously have "safe" duck typing and avoid the
> use of imports, unless you use some kind of global naming scheme, as in Java.
>
>
> >I object to the suggestion that seems to be implied here that in the
> >future we'll all be writing ducklib.quack(ob) instead of the much
> >simpler ob.quack().
>
> No, I'm saying that in any case where current interface proposals would do
> this:
>
>      if implements(ob, IDuck):
>          ob.quack()
>
> I would argue that you are better off with *either of*:
>
>      ducklib.quack(ob)
>
> OR:
>
>      ob.quack()
>
> And that most code that isn't trying to be explicitly reusable would in
> fact use the second, "normal" syntax.  (In all likelihood, the code using
> ducklib.quack would only be code *in* ducklib, actually.)  GF's really only
> come into play when you want to make a library extensible or generic, like
> for pickling, pretty-printing, AST-visiting, etc. etc.  Or, if you have
> some circumstance that requires you to add custom code (like the sendmail()
> overload example for 'str').
>
>
> >I really like that when I have an object in my
> >hands, I don't need to import anything else in order to manipulate it,
> >as long as the manipulation can be done through methods. I also don't
> >think that the homonym problem that you (and CLOS) are trying to solve
> >here is all that important in practice.
>
> Homonyms aren't the problem that's being solved, it's context-specific
> extensibility and library composability.  Those are much bigger problems
> than name collisions.  For example, a major source of Zope 2's
> architectural difficulties was the direct result of using the same type
> of  inspection that's being promoted here.  (Here being the Py3K list.)  I
> think it would be wise to learn from that experience.
>
> Yes, Zope 2 used hasattr() checks, not interface checks, but the effect is
> the same: people fudging what they provide in order to trick Zope into
> doing the right thing(s), instead of being able to just directly define the
> desired behavior, as they can do with adapters or GF's.
>
> Now, I suppose you could look at these things as being no worse than they
> are in various other languages, and that's probably true.  On the other
> hand, you could look at how much more composable libraries are in languages
> with generic functions, and observe that as a general rule, such languages
> do not have massive, all-inclusive branded frameworks like Zope, Twisted,
> PEAK, etc.  And the reason for that, is that in GF-based languages,
> *libraries are composable on a larger scale*.  So, there isn't a need for
> single integrators to pull together any huge "all-in-one" frameworks.
>
> Of course, packaging is also a factor: Twisted, Zope, and PEAK are
> all-inclusive in part because the historical cost of depending on other
> Python packages is high.  I created setuptools specifically to address that
> problem.
>
> But the other major factor is integration: all-inclusive frameworks work
> around the difficulties inherent in wrapping other packages.  Many was the
> time in developing PEAK that I wanted to use some existing library (or even
> stdlib module), but couldn't because there was no way to add the necessary
> "glue" without awkward monkeypatching.  I would thus end up rolling my own
> libraries more often than I wanted to.
>
> GF-based frameworks, on the other hand, don't seem like frameworks at
> all.  They're just libraries that can be combined with other
> libraries.  Interfaces don't give you this ability on their own, and even
> adaptation only gets you part of the way (and requires writing more code to
> do the same things).
>
> I'd like to see a Python where this type of composability falls out
> naturally as a side effect of using the language idiomatically.  Being able
> to add overloads to existing functions makes it straightforward to either
> override behaviors or add adapters as appropriate, when gluing disparate
> libraries together.
>
>


-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)