[Python-Dev] Re: PEP 246 and Protocols (Was: Sneaky 'super' instances)

[Phillip J. Eby]

At 03:42 PM 6/12/03 +0100, Moore, Paul wrote:
>From: Phillip J. Eby [mailto:pje@telecommunity.com]
> > Open protocols solve the chicken and egg problem by allowing one
> > to make declarations about third-party objects.
>
>OK, I've tried to put together a simple example of "what I expect" and
>I find I can't. I want to continue to write code which "just assumes"
>that it gets the types it needs - I don't want to explicitly state that
>I need specific interface types - that feels like type declaration and
>Java. Your IFile example reinforces that, both in terms of its naming
>convention, and in the assumption that there *is* a single, usable,
>"file" protocol. I know it was my example, but I pointed out later in
>the same message that I really sometimes wanted seekable readline-only
>files, and other times block read/write (potentially unseekable) files.
>
>Expecting library writers to declare interface "classes" for every
>subtle variation of requirements seems impractical. Expecting the
>requirements to be *documented* is fine - it's having a concrete class
>which encapsulates them that I don't see happening - no-one would ever
>look to see if there was already a "standard" interface which said "I
>have readline, seek, and tell" - they'd just write a new one. There
>goes any hope of reuse. (This may be what you see as a "chicken and egg"
>problem - if so, I see it as a "hopelessly unrealistic expectations"
>problem instead, because it's never going to happen...)

Even if it's as simple as saying this:

     class ReadlineSeekAndTell(Interface):
         """You must have readline, seek, and tell if you pass this to me"""
         advise(protocolIsSubsetof=[IFile])


Or, even more briefly, using Samuele's way:

     RST = subproto(IFile,['readline','seek','tell'])



>On the other hand, expecting callers to write stuff to adapt existing
>classes to the requirements of library routines is (IMHO) a non-issue.
>I think that's what PEP 246 was getting at in the statement that "The
>typical Python programmer is an integrator" which you quote. It's
>common to write
>
>     class wrap_source:
>         def __init__(self, source):
>             self.source = source
>         def read(self, n = 1024):
>             return self.source.get_data(n)
>
>     lib_fn(wrap_source(my_source))
>
>So PEP 246 is trying to make writing that sort of boilerplate easier
>(in my view). The *caller* should be calling adapt(), not the callee.

That depends.  It's common in both Zope and Twisted for framework code to 
do the Zope or Twisted equivalents of 'adapt()'.  But yes, it is also 
common to state that some function in effect requires an already adapted 
object.


><snip>
>Which is appropriate is basically down to which came first, string or
>file. But both suffer from the problem of putting all the knowledge
>in one location (and using a type switch as well).

Right.  That's what the point of the open protocols system is.  The problem 
is that in every Python interface system I know of (except PyProtocols), 
you can't declare that package A, protocol X, implies package B, protocol 
Y, unless you are the *author* of package B's protocol Y.

But if package A's author used an open protocol to define protocol X, then 
you can use PyProtocols today to say that it implies package B's protocol 
Y, as long as protocol Y can be adapted to the declaration API.

Translation: anybody who defines protocols for their packages today using 
PyProtocols, will be able to have third parties define adapters to 
interfaces provided by Zope or Twisted, or that are defined using Zope or 
Twisted interfaces.


>The third option, which is the "external registry" allows a *user* of
>the string and file "libraries" (bear with me here...) to say how he
>wants to make strings look like files:
>
><snip>
>So I see PEP 246 as more or less entirely a class based mechanism. It
>has very little to do with constraining (or even documenting) the
>types of function parameters.

PEP 246 specifically said it wasn't tackling that case, however, and I 
personally don't feel that a singleton of such broad scope is practical; it 
seems more appropriate to make protocols responsible for their own adapter 
registries, which is what PyProtocols (and to a lesser extent Twisted) do.


>Of course, a library writer can define interface classes, and the
>adaptation mechanism will allow concrete classes to be made to
>"conform" to those interfaces, but it's not necessary. And given
>the general apathy of the Python community towards interfaces (at
>least as far as I see) I don't imagine this being a very popular use
>pattern.

When designing PyProtocols I did some research on the type-sig, and other 
places...  I got the distinct impression that lots of Pythonistas like 
interfaces, as long as they're abstract base classes.  PyProtocols allows 
that.  But, a protocol can also be a pure symbol, e.g.:

import protocols

myInterface = protocols.Protocol()


Voila.  There's your interface.  Let the user register what they 
will.  Sure, it doesn't *document* anything that isn't stated by its name, 
but that's your users' problem.  ;)  At least they can import 
'yourpackage.myInterface' and register adapters now.


>And describing a PEP 246 style facility in terms of
>interfaces could be a big turn-off. (This applies strongly to your
>PyProtocols code - I looked for something like the pep246.register
>function I suggested above, but I couldn't manage to wade through all
>the IThis and IThat stuff to find it, if it was there...)

Unfortunately, I biased the reference documentation towards explaining the 
architecture and what you need to do to integrate with the framework 
itself, even though 90% of the actual intended audience of PyProtocols will 
never need to do anything like that.  In a sense, you could say that the 
current reference manual is much more of an embedding and extending manual, 
instead of being the Python tutorial.


>Once again, I apologise if I've missed your point. But I hope I've
>explained what I see as the point of PEP 246, and where your proposal
>is going off in a different direction.

I understand what you're saying, although I'm not sure how you interpreted 
PEP 246 that way, when it explicitly states that it *doesn't* cover case 
"e".  And I didn't diverge from PEP 246 in that respect.  What I 
effectively said is, "well, let's make it so that case "e" is irrelevant, 
because there are plenty of __adapt__-capable protocols, and because you 
can register the relationships between them."  Thus, the solution to case 
"e" is effectively:

Don't use built-in types as protocols!


>But the fact remains, that neither PEP 246 nor PyProtocols has any
>need to be in the core, or even in the standard library (yet). Wide
>usage could change that. Usage in something that is destined for the
>standard library could, too, but that one could go the other way (the
>"something" gets rejected because it depends on PEP246/PyProtocols).

Right, that was why I was asking.


>PS This is *way* off-topic for python-dev by now. I suggest that we
>    leave things here. I don't have anything else worth adding...

I suppose we could always resurrect the types-sig....  Although, as Tim 
Peters has frequently suggested, nothing ends discussion more quickly than 
having an implementation available.  :)