[Python-Dev] Re: PEP 246 and Protocols (Was: Sneaky 'super' instances)

[Clark C. Evans]

Paul,

  The goal of PEP 246 is to create a loose coupling between various
  framework components to facilitate the 'adapter' pattern as described
  in the GOF pattern book.   It is also motivated by Microsoft's
  QueryInterface, which when used intelligently is extremely powerful.

  The primary audience of PEP 246 are component/library developres
  who wish for their components to interoperate between frameworks.
  Unless you are writing components and wish for them to work across,
  say Zope, Twisted, and Webware, using a single code base in a
  managable manner then you may not understand the PEP.   In particular,
  PEP 246 will be successful if your average programmer does not
  even know about adapt!  He simply pluggs component X into framework
  Y and it works... automagically. 

  For example, I have a 'Flow' manager (a generator based cooperative
  multitasking library).   Currently it is dependent upon twisted, 
  however, there are really only two protocols I depend upon:
    (a) wrapping an exception for later delivery, 
    (b) some sort of event loop for scheduling delayed execution.
  With PEP 246 I'd write an interface flow.IFailure and flow.IReactor,
  and provide clean ways for it to work with Twisted.  Then, if some
  other framework wanted to "re-use" my component, it'd be easy, they'd
  simply add a __conform__ to their objects, and __adapt__ my objects.
  This is far more powerful than requiring my code or their code to
  directly know about each other and the interface mechanism that each
  of them uses.    

I hope this helps,

Clark

On Thu, Jun 12, 2003 at 12:48:51PM -0400, Phillip J. Eby wrote:
| At 03:42 PM 6/12/03 +0100, Moore, Paul wrote:
| >From: Phillip J. Eby [mailto:pje@telecommunity.com]
| >> Open protocols solve the chicken and egg problem by allowing one
| >> to make declarations about third-party objects.
| >
| >OK, I've tried to put together a simple example of "what I expect" and
| >I find I can't. I want to continue to write code which "just assumes"
| >that it gets the types it needs - I don't want to explicitly state that
| >I need specific interface types - that feels like type declaration and
| >Java. Your IFile example reinforces that, both in terms of its naming
| >convention, and in the assumption that there *is* a single, usable,
| >"file" protocol. I know it was my example, but I pointed out later in
| >the same message that I really sometimes wanted seekable readline-only
| >files, and other times block read/write (potentially unseekable) files.
| >
| >Expecting library writers to declare interface "classes" for every
| >subtle variation of requirements seems impractical. Expecting the
| >requirements to be *documented* is fine - it's having a concrete class
| >which encapsulates them that I don't see happening - no-one would ever
| >look to see if there was already a "standard" interface which said "I
| >have readline, seek, and tell" - they'd just write a new one. There
| >goes any hope of reuse. (This may be what you see as a "chicken and egg"
| >problem - if so, I see it as a "hopelessly unrealistic expectations"
| >problem instead, because it's never going to happen...)
| 
| Even if it's as simple as saying this:
| 
|     class ReadlineSeekAndTell(Interface):
|         """You must have readline, seek, and tell if you pass this to me"""
|         advise(protocolIsSubsetof=[IFile])
| 
| 
| Or, even more briefly, using Samuele's way:
| 
|     RST = subproto(IFile,['readline','seek','tell'])
| 
| 
| 
| >On the other hand, expecting callers to write stuff to adapt existing
| >classes to the requirements of library routines is (IMHO) a non-issue.
| >I think that's what PEP 246 was getting at in the statement that "The
| >typical Python programmer is an integrator" which you quote. It's
| >common to write
| >
| >    class wrap_source:
| >        def __init__(self, source):
| >            self.source = source
| >        def read(self, n = 1024):
| >            return self.source.get_data(n)
| >
| >    lib_fn(wrap_source(my_source))
| >
| >So PEP 246 is trying to make writing that sort of boilerplate easier
| >(in my view). The *caller* should be calling adapt(), not the callee.
| 
| That depends.  It's common in both Zope and Twisted for framework code to 
| do the Zope or Twisted equivalents of 'adapt()'.  But yes, it is also 
| common to state that some function in effect requires an already adapted 
| object.
| 
| 
| ><snip>
| >Which is appropriate is basically down to which came first, string or
| >file. But both suffer from the problem of putting all the knowledge
| >in one location (and using a type switch as well).
| 
| Right.  That's what the point of the open protocols system is.  The problem 
| is that in every Python interface system I know of (except PyProtocols), 
| you can't declare that package A, protocol X, implies package B, protocol 
| Y, unless you are the *author* of package B's protocol Y.
| 
| But if package A's author used an open protocol to define protocol X, then 
| you can use PyProtocols today to say that it implies package B's protocol 
| Y, as long as protocol Y can be adapted to the declaration API.
| 
| Translation: anybody who defines protocols for their packages today using 
| PyProtocols, will be able to have third parties define adapters to 
| interfaces provided by Zope or Twisted, or that are defined using Zope or 
| Twisted interfaces.
| 
| 
| >The third option, which is the "external registry" allows a *user* of
| >the string and file "libraries" (bear with me here...) to say how he
| >wants to make strings look like files:
| >
| ><snip>
| >So I see PEP 246 as more or less entirely a class based mechanism. It
| >has very little to do with constraining (or even documenting) the
| >types of function parameters.
| 
| PEP 246 specifically said it wasn't tackling that case, however, and I 
| personally don't feel that a singleton of such broad scope is practical; it 
| seems more appropriate to make protocols responsible for their own adapter 
| registries, which is what PyProtocols (and to a lesser extent Twisted) do.
| 
| 
| >Of course, a library writer can define interface classes, and the
| >adaptation mechanism will allow concrete classes to be made to
| >"conform" to those interfaces, but it's not necessary. And given
| >the general apathy of the Python community towards interfaces (at
| >least as far as I see) I don't imagine this being a very popular use
| >pattern.
| 
| When designing PyProtocols I did some research on the type-sig, and other 
| places...  I got the distinct impression that lots of Pythonistas like 
| interfaces, as long as they're abstract base classes.  PyProtocols allows 
| that.  But, a protocol can also be a pure symbol, e.g.:
| 
| import protocols
| 
| myInterface = protocols.Protocol()
| 
| 
| Voila.  There's your interface.  Let the user register what they 
| will.  Sure, it doesn't *document* anything that isn't stated by its name, 
| but that's your users' problem.  ;)  At least they can import 
| 'yourpackage.myInterface' and register adapters now.
| 
| 
| >And describing a PEP 246 style facility in terms of
| >interfaces could be a big turn-off. (This applies strongly to your
| >PyProtocols code - I looked for something like the pep246.register
| >function I suggested above, but I couldn't manage to wade through all
| >the IThis and IThat stuff to find it, if it was there...)
| 
| Unfortunately, I biased the reference documentation towards explaining the 
| architecture and what you need to do to integrate with the framework 
| itself, even though 90% of the actual intended audience of PyProtocols will 
| never need to do anything like that.  In a sense, you could say that the 
| current reference manual is much more of an embedding and extending manual, 
| instead of being the Python tutorial.
| 
| 
| >Once again, I apologise if I've missed your point. But I hope I've
| >explained what I see as the point of PEP 246, and where your proposal
| >is going off in a different direction.
| 
| I understand what you're saying, although I'm not sure how you interpreted 
| PEP 246 that way, when it explicitly states that it *doesn't* cover case 
| "e".  And I didn't diverge from PEP 246 in that respect.  What I 
| effectively said is, "well, let's make it so that case "e" is irrelevant, 
| because there are plenty of __adapt__-capable protocols, and because you 
| can register the relationships between them."  Thus, the solution to case 
| "e" is effectively:
| 
| Don't use built-in types as protocols!
| 
| 
| >But the fact remains, that neither PEP 246 nor PyProtocols has any
| >need to be in the core, or even in the standard library (yet). Wide
| >usage could change that. Usage in something that is destined for the
| >standard library could, too, but that one could go the other way (the
| >"something" gets rejected because it depends on PEP246/PyProtocols).
| 
| Right, that was why I was asking.
| 
| 
| >PS This is *way* off-topic for python-dev by now. I suggest that we
| >   leave things here. I don't have anything else worth adding...
| 
| I suppose we could always resurrect the types-sig....  Although, as Tim 
| Peters has frequently suggested, nothing ends discussion more quickly than 
| having an implementation available.  :)
| 
| 
| _______________________________________________
| Python-Dev mailing list
| Python-Dev@python.org
| http://mail.python.org/mailman/listinfo/python-dev