[Python-3000] Sane transitive adaptation

[Nick Coghlan]

Guido van Rossum wrote:
> On 4/7/06, Nick Coghlan <ncoghlan at iinet.net.au> wrote:
>> I've been stealing liberally from PyProtocols all along, usually trying to
>> grasp ways to take *concepts* from it that I liked, and turn them into
>> something that seemed *usable* (with aesthetics playing a large part in that).
> 
> Perhaps you could give us a hint on where the parts of PyProtocols
> that you like are documented on the web? Scanning the
> peak.telecommunity.com site I always seem to get lost. For example, I
> have so far had no luck finding either the source code or the docs for
> RuleDispatch there. I got tantalizingly close when I found a reference
> to a module named 'dispatch' in core.py in the peak svn tree, but
> there was nothing named dispatch or dispatch.py nearby... :-( The "API
> docs" similarly apear as a forest of dead leaves to me.

Mostly just the API docs:
http://peak.telecommunity.com/protocol_ref/module-protocols.html

This was the specific page about using a concrete API on the protocols themselves:
http://peak.telecommunity.com/protocol_ref/protocols-calling.html

> 
>> That's why one of the first things I hit on in rereading PEP 246 was "hang on,
>> why are we use a pair of ad hoc protocols to build an adaptation system"? And
>> lo and behold, PyProtocols had a much easier to use system based on concrete
>> protocol objects, where you manipulated the target protocol directly.
>>
>> And my last message about adaptation, which was a (massively) simplified
>> version of some of the ideas in PyProtocols, let me understand how
>> transitivity would apply in a generic function context: you have function_A
>> with a certain signature, and existing operations and function_B that can be
>> implemented in terms of function_A.
> 
> This is clear as mud to me.

It's not really that important, except in convincing me personally that 
overloadable functions really are a superset of protocol adaptation.

Another way of looking at it:

   1. Suppose function A and function B do the roughly the same thing

   2. Suppose function A can be overloaded/extended

   3. It would then be straightforward to add additional extensions to
   function A that delegated certain call signatures to function B (possibly
   with a wrapper to deal with slight differences in the order of arguments or
   the type of the return value.

Turning that into *transitivity* is a matter of asking the question, "well , 
what if B can be overloaded as well?". The idea then is that you would pass B 
a callback to be invoked whenever a new signature was registered with B. The 
callback would take the new signature, rearrange it if necessary, and register 
the new version with function A.

   @function_B.call_for_new_registrations
   def extending_B_extends_A_too(signature, extension):
       # The decorator means this function would be called whenever a new
       # extension was registered with function B
       # This particular callback implements the rule "any extension of B
       # is automatically an extension of A as well"
       function_A.register(signature)(extension)

(using PJE's terminology here, because I'm not sure of the right word for the 
extending functions when the concept is called "overloading")

>> I like being able to use 'signature' and 'specialization' as the terms, too,
>> as they describe exactly what's going on.
> 
> Not to me. I think of "signature" as a tuple of types, and maybe
> that's what you mean it to describe. But what's a specialization? Is
> it a callable? Is it a string? What properties does a specialization
> have?
> 
> (This reminds me of the kind of conversation I sometimes have with
> Orlijn. Sometimes he'll say "I saw a frizzlefrazzle today!" I'll say
> "what's a frizzlefrazzle?" He'll say something that sounds to me like
> "Not frizzlefrazzle, frizzlefrazzle!" After a few tries I give up, and
> I have to change my line of questioning to get him to describe a
> frizzlefrazzle. The most effective questions are usually "what does a
> frizzlefrazzle do?" or "what does a frizzlefrazzle have?" That's what
> I'm asking you here. :-)

A specialization would be a tailored version of the function registered to 
deal with a particular signature.

An adapter in protocol terms, an extension in PJE's terms, and, as noted 
above, I'm not sure what to call it when the concept is referred to as 
function overloading :)

>> Further, if all builtin function
>> objects (including C function descriptors and method wrappers) were
>> potentially generic,
> 
> This seems a rather extreme anticipation. I don't think anybody has
> assumed this is where we might be going.

There's certain attractive properties to doing it though - if we can get it 
down to the point where the runtime hit is minimal for non-extended functions, 
then it avoids the scenario "if only framework X had made function Y 
extensible, the fixing this would be easy!".

OTOH, if there's an unavoidable call time cost, then a trivial @extensible or 
@overloadable decorator would do the trick.

> Forgetting that str is really a type; assuming str were an @overloaded
> function (I'm sticking to my terminology :-) then I could have written
> that as
> 
> @str.register(SomeClass)
> def str_SomeClass(obj):
>     return obj.stringify()
> 
> I'm not sure though what @specialize(str) adds to this.
> 
> Assuming we can't change str into an @overloaded function but we do
> want to allow this style of overriding, I guess we could have some
> external registry object which str consults before looking for a
> __str__ method or using hardcoded knowledge.

And that's exactly where the benefits of having an "overload" or "extend" 
function comes in. Using PJE's terms, we might write:

   _cls_extensions = {}
   class ClassRegistry(object):
       """This would expose all the methods of an extensible function"""
       def __init__(self, cls):
           self.cls = cls
           self.registry = {}
       def for_signature(self, *signature):
           # This is the same as register()
           # but reads better with the preceding function call
           def helper(f):
               self.registry[signature] = f
               return f
           return helper

   @extend(extend).for_signature(type)
   def get_registry(cls):
       # This function retrieves the extension registry for a class
       try:
           registry = _cls_extensions[cls]
       except KeyError:
           registry = _cls_extensions[cls] = ClassRegistry(cls)
       return registry

  With a change in type.__call__() to check the class extension registry 
before calling cls.__new__(), the above would put us well on the way to making 
all types extensible. For example, we could make a class of our own behave 
like a dictionary, returning its keys when converted to a list, and its 
key-value pairs when converted to a dictionary:

   @extend(list).for_signature(MyMapping):
   def get_keys(obj)
       return obj.keys()

   @extend(dict).for_signature(MyMapping):
   def get_items(obj)
       return obj.items()

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------
             http://www.boredomandlaziness.org