[Python-3000] Adaptation [was:Re: Iterators for dict keys, values, and items == annoying :)]

[Tim Hochberg]

Greg Ewing wrote:
> Nick Coghlan wrote:
> 
> 
>>In this case, it makes far more sense to me to move the adapter 
>>registration out to the individual protocols.
> 
> 
> That would alleviate some of my concerns as well. This
> seems more Pythonic: Namespaces are one honking... etc.
> 
> 
>>In a glorious fit of self-referentiality, one of the first things needed by 
>>such an approach would be a protocol for the protocol interface that allowed 
>>other protocol objects to register themselves as implementing it. This would 
>>then allow the ever popular generic adaptation function to be written as:
>>
>>   def adapt(obj, target_protocol):
>>       return Protocol.adapt(target_protocol).adapt(obj)
> 
> 
> Surely the protocol protocol would be well-known enough
> that other protocols would simply be duck-typed to it,
> rather than having to be adapted to it?
> 
> Also I prefer the idea of the protocol object being
> callable, so you just do
> 
>    target_protocol(obj)
> 
> to do an adaptation.

I've been rewriting copy_reg.pickle to use, or preferably, be replaced 
by an adaption framework. Copy_reg seems to come up every time adaption 
does, and this is the registration part of copy_reg. So far I've done 
generic T->P, generic P->P and this version, let's call it distributed 
adaption. So far distributed adaption seems to be the best. One caveat 
-- I've tried to literarly recreate copy_reg.pickle's behaviour, it's 
possible that something better could be done with a recasting of the 
whole pickle framework or some such. I'm not going to go there.

First, let's look at copy_reg.pickle. I've removed a little bit of it 
that is marked as vestigal so that we can focus on the remainder:

def pickle(ob_type, pickle_function):
     if type(ob_type) is _ClassType:
         raise TypeError("copy_reg is not intended for use with classes")
     if not callable(pickle_function):
         raise TypeError("reduction functions must be callable")
     dispatch_table[ob_type] = pickle_function

In addition to the basic dispatch table, we need to do two things: check 
that pickle_function is a callable and test that ob_type is not an 
instance of _ClassType. The first is probably a good thing for any 
adaption framework to do, so I went ahead and added that check to the 
register call in all cases. That leaves us to deal with the ClassType check.

1. "Generic" Type->Protocol Adaption (T->P adaption):

########################################################################
# In copy_reg
def pickle(ob_type, pickle_function):
     if type(ob_type) is _ClassType:
         raise TypeError("copy_reg ...")
     register_adapter(pickle_func, ob_type, ipickled)

# Usage (same as before)
copy_reg.pickle(some_type, some_pickle_function)
########################################################################

In this case, I couldn't find any way to really use the adaption 
framework to good effect. Sure I managed to shave off two lines of code, 
but that hardly seems satisfying.



2. "Generic" Protocol-Protocol Adaption (P->P adaption):

########################################################################
# In copy_reg
ipickled = Protocol('pickled-object')
def pickleable(ob_type):
     if type(ob_type) is _ClassType:
         raise TypeError("picklable ...")
     return get_type_protocol(ob_type)

# Usage
adaption.register_adapter(some_pickle_function,
                           copy_reg.pickleable(some_type),
                           copy_reg.ipickled)
########################################################################

This seems closer, maybe. At least your actually using the registry that 
we worked to build. Still you haven't gained much, if anything, and it 
seems rather kludgy to be almost doing T->P adaption in a P->P 
framework. If you're wondering, that's what get_type_protocol is doing, 
it's a helper function to help you fake T->P semantics. The fact that 
you want it is probably a bad sign.

When I implemented these, they both seemed to be complexity magnets. The 
basics are quite simple, but it seemed like they would need this helper 
function and that extra feature and so one until the basic simplicity 
gets obscured. P->P was probably somewhat worse in this regard, but both 
schemes suffer from this.


3. Distributed Adaption.

########################################################################
# In copy_reg
class PickleProtocol(adaption.Protocol):
     def register(self, adapter, *types):
         if _ClassType in (type(t) for t in types):
             raise TypeError("%s is not intended for use with cla
         adaption.Protocol.register(self, adapter, *types)
pickler = PickleProtocol('pickle')

# Usage
copy_reg.pickler.register(some_pickle_function, some_type)
########################################################################

This seems more like it. You still haven't saved any signifigant code, 
but this seems clean and extensible. The key feature seems to be that 
each Protocol can override the register and adapt methods (in this case 
I'm using __call__ for adapt as Greg suggested).

Also, in use this code doesn't seem to want to be complicated to nearly 
the same extent that the others do.


If I find time, I plan to also see how the __index__ protocol would have 
looked had it been implemented using each of these frameworks.

The code that implements T->P and P->P adaption is here:
	http://members.cox.net/~tim.hochberg/adaption4.py
The code that implements distributed adaption is here:
	http://members.cox.net/~tim.hochberg/adaption5.py

Be warned that in the first case the module docstring is, at this point, 
ridiculously long and meandering and the code has probably accumulated a 
bit of cruft.

Regards,

-tim