[Python-3000] Adaption & generic functions [was Generic functions]

[Guido van Rossum]

On 4/4/06, Tim Hochberg <tim.hochberg at cox.net> wrote:

(Could you not change the subject each time? Or if you do, could you
assume the reader hasn't necessarily read your previous posts? For
gmail users like me, each subject change starts a new thread -- like
in newsgroups -- and having so far ignored any thread that's grown to
100 messages or so I eagerly jump into any new thread; only to find
that it's hard to follow the first message there since it's assuming I
read the tail end of that other thread.)

(At least you included your Protocol class at the end, making your
post slightly self-contained.)

> Considering generic function in combination adaption led to one more
> small change in the Protocol implementation that I'm playing with, and
> all of the sudden I'm left with something that I could actually use.

Cool. Could you comment on the Protocol implementation I posted separately?

> What I realized was that you could factor out the part that potentially
> has a lot of variation and suddenly you have a relatively simple
> framework that's extensible in all sorts of ways. By default, everything
> is the same as in the last iteration, but now it's much easier to change
> the behaviour by subtyping.

Cool. We seem to be converging on a new kind of adaptation
implementation (I call it 2nd generation adaptation) where the
protocol is an object with register() and adapt() methods. In true
duck typing style all we have to do is agree on how to call adapt()
and register(), and whether there are other methods (like a
convenience to register using a decorator). You & I seem to have
chosen slightly different styles, but in essence we're very close.

(Except that I can't see I understand the point of your TV example,
which seems very artificial.)

> Let's return to the case that first bugged me about T->P dispatch. I
> have some tagged values and I want to assemble two of them  to form a
> complex number.
>
> First we need a tagged value type. In reality I'd have this already
>
>  >>> class TV(object):
> ...     def __init__(self, value, tag):
> ...         self.value = value
> ...         self.tag = tag

I suppose this is implementing some theoretical abstraction popular in
some literature you've been reading recently? Us non-academicians
could use some help in gauging the significance of this class.

> Then we need to subclass Protocol. I change the behaviour of keysof so
> that now things are looked up in the registry strictly based on their
> tags.

I can't say I fully fathom the concept of "keysof". Perhaps you can
explain it better and then we can come up with a better name?

(Actually I think now I do understand it, and I still find the name
horrible. It represents all the keys that you want to look up in the
registry looking for an adapter. It gets passed the arguments to
__call__ -- which is really a shortcut to spell adapt in a cute way;
that confused me too -- and the base class assumes __call__ -- or
adapt -- is called with a single argument, like a typical adapt()
call. But your subclass calls it with tagged values. OK, so what you
*really* wanted there was keyword arguments?)

> Note that this can take an arbitrary number of arguments. In the
> default implementation (see below) only a single argument is allowed,
> which gives you the basic adapter(obj) -> newobj behaviour.

Which puzzled me at first (I had to read the Protocol class at the end
of your post before I could understand the rest).

>  >>> class TVProtocol(Protocol):
> ...     def keysof(self, *args):
> ...         try:
> ...             yield tuple(x.tag for x in args)
> ...         except AttributeError:
> ...             pass
>  >>> ascomplex = TVProtocol('complex_from_tagged')

Shouldn't that be

    ascomplex = TVProtocol('ascomplex')

?

> Then I define some converters:
>
>  >>> import cmath
>  >>> @ascomplex.when(('real', 'imag'))
> ... def complex_from_real_imag(real, imag):
> ...     return real.value + 1j*imag.value
>  >>> @ascomplex.when(('mag', 'angle'))
> ... def complex_from_mag_angle(mag, angle):
> ...     return mag.value * cmath.exp(1j * cmath.pi / 180 * angle.value)
>  >>> @ascomplex.when(('db', 'angle'))
> ... def complex_from_db_angle(db, angle):
> ...     return 10**(db.value/20.0) * cmath.exp(1j * cmath.pi / 180 * angle.value)

Can we please get rid of the convention of naming the registration
function when()? I don't find it cute at all, and it's not
particularly indicative of what it does (registration).

> Here's some values that I can assume came from elsewhere:
>
>  >>> tv_re, tv_im = TV(1, 'real'), TV(2, 'imag')
>  >>> tv_db, tv_ang, tv_mag = TV(0, 'db'), TV(90, 'angle'), TV(2, 'mag')

(Aside: I wonder if this would look less awkward if you changed
TV.__init__ so that you can write TV(real=1), TV(imag=2) etc.)

> And here's how I'd use it:
>
>  >>> ascomplex(tv_re, tv_im)
> (1+2j)
>  >>> ascomplex(tv_db, tv_ang)
> (6.1230317691118863e-017+1j)
>  >>> ascomplex(tv_mag, tv_ang)
> (1.2246063538223773e-016+2j)
>  >>> ascomplex(tv_db, tv_mag)
> Traceback (most recent call last):
>  ...
> ValueError: adapter not found

Or is anyone of these calling keysof() with multiple args?

> All of the sudden this is looking like something I could probably use.

I'd like to hear what you think of my version and how you'd refactor it.

> I also tried a simple generic function implementation on top of this (no
> inheritance, keysof just returned a tuple of types). That was also easy.

Post this, please!

> Could full blown generic dispatch be added just by subclassing and
> adding the correct, and obviously much more complex, version of keysof?
> It seems likely, but I'm not certain. If so, this is starting to look
> like a very promising approach.

It would be useful to compare this to adaptation built on top of
generic functions. Alex seems to be convinced that adaptation is more
powerful than generic functions; but he hadn't considered the
possibility of having a generic function that's a factory (like the
iterator-factory in my example). I don't know if that affects his
opinion though; he seems to find it important that adaptation can
return an object that has multiple methods that belong together.

> The updated Protocol implementation is below.
>
> class Protocol(object):
>     all_protocols = set()

What's the signigificance of all_protocols? You're not using it. The
key attraction of this class is that it *doesn't* need to keep track
of all protocols. In fact, IMO there's no use for concepts like "all
protocols that object X implements". Perhaps there could be a need for
this in a narrower concept (I can't rule out that Zope has some use
for this) but that could always be done by requiring all *relevant*
protocols subclass a certain base implementation that keeps track of
this. For general Python protocols I don't think it's necessary. For
that matter, after reading Alex's seminal post, I don't see any reason
why protocols should have anything to do with interfaces (although
it's fine for some framework's interfaces to be protocols).

>     def __init__(self, name, doc=''):
>         self.name = name
>         self.registry = {}
>         self.__doc__ = doc
>         self.all_protocols.add(self)
>     def __repr__(self):
>         return "<protocol %r>" % self.name
>     __str__ = __repr__
>     def __call__(self, *args):
>         for key in self.keysof(*args):
>             adapter = self.registry.get(key, None)
>             if adapter is not None:
>                 return adapter(*args)
>         raise ValueError('adapter not found')

So __call__ is what used to be call adapt. Or, rather, where Alex used
to write adapt(x, P) and where I write P.adapt(x), you just write P().
Clever.

Perhaps this could become the key to generic functions on top of
adaptation? That would be revolutionary!

>     def keysof(self, *args):
>         if len(args) != 1:
>             raise TypeError("%s expects 1-argument, got %s" (self, len(args)))
>         obj = args[0]
>         mro = type(obj).__mro__
>         for cls in mro:
>             yield cls
>     def register(self, adapter, *types):
>         if not callable(adapter):
>             raise TypeError("adapters must be callable")
>         for t in types:
>             self.registry[t] = adapter
>     def when(self, *types):
>         def decorator(adapter):
>             self.register(adapter, *types)
>             return adapter
>         return decorator

Sorry for the thoroughly random nature of this post. I kept going back
and forth and now I have to go -- but I still want to post it.

--
--Guido van Rossum (home page: http://www.python.org/~guido/)