[Python-3000] Generic functions

[Guido van Rossum]

On 4/3/06, Ian Bicking <ianb at colorstudy.com> wrote:
> As an alternative to adaptation, I'd like to propose generic functions.
>   I think they play much the same role, except they are much simpler to
> use and think about.

Given that Phillip Eby is another proponent of generic functions I
seriously doubt the latter. Certainly your post causes me to produce
an endless list of questions (some of which I'll interject below).

> Though RuleDispatch offers considerably more features through predicate
> dispatch, it would probably be best to just consider type based
> dispatch, as that's more equivalent to adaptation.

I may have missed some context (I have so far skipped all threads
mentioning adaptation), but since you're starting a new thread here,
could you perhaps explain what RuleDispatch refers to?

> So, the copy_reg module is one example where adaptation has been
> proposed.  The idea is that you adapt an object to a pickleable object,
> or something along those lines.

That definition doesn't sound right; but since you are waving your
hands and I haven't yet read the thread about adaptation and copy_reg
I'll leave it at that rather than give me own version of how copy_reg
could be seen as adaptation (actually I'm not sure yet).

> It's a little vague, because while you
> typically adapt an instance coming in, you "adapt" a string to a new
> instance on the way out.

You only use adaptation for pickling; for unpickling all the
information is in the pickle data. Pickling is a thoroughly asymmetric
API.

> Or, I dunno, it's not clear to me.  In fact,
> though that's the typical example, I'm going to bail on that because the
> pickling protocol is an aside to this and too complex for me to digest
> right now.  pprint is a lot easier, and conveniently is much nicer with
> generic functions than adaptation ;)

Watch it though. it may be a great example to explain generic
functions. But it may be the only example, and its existence may not
be enough of a use case to motivate the introduction of gneric
functions.

> Anyway, pprint could work like:
>
> class PrettyPrinter:
>      @generic
>      def pformat(self, object):
>          """Return the pretty string representation of object"""
>          return repr(object)
>      # pformat is now "more" than just a function, it's a callable
>      # object that does type-based dispatch using an internal registery
>      # of implementations, with the implementation above as the fallback.

Whoa! First of all, my gut reaction is already the same as for
adaptation: having a single global registry somehow feels wrong. (Or
is it not global? "internal" certainly sounds like that's what you
meant; but for methods this seems wrong, one would expect a registry
per class, or something like that.)

Second, I'm curious if the fact that the argument name is 'object'
(which is also a built-in type) is an accident, or has significance.

Next, I wonder what the purpose of the PrettyPrinter class is. Is it
just there because the real pprint module defines a class by that
name? Or does it have some special significance? Are generic functions
really methods? Can they be either?

> # It also now can be used as a decorator that registers implementations:
> @PrettyPrinter.pformat.when(object=list)
> def pformat_list(self, object):
>      s = '['
>      for item in object:
>          s += (' '*self.indent) + self.pformat(item) + ',\n'
>      return s + (' '*self.indent) + ']'

Ah, the infamous "when" syntax again, which has an infinite number of
alternative calling conventions, each of which is designed to address
some "but what if...?" objection that might be raised.

If pformat is a method of the quite ordinary class PrettyPrinter, why
isn't the pformat_list() method/function declared inside that class?

Sooner or later the name conflict between your argument and the
built-in type is going to cause problems, either because you need to
access the built-in type or because the reader is confused.

What does when(object=list) mean? Does it do an isinstance() check?

Is there any significance to the name pformat_list? Could I have
called it foobar? Why not just pformat?

> Some things to note:
>
> * There's no interface created here.  There's no hidden interface
> lurking in the background either.

You seem to be expecting a particular objection that I don't have. :-)
Never in a thousand years would I have thought of interfaces here.
What context am I missing?

> * It requires cooperation from the original function (pformat -- I'm
> using "function" and "method" interchangably).

Thereby not helping the poor reader who doesn't understand all of this
as well as you and Phillip apparently do.

> It does not require any
> cooperation from classes like list, similar to adaptation and dissimilar
> to magic methods.

What do you mean here? Is list used because it appears in the when clause?

I'm guessing that you are contrasting it with len(), which could be
seen as a special kind of built-in "generic function" if one squints
enough, but one that requires the argument to provide the __len__
magic method. But since len() *does* require the magic method, doesn't
that disqualify it from competing?

> Adaptation also requires cooperation from the caller,
> as the adaptation would be applied inside pformat.

Huh? Aren't you contradicting yourself here? If the adaptation is done
inside pformat (being the callee) what does the caller have to do
(except from the obvious "call pformat")?

> * The function is mostly self-describing.

Perhaps once you've wrapped your head around the when() syntax. To me
it's all magic; I feel like I'm back in the situation again where I'm
learning a new language and I haven't quite figured out which
characters are operators, which are separators, which are part of
identifiers, and which have some other magical meaning. IOW it's not
describing anything for me, nor (I presume) for most Python users at
this point.

> If it has certain return
> values, then you state what those values are in documentation; there's
> no need to be formal about it.  In contrast you have to come up with a
> whole collection of interfaces to start using adaptation.
>
> * The function is the hub of all the registration, which seems very
> natural, since you are extending the function.

Tell us more about the registration machinery. Revealing that (perhaps
simplified) could do a lot towards removing the magical feel.

> * Like adaptation, you must import a module that defines extra
> specializations of the generic function before those are active (just
> like you have to import adapter declarations).  This strikes me as a
> significant problem.  I assume ZCML addresses this, but I also assume
> that's not a reasonable solution for core Python.

You have to import these modules for their side effects (on the
registry), not so  much because they make some objects or names
available that you use directly, right?

> * Magic methods do *not* have this import problem, because once you have
> an object you have all its methods, including magic methods.

Well, of course that works only until you need a new magic method.

> RuleDispatch has a bunch more features than just simple type-based
> generic functions.  But I think that type-based generic functions would
> be an easier or more comfortable place to start, and wouldn't preclude a
> more featureful implementation later.

If RuleDispatch is the thing defining Phillip's full when() syntax,
yes, please focus on the simpler rules.

> Type-based generic functions and adaptation are more-or-less equivalent.
>   That is, you can express one in terms of the other, at least
> functionally if not syntactically.

Could you elaborate this with a concrete example?

> If you really wanted adaptation,
> then the interface becomes a things-obeying-this-interface factory --
> i.e., a generic function.  Generic functions are similer to
> multi-adaptaters, where you adapt a tuple of objects, similar to the
> tuple of arguments to a function.  This is technically like generic
> functions, but syntactically rather awkward.
>
> [Predicate-based dispatching goes considerably further, allowing real
> duck typing, e.g., you could implement a pformat method for everything
> that has an "__iter__" method and no "next" method (i.e., all iterables,
> but not iterators which could lead to unintentionally consuming the
> iterator).]
>
> Anyway, I think generic functions are very compatible with Python syntax
> and style, and Python's greater emphasis on what an object or function
> can *do*, as opposed to what an object *is*, as well as the use of
> functions instead of methods for many operations.  People sometimes see
> the use of functions instead of methods in Python as a weakness; I think
> generic functions turns that into a real strength.

Perhaps. The import-for-side-effect requirement sounds like a
showstopper though.

--
--Guido van Rossum (home page: http://www.python.org/~guido/)