[Python-ideas] Democratic multiple dispatch doomed to fail (probably)

[Neil Toronto]

Stephen J. Turnbull wrote:
> Neil Toronto writes:
> 
>  > Python's __op__/__rop__ is a rather clever solution to the problem in 
>  > binary form. Do you know if there's a generalization of this? CLOS is 
>  > almost it, but Python lets the class containing __op__ or __rop__ own 
>  > the operation after it finds one that implements it.
> 
> Sorry, no.  I'm a professional economist (my early work was related to
> Gibbard-Satterthwaite, thus my interest here) and wannabe developer,
> who happens to have enjoyed the occasional guidance of Ken Arrow a
> couple decades ago.

Excellent.

> If you think there's something to the application
> of multiobjective optimization to this problem, I'd love to take a
> hack at it ... no classes to teach winter term, I've got some time to
> bone up on the multidispatch problem itself.

For finding the first matching function, you could say it's picking some 
"best fit" from the Pareto optimal front. AFAIK all ways of doing that 
suck somehow. :) After that, the problem is calling a next-method, which 
requires imposing a total order on the entire space of fitting 
functions. (This had better define the "best fit" as well.)

The big hurdles to multiple dispatch I see are:

1. It has to "make sense" somehow - maybe by satisfying some set of 
intuitive axioms.
2. It seems unlikely a majority will agree on the axioms. (Yay! Another 
social choice problem!)
3. It has to be simple enough that it's easy to predict what a method or 
next-method call does.

Because this seems nigh-impossible (particularly #3), I'm starting to 
look elsewhere for solutions to "the problem". Speaking of which, it's 
really easy to forget the two primary use-cases that motivate multiple 
dispatch in the first place:

1. Interop with other types. For instance, I define a BarNumber type, 
and I want fooing of BarNumber with anything else to return a new 
BarNumber. Or perhaps I need some class I currently would have no 
control over to treat my custom type specially or apply a transformation 
before doing what it usually does. CLOS (Common Lisp Object System) 
handles this. Python's __op__/__rop__ does for a finite set of binary 
operators.

2. True dynamic overloading so we don't have to do type checks at the 
heads of our functions. CLOS handles this.

#1 can always be done with clever delegation. For example, numpy's 
arrays do it halfway by applying __array__ to a function argument before 
operating on it. If they also applied, say, a type's fromarray 
classmethod upon return it'd provide a full solution.

But as with numpy's arrays, users don't have enough control to make it 
work exactly as they want if the only solution is delegation. The class 
creator has to anticipate how his class will be used and risk 
over-generalizing it. In the worst cases, you'd end up with a crazy 
proliferation of types like FooLoggable and FooAppendable and methods 
like __foo_loggable__ and __foo_appendable__...

The most general way to do #2 is the way it's done now. Sure it's 
Turing-complete, but it'd be nice to be able to handle the common cases 
automatically.

PJE has a lot of further reading here:

   http://mail.python.org/pipermail/python-3000/2006-November/004421.html

Neil