[Python-3000] Adaptation vs. Generic Functions

[Michael Chermside]

Guido writes:
> the conventional way of treating __add__
> etc. has a different need for resolving conflicts than Phillip Eby's
> idea of requiring strict dominance of the selected solution over all
> other candidates. So now that you mention it I wonder if the whole
> strict dominance requirement isn't a red herring? And perhaps we
> should in general use "returns NotImplemented" as a signal to try the
> next best candidate, strict dominance be damned...

Maybe I'm getting confused here, but I disagree... as I look at the
__add__()/__radd__() example it convinces me that Phillip's strict
cominance requirement is very wise. I'll talk through my reasoning
with what I consider a realistic use case and someone else can write
in and tell me what obvious point I'm missing.

Okay, Guido fires up the time machine and goes back to early Python
and instead of introducing __add__()/__radd__() in the first place,
he uses operator.add(a,b) with overloading / multiple-dispatch /
generic-functions / whatever-we-call-it.

So then the mpz library introduces arbitrary precision numbers based
on the Gnu library. They create a type "mpz" and define the following:

     @operator.add.register(mpz, mpz)
     def add(mpz1, mpz2):
         ... # efficient mpz add

     @operator.add.register(mpz, object)
     def leftAdd(mpz1, x):
         ... # convert x and add, returning a mpz

     @operator.add.register(object, mpz)
     def rightAdd(x, mpz1):
         ... # convert x and add, returning a mpz

These work properly and all is good. Meanwhile, somewhere else the
NumPy folks (in this history they named it "NumPy" first then changed
it to "Numeric" later... it's just this sort of thing that makes it
so confusing to mess around with the time machine), are inventing
some nifty arrays, which are able to be added to each other (if the
sizes are appropriate) or to integers. They create an type named
"array" and define the following:

     @operator.add.register(NumPy.array, NumPy.array)
     def add(array1, array2):
         ... # add component-wise or raise ShapeMismatchException

     @operator.add.register(NumPy.array, object)
     def leftAdd(array1, x):
         ... # add x to each component

     @operator.add.register(object, NumPy.array)
     def rightAdd(x, array1):
         ... # add x to each component

Again, all is well. Everything a user wants to do has a clear
dominance.

... Until the day when Suzy Programmer chooses to import *both*
mpz AND NumPy. Actually, that still doesn't cause any problems
until she chooses to add the two types, with this line:

     my_x = my_mpz + my_array

THIS then raises an exception because there is no one dominant
definition. To me, that seems like a Good Thing. It seems rather
Pythonic that there is no attempt to guess what to do, either
the programmer makes it very clear and indicates the correct
behavior by registering a function, or they don't, and Python
refuses to guess. "Explicit is better than Implicit", "In the
Face of Ambiguity, Refuse to Guess". If Suzy really knows what
she wants to do, she can register her own operator.add(), which
shouldn't take more than a couple of lines -- I can't imagine
that a couple of lines is too much of a burden to impose on
someone trying to integrate two unrelated libraries!

But in a more realistic situation, the NumPy folks realize that
many of their users are doing scientific work and a noticable
number make use of mpz. (The mpz folks are just wrapping an
external library so they don't think of these things, but that's
OK since only one of them needs to.) The NumPy folks add the
following lines of code:

     try:
         import mpz

         @operator.add.register(NumPy.array, mpz.mpz)
         def leftAdd(array1, mpz1):
             ... # invoke mpz.rightAdd() on each component

         @operator.add.register(mpz.mpz, NumPy.array)
         def rightAdd(mpz1, array1):
             ... # invoke mpz.leftAdd() on each component

     except ImportError:
         pass

And now Suzy can effortlessly integrate the libraries.

I realize that it is *possible* for two different integrators
to invent competing definitions of what to do when combining a
mpz and NumPy.array, and it would be nice to avoid having the
registry be global... but somehow I can't get _too_ excited
about it -- somehow it seems like it wouldn't arise too often
in practice. Perhaps there's a clever solution similar to
Decimal contexts, involving use of a "with" statement... but
it doesn't feel like a roadblock. And the inability of the two
libraries to interface *automatically* in a protocol designed
for adding numbers, which never quite expected either library
feels like an advantage, not a weakness.

-- Michael Chermside