[Python-3000] Special methods and interface-based type system

[Phillip J. Eby]

At 08:29 PM 11/22/2006 -0800, Guido van Rossum wrote:
>One thing that rubs me the wrong way about generic functions is that
>it appears to go against OO. Now I'm not someone to take OO as
>religion, but there's something uncomfortable (for me) about how, in
>Phillip's world, many things become functions instead of methods,
>which brings along concerns about the global namespace filling up, and
>also about functionality being spread randomly across too many
>modules. I fear I will miss the class as a convenient focus for
>related functionality.

I originally proposed a solution for this back in January '05, but it was 
too premature.  But since you have now stated the problem that the proposal 
was intended to solve, perhaps the solution has a chance now.  :)

I will try to be as concrete as possible.  Let's start with an actual, 
hopefully non-exploding 'Interface' implementation, based on an assumption 
that we have generic functions available:

     class InterfaceClass(type):
         def __init__(cls, name, bases, cdict):
             for k,v in cdict.items():
                 # XXX this should probably skip at least __slots__,
                 #     __metaclass__, and __module__, but oh well
                 cdict[k] = AdaptingDescriptor(v)

     class Interface:
         __metaclass__ = InterfaceClass
         __slots__ = '__self__'

         def __init__(self, subject):
             # this isinstance() check should be replaced by an
             # 'unwrap()' generic function, so other adapter types
             # will work, but this is just an example, so...
             if isinstance(subject, Interface):
                 subject = subject.__self__
             self.__self__ = subject

     class AdaptingDescriptor:
         def __init__(self, descriptor):
             self.wrapped = descriptor
         def __get__(self, ob, typ=None):
             if ob is None:
                 return self
             return self.wrapped.__get__(ob.__self__, typ)

Now, using this new "interface framework", let's implement a small 
"mapping" typeclas... er, interface.

     class Mapping(Interface):
         def keys(self):
             return [k for k,v in self.items()]
         def items(self):
             return [k,self[k] for k in self.keys()]
         # ... other self-recursive definitions

What does this do?  Well, we can now call Mapping(foo) to turn an arbitrary 
object into something that has Mapping's generic functions as its methods, 
and invokes them on foo!  (I am assuming here that normal functions are 
implicitly overloadable, even if that means they change type at runtime to 
do so.)  We could even use interfaces for argument type declarations, to 
automatically put things in the "right namespace" for what the code expects 
to use.  That is, if you declare an argument to be a Mapping, then that's 
what you get.  If you call .keys() on the resulting adapted object and the 
type doesn't support the operation, you get an error.

Too late a form of error checking you say?  Well, make a more sophisticated 
factory mechanism in Interface.__new__ that actually creates (and caches) 
different adapter types based on the type of object being adapted, so that 
hasattr() tests will work on the wrapped type, or so that you can get an 
early error if none of the wrapped generic functions has a method defined 
for the target type.

A few important points here:

1. A basic interface mechanism is extemely simple to implement, given 
generic functions

2. It is highly customizable with respect to error checking and other 
features, even on a per-user basis, because there doesn't have to be only 
one "true" Interface type to rule them all (or one true generic function 
type either, but that's a separate discussion).

3. It allows interfaces to include partial implementations, ala Ping and 
Alex's past proposals, thus allowing you to implement partial mapping or 
"file" objects and have the rest of the interface's implementation filled 
in for you

4. It allows you to hide the very existence of the notion of a "generic 
function", if you prefer not to think about such things

5. It even supports interface inheritance and interface algebra: 
subclassing an interface allows adding new operations, and simple 
assignment suffices to compose new interfaces, e.g.:

     class MappingItems(Interface):
         items = Mapping.items

Notice that nothing special is required, this "just works" as a natural 
consequence of the rest of the implementation shown.

Okay, so now you want to know how to *implement* a "Mapping".  Well, 
simplest but most tedious, you can just register operations directly, e.g.:

      class MyMapping:
          def __init__(self, data):
              self.data = dict(data)
          defop operator.getitem(self, key):
              return self.data[key]
          defop Mapping.items(self):
              return self.data.items()

But as you can imagine, this would probably get a bit tedious if you're 
implementing lots of methods.  So, we can add metaclasses or class 
decorators here to say, "I implement these interfaces, so any methods I 
have whose names match the method names in the interfaces, please hook 'em 
up for me."  I'm going to leave out the implementation, as it should be a 
straightforward exercise for the reader to come up with many ways by which 
it can be accomplished.  The spelling might be something like:

     class MyMapping:
         implements(Mapping)

         def items(self):
             ...

         #etc.

At which point, we have now come full circle to being able to provide all 
of the features of interfaces, adaptation, and generic functions, without 
forcing anyone to give up the tasty OO flavor of method calls.  Heck, they 
can even keep the way they spell existing adaptation calls (e.g. IFoo(bar) 
to adapt bar to IFoo) in PEAK, Twisted, and Zope!

And finally, note that if you only want to perform one method call on a 
given object, you can also use the generics directly, e.g. 
Mapping.items(foo) instead of Mapping(foo).items().

Voila -- generic goodness and classic OO method-calling simplicity, all in 
one simple to implement package.  It should now be apparent why I said that 
interfaces are trivial to implement if you define them as namespaces for 
generic functions, rather than as namespaces for methods.

There are many spinoffs possible, too.  For example, you could have a 
factory function that turns an existing class's public operations into an 
interface object.  There are also probably also some dark corners of the 
idea that haven't been explored, because when I first proposed basically 
this idea in '05, nobody was ready for it.  Now maybe we can actually talk 
about the implications.