[Python-Dev] proposal for interfaces

[Esteban U.C.. Castro]

Thanks for your reply-- and thanks for taking the work of fixing the=20
formatting in the quotations. I thought it would maybe be considered=20
spam to send the message again just to fix the mess. (I think I know
what caused the problem last time; I hope this one reads fine).=20
Is there some sort of policy on this?


> Well, aside from the ego boost of having my very own PEP, I think the
> nature of interfaces is such that they're vastly more useful if lots=20
> of people use them. Putting something in the standard distribution=20
> almost guarantees that.

Agreed. I was just curious about what is exactly that _something_ you=20
want to put in the standard distribution. Just a recommendation on the=20
docs? Have some of the existing modules refactored to use this method?=20

[P.S.: I'm afraid your ego will have to give up on having your own=20
interfaces PEP, since there is already one. See below.]


I don't know if there are precedents of code practices being
'officially'=20
endorsed by the python development team, when it implies no changes to=20
either the language or the standard library. To start with a
non-intrusive=20
addition, I can think of a module defining precisely the interfaces of=20
builtin types, and other commonplace de facto interfaces that already=20
exist in the standard library.

Imagine you want to use a function that takes a builtin type, but you=20
want to pass your own fake instance. Making it just inherit the builtin=20
type is an option, but maybe that's not practical, or yours is, too, a=20
type not implemented in python, or you don't want to inherit the=20
behavior of the builtin type.

Having the interface formalized somewhere will help you know what will=20
be expected from your custom object.

Most 'interfaces' in standard python libraries (eg. iterable, iterator,
stream...).  are really simple, and it has worked quite well to have=20
them as just de facto. I think code checking for an interface would be
more expressive than code checking for method names or just trying to=20
use the object and catch exceptions (you can still do this for other=20
reasons if you want). The latter also relies on everyone knowing and=20
agreeing on what makes an int an int, and a list a list.

Now, many functions take lists, but only use a subset of the list=20
interface. It would be handy if common subsets of (e.g.) the list=20
interface could be identified and formalized in a standard module. We=20
could define a SimpleReadList that would only guarantee getting=20
items by index, a SliceRWDList() that would guarantee index and slice
getting, setting, and deletion, and the complete List interface with
all the append, insert, sort methods.=20

[Note: of course it wouldn't be done like this, but I don't have a=20
good idea of what are the most commonly used subsets of the list=20
interface. Names ugly on purpose, so you won't take them seriously=20
:).]

This is also and example of why I see the usefulness or implicit=20
interface satisfaction. This way interface definitions can have=20
'retroactive' effect, so you don't have to mess with the builtin=20
types at all, in order to define how they interact with other objects=20
(this is, to define and use their interfaces).


> As much as I love Python, sometimes I really miss the rigor of more=20
> stongly-typed languages, so rather than trying to make something=20
> very Pythonic, I've tried to go the opposite direction in order to=20
> complent what's there already.

It looks like you are not alone in this-- or at least others have=20
been where you are. You may want to take a look at the homepage for=20
this (retired) Special Interest Group:

http://www.python.org/sigs/types-sig/



While looking for this, I found a PEP, proposed by this group, which=20
addresses interfaces too:

http://www.python.org/peps/pep-0245.html


I think there's a point in continuing this discussion though, as

> Dissenting Opinion
>    This PEP has not yet been discussed on python-dev.


If this is not a good time/place to talk about this, I guess we'll be
warned :).

I got a strong deja-vu reading this PEP. It matches so closely some=20
of the changes I proposed to your model, that I must have read it=20
long ago and then forgotten about it. Still, this suffers from some=20
statically-typed background and may have the problems I pointed about=20
your proposal; it seems to limit itself to the functionality and needs=20
of Java interfaces (the only example I am more or less familiar with).=20

Since there is a group effort behind it, I guess they have taken this=20
into account and still agreed on this solution for some reason; I=20
haven't looked at the archives yet.

Also, this PEP is less shy about proposing changes to the language=20
itself. This is maybe a good idea; if one of the important features=20
for the usefulness of interfaces is that they will be widely used, it
would help if there was only one universally accepted way to use=20
them. Endorsing one such way in the language itself should help.


> Actually I wanted to have attributes, too (and operators, since
they're
> just methods).  This brings up one of the uses I had in mind for the
> prefixes in front of the method names.  To add a property, you could
do
> something like this:
>
>  doc_myProperty =3D "docstring for myproperty"
>  readonly_myOtherProperty =3D "docstring for a read-only property"
>  writeonly_myLastProperty =3D "docstring for write-only property"

Again, why not just:

 #   name    access         docstring
 myProperty =3D "rw", "docstring for myProperty"
 myOtherProperty =3D "r", "docstring for a read-only property"
 myLastProperty =3D "w", "docstring for write-only property"


> This would require implementions to either have properties named
> "myProperty", "myOtherProperty" and "myLastProperty", or define
methods
> named __get__myProperty, __set__myProperty, __get__myOtherProperty,
and
> __set__myLastProperty.
       =20
It could just require that implementations can be called getattr and/or
setattr (depending on the access declared). There is the problem to
check
setattr non-destructively when the object is read-only. Maybe this is an

issue somewhere else too? I wish this can be solved to keep the syntax
as=20
simple as possible.=20



> Something I didn't include in the original message was the
> design-by-contract feature, which would allow pre- and postconditions
to
> be specified for any method, like this:
>
>  def check_foo(self, a, b):
>    "docstring for foo"
>    if not (some precondition for foo):
>      raise ExceptionOfYourChoice
>    if not (some other precondition for foo):
>      raise DifferentExceptionOfYourChoice
>    return lambda result: (postconditions of foo) # optional


I think the ability to set attributes on python functions, or builtin=20
properties (I'd have to refresh my memory on these :) could be used for=20
this. Either way, the syntax for the client user could be something=20
like:

 class SomeInterface(interface):=20
   def foo(self, a, b): "foo doc"
   def bar(self, a, b): "bar doc"
  =20
   foo.__pre__ =3D some_checking_func # takes a, b, raises something if=20
						# they're wrong

   foo.__post__ =3D other_checking_func # takes the return value, makes
sure
						  # it's not broken

   bar.__around__ =3D yet_another_checking_func # this is called =
_instead_
of
							    # the
function, and _should_
           						    # call the
function in turn


This makes it more explicit that __pre__, __post__ and/or __around__ are

something that relates to foo in some way. Your second approach (bar,=20
__default__bar) comes closer to this, and it may be more convenient than

first defining the function, then assigning it. Having part of the name=20
of an object have an special meaning is convenient but a bit hacky. I=20
myself do it often, but I don't think I'd propose a standard based on=20
that. A matter of personal taste, I guess.



>>  - For the uses you have given to the prefix_methodname notation so
far,
>>    I don't think it's really needed. Isn't the following sufficient?
>> =20
>>  class Foo(interface):
>>     def foo(self, a, b):
>>         "foo docstring"
>>         # nothing here; no default definition
>>     def bar(self, a, b):
>>         pass # no docstring, _empty definition_
>
> Not IMHO, since I'd want methods with no implementation to raise
> NotImplementedError instead of silently returning None.  One this that
> *could* be done to make the the simplest case (just a docstring?)
> easier (and make the syntax look more Pythonic) would be this:

The absence of 'pass' in an interface method (in foo) would be
considered=20
absence of any default implementation and therefore you'd get an=20
InterfaceError when trying to validate/get proxy on the object if it=20
doesn't implement that method.

In second thought, I admit this is dirty, and I don't know if this magic

is even possible. A more explicit approach could be:

 class Foo(interface):
    def foo(self, a, b):
        "foo docstring"
        raise NotImplementedError

    def bar(self, a, b):
        pass # no docstring, _empty definition_


Or, for consistency with the proposal above (and with the existing PEP):

 class Foo(interface):
    def foo(self, a, b):
        "foo docstring"
        # no definition allowed here, sorry

    def bar(self, a, b):
        "" # empty docstring required at the very least
    bar.__default__ =3D lambda a, b: None
   =20

Now that I think about it, the __underscores__ could maybe be taken out=20
for interface method special attributes. They remind us that we are=20
looking at magic stuff that the 'system' will be using in special ways,
but if you are not expected to assign arbitrary attributes to interface=20
methods, then there is no ambiguity. Aesthetic choice, again.




> I got the idea of defaults from Haskell, where it's common to see an
> interface define methods with mutually recursive default definitions,
> kind of like (to use a somewhat silly Python example) defining __eq__,
> __ne__, __cmp__, etc. all in terms of one another and expecting
> implementations to define enough of the methods that everything works.

I like it! :) I may be being too purist at this, but I still think that=20
doesn't belong in the interface definition. I admit putting defaults=20
there is convenient, but I wish we could find a solution that is both
convenient and keeps implementation details out of the interface=20
definition.

Although the existing interface PEP stresses the separation between=20
interface and class, it provides one possible solution for this: it=20
talks about a deferred() method (in interfaces) that will return a=20
class that implements the interface. In the PEP, it seems this is only=20
meant to provide error reporting, but I guess it could be put to good=20
use in other ways.=20

I admit I don't understand the 'deferred' name :), I'm not sure how=20
that default class would be defined, and whether it is intended to be=20
customizable in the PEP. Having a convenient, standard way to define=20
a default class and associate it with an interface without looking=20
like something intrinsic to it, _that_ would be, IMHO, the ideal=20
solution.
	=09


>> In your example:
>>
>>   def foo_proc(foo_arg):
>>     foo_proxy =3D Foo(foo_arg)
>>     ...
>>     x =3D foo_proxy.foo(a, b)
>>
>> [added a, b again]
>>
>> imagine foo_proc may only really cares that foo_arg is an object that
>> has a foo() method that takes (a, b) arguments (this is all Foo
>> guarantees).
>
> Here's where my compiled language bias comes in.  If you only care the
> foo_arg has a certain method, you don't want to use interfaces at all.
> Using the interface doesn't just imply that foo_arg has a method named
> foo, but also that the method satisfies the requirements laid out in
the
> interface definition.

I agree; this case was only simplified for the sake of exposition. The=20
point here is that the requirements laid out in the interface definition
may possibly be checked on objects (as opposed to classes), at runtime=20
(as opposed to, um, import-time :).=20

If you really want to check the class you can do so in one specific=20
interface.

If you don't want the overhead of checking every time, and you know you
won't be messing with class instances or objects, you may either check
by class, or maybe even cache the results of checking on
classes/objects.



>>    In the second case, if someone has been fiddling with foo_arg or
some
>>    of its base classes, foo_arg.foo() may no longer exist or it may
have
>>    a different signature.
>
> You can't really stop people from shooting themselves in the foot.
> Making methods disappear is black magic in my book.

Very true. This is of course no crucial point. Still, since method calls

are always late-bound I think it only makes sense that restrictions on=20
them _can_, at least, be late-checked.


>> * I agree the declaration had better be included in the class
>>   definition, at least as an option.
>
> I agree, but as an option.

The __implements__ alternative lets you fiddle with interfaces outside=20
the class too:

 # SomeClass itself defined elsewhere
 SomeClass.__implements__ +=3D (Foo,)=20

All in all, the syntax I like the most so far is the one described in=20
the existing PEP.



>> * Declarative better than procedural, for this purpose.
>
> Whether this is declarative of procedural is mostly a matter of
> perpective, IMHO.  In my imagination, calls to bind occur only at the
> module level, and almost always immediately after the class or
interface
> definition, so the "flavor" is declarative.

You're right; an assignment is just as procedural as a function call.=20
It was very personal aesthetic appreciation again; a function call looks

more like it's "doing" something, to me. But that is very arguable.


>> * Only classes (not instances) can declare that they implement an
>>   interface.
>
> Maybe there should be something like a "bindinstance" method as well.
> I'm sure there's a way to do it, but I don't consider it a very high
> priority.

Me neither. I think declaration would typically be done in a per class,
not per instance, basis. Still, due to the nature of python it is the=20
instance (the object, really) who implements (or fails to) the
interface.=20
So I would add it, for completeness and to reflect this, *if* it would=20
not require any specific syntax or additional complication. I think the=20
way __implements__ would be searched would be natural for python users,=20
since it is consistent with what is being done for __dict__, for
example.=20



> Good ideas.  Your method has a lot of advantages, but it would be hard
> (and messy) to make it do everything you can do with seprarate method
> calls, and one thing I'm *very* reluctant to do is have two ways of
> doing everything with only subtle or stylistic differences between
them.

I agree there should not be two ways. :>=20


> The best thing to do here may be to allow your style for simple cases
> (the 90% that would have required only a single "bind" call), but
> use the method-based syntax for anything more elaborate, like method
> renaming and unbinding subclasses.

Method renaming and unbinding subclasses are supported in the
alternative
and rather easy; I guess the weird formatting obscured this :).

Method renaming:
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D

 class I1(interface):
   def f(self): ""

 class I2(interface):
   def f(self, a, r, g, s): ""

 class SomeClass:

   def i_f(self): pass
   i1f.__implements__ =3D I1.f

   def g_f(self, a, r, g, s): pass
   i2f.__implements__ =3D I2.f

[Note: if 'implements' is introduced as a keyword, as in the PEP, we=20
could just as well declare

 def g_f(self, a, r, g, s) implements I2.f:
   ...
]


Unbinding subclasses:
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D

 class I1(interface): ...
 class I2interface): ...

 class Base:=20
   __implements__ =3D (I1, I2)

 class Sub(Base):
   # it would be more pretty to say (not I2,), but it would involve
   # some language hacking I guess, while (-I2,) only requires to
   # override __neg__ in the interface=20
   __implements__ =3D (-I2,)




>> [using proxies] should be optional too, but your syntax does this=20
>> nicely; you can call Foo() as an assertion of sorts and ignore the=20
>> result.
>
> I think it would be very misleading to require than an object support
a
> formal interface but then expect it to also support methods not
> specified in the interface.  If this is what you want, the right thing
> to do is derive a new interface from the old one that has the extra
> functionality you need.

As you pointed out, in the real world you won't be defining interfaces
for very simple things like "has a write() method". This means, you may=20
still want to use other forms of validation, or do without validation
at all (for aspects of the code which are still in experimental phase,=20
for example). You may even use the interface more informally as=20
a mere sanity check.=20

Anyway, method redirection almost enforces that the interface-specific=20
functionality of an object be accessed through a proxy. Doing so looks=20
like the right think to do anyway, so I don't consider this unfortunate=20
at all.

I like the proxy idea and the syntax you propose for it. The iterator=20
example was a nice read, anyway :).

Was this a typo?

  class Iterator(Iterable):

Meant class Iterator(interface): ?



As a conclussion, I am glad about the stress, in python, to generally
make things easier for you, rather than paternalistically try to keep=20
you from doing the evil. Since I came from Java, it took a bit using=20
to, but I have seen python scaling nicely to rather big sized projects=20
without this becoming a serious issue. I think there is a point to=20
standardize and automate type checking in python, but I believe it can
and should be done without betraying this philosophy.

Enough ranting for today! :)=20


Esteban.