[Python-Dev] PEP 246, redux

[Phillip J. Eby]

At 03:42 PM 1/10/05 +0100, Alex Martelli wrote:
>     The fourth case above is subtle.  A break of substitutability can
>     occur when a subclass changes a method's signature, or restricts
>     the domains accepted for a method's argument ("co-variance" on
>     arguments types), or extends the co-domain to include return
>     values which the base class may never produce ("contra-variance"
>     on return types).  While compliance based on class inheritance
>     _should_ be automatic, this proposal allows an object to signal
>     that it is not compliant with a base class protocol.

-1 if this introduces a performance penalty to a wide range of adaptations 
(i.e. those using abstract base classes), just to support people who want 
to create deliberate Liskov violations.  I personally don't think that we 
should pander to Liskov violators, especially since Guido seems to be 
saying that there will be some kind of interface objects available in 
future Pythons.


>     Just like any other special method in today's Python, __conform__
>     is meant to be taken from the object's class, not from the object
>     itself (for all objects, except instances of "classic classes" as
>     long as we must still support the latter).  This enables a
>     possible 'tp_conform' slot to be added to Python's type objects in
>     the future, if desired.

One note here: Zope and PEAK sometimes use interfaces that a function or 
module may implement.  PyProtocols' implementation does this by adding a 
__conform__ object to the function's dictionary so that the function can 
conform to a particular signature.  If and when __conform__ becomes 
tp_conform, this may not be necessary any more, at least for functions, 
because there will probably be some way for an interface to tell if the 
function at least conforms to the appropriate signature.  But for modules 
this will still be an issue.

I am not saying we shouldn't have a tp_conform; just suggesting that it may 
be appropriate for functions and modules (as well as classic classes) to 
have their tp_conform delegate back to self.__dict__['__conform__'] instead 
of a null implementation.



>     The object may return itself as the result of __conform__ to
>     indicate compliance.  Alternatively, the object also has the
>     option of returning a wrapper object compliant with the protocol.
>     If the object knows it is not compliant although it belongs to a
>     type which is a subclass of the protocol, then __conform__ should
>     raise a LiskovViolation exception (a subclass of AdaptationError).
>     Finally, if the object cannot determine its compliance, it should
>     return None to enable the remaining mechanisms.  If __conform__
>     raises any other exception, "adapt" just propagates it.
>
>     To enable the third case, when the protocol knows about the
>     object, the protocol must have an __adapt__() method.  This
>     optional method takes two arguments:
>
>     - `self', the protocol requested
>
>     - `obj', the object being adapted
>
>     If the protocol finds the object to be compliant, it can return
>     obj directly.  Alternatively, the method may return a wrapper
>     compliant with the protocol.  If the protocol knows the object is
>     not compliant although it belongs to a type which is a subclass of
>     the protocol, then __adapt__ should raise a LiskovViolation
>     exception (a subclass of AdaptationError).  Finally, when
>     compliance cannot be determined, this method should return None to
>     enable the remaining mechanisms.  If __adapt__ raises any other
>     exception, "adapt" just propagates it.
>     The fourth case, when the object's class is a sub-class of the
>     protocol, is handled by the built-in adapt() function.  Under
>     normal circumstances, if "isinstance(object, protocol)" then
>     adapt() returns the object directly.  However, if the object is
>     not substitutable, either the __conform__() or __adapt__()
>     methods, as above mentioned, may raise an LiskovViolation (a
>     subclass of AdaptationError) to prevent this default behavior.

I don't see the benefit of LiskovViolation, or of doing the exact type 
check vs. the loose check.  What is the use case for these?  Is it to allow 
subclasses to say, "Hey I'm not my superclass?"  It's also a bit confusing 
to say that if the routines "raise any other exceptions" they're 
propagated.  Are you saying that LiskovViolation is *not* propagated?



>     If none of the first four mechanisms worked, as a last-ditch
>     attempt, 'adapt' falls back to checking a registry of adapter
>     factories, indexed by the protocol and the type of `obj', to meet
>     the fifth case.  Adapter factories may be dynamically registered
>     and removed from that registry to provide "third party adaptation"
>     of objects and protocols that have no knowledge of each other, in
>     a way that is not invasive to either the object or the protocols.

This should either be fleshed out to a concrete proposal, or 
dropped.  There are many details that would need to be answered, such as 
whether "type" includes subtypes and whether it really means type or 
__class__.  (Note that isinstance() now uses __class__, allowing proxy 
objects to lie about their class; the adaptation system should support this 
too, and both the Zope and PyProtocols interface systems and PyProtocols' 
generic functions support it.)

One other issue: it's not possible to have standalone interoperable PEP 246 
implementations using a registry, unless there's a standardized place to 
put it, and a specification for how it gets there.  Otherwise, if someone 
is using both say Zope and PEAK in the same application, they would have to 
take care to register adaptations in both places.  This is actually a 
pretty minor issue since in practice both frameworks' interfaces handle 
adaptation, so there is no *need* for this extra registry in such cases.



>     Adaptation is NOT "casting".  When object X itself does not
>     conform to protocol Y, adapting X to Y means using some kind of
>     wrapper object Z, which holds a reference to X, and implements
>     whatever operation Y requires, mostly by delegating to X in
>     appropriate ways.  For example, if X is a string and Y is 'file',
>     the proper way to adapt X to Y is to make a StringIO(X), *NOT* to
>     call file(X) [which would try to open a file named by X].
>
>     Numeric types and protocols may need to be an exception to this
>     "adaptation is not casting" mantra, however.

The issue isn't that adaptation isn't casting; why would casting a string 
to a file mean that you should open that filename?  I don't think that 
"adaptation isn't casting" is enough to explain appropriate use of 
adaptation.  For example, I think it's quite valid to adapt a filename to a 
*factory* for opening files, or a string to a "file designator".  However, 
it doesn't make any sense (to me at least) to adapt from a file designator 
to a file, which IMO is the reason it's wrong to adapt from a string to a 
file in the way you suggest.  However, casting doesn't come into it 
anywhere that I can see.

If I were going to say anything about that case, I'd say that adaptation 
should not be "lossy"; adapting from a designator to a file loses 
information like what mode the file should be opened in.  (Similarly, I 
don't see adapting from float to int; if you want a cast to int, cast 
it.)  Or to put it another way, adaptability should imply substitutability: 
a string may be used as a filename, a filename may be used to designate a 
file.  But a filename cannot be used as a file; that makes no sense.


>Reference Implementation and Test Cases
>
>     The following reference implementation does not deal with classic
>     classes: it consider only new-style classes.  If classic classes
>     need to be supported, the additions should be pretty clear, though
>     a bit messy (x.__class__ vs type(x), getting boundmethods directly
>     from the object rather than from the type, and so on).

Please base a reference implementation off of either Zope or PyProtocols' 
field-tested implementations which deal correctly with __class__ vs. 
type(), and can detect whether they're calling a __conform__ or __adapt__ 
at the wrong metaclass level, etc.  Then, if there is a reasonable use case 
for LiskovViolation and the new type checking rules that justifies adding 
them, let's do so.



>     Transitivity of adaptation is in fact somewhat controversial, as
>     is the relationship (if any) between adaptation and inheritance.

The issue is simply this: what is substitutability?  If you say that 
interface B is substitutable for A, and C is substitutable for B, then C 
*must* be substitutable for A, or we have inadequately defined 
"substitutability".

If adaptation is intended to denote substitutability, then there can be 
absolutely no question that it is transitive, or else it is not possible to 
have any meaning for interface inheritance!

Thus, the controversies are: 1) whether adaptation should be required to 
indicate substitutability (and I think that your own presentation of the 
string->file example supports this), and 2) whether the adaptation system 
should automatically provide an A when provided with a C.  Existing 
implementations of interfaces for Python all do this where interface C is a 
subclass of A.  However, they differ as to whether *all* adaptation should 
indicate substitutability.  The Zope and Twisted designers believe that 
adaptation should not be required to imply substitutability, and that only 
interface and implementation inheritance imply 
substitutability.  (Although, as you point out, the latter is not always 
the case.)

PyProtocols OTOH believes that *all* adaptation must imply 
substitutability; non-substitutable adaptation or inheritance is a design 
error: "adaptation abuse", if you will.  So, in the PyProtocols view, it 
would never make sense to define an adaptation from float or decimal to 
integer that would permit loss of precision.  If you did define such an 
adaptation, it must refuse to adapt a float or decimal with a fractional 
part, since the number would no longer be substitutable if data loss occurred.

Of course, this is a separate issue from automatic transitive adaptation, 
in the sense that even if you agree that adaptation must imply 
substitutability, you can still disagree as to whether automatically 
locating a multi-step adaptation is desirable enough to be worth 
implementing.  However, if substitutability is guaranteed, then such 
multi-step adaptation cannot result in anything "controversial" occurring.


>     The latter would not be controversial if we knew that inheritance
>     always implies Liskov substitutability, which, unfortunately we
>     don't.  If some special form, such as the interfaces proposed in
>     [4], could indeed ensure Liskov substitutability, then for that
>     kind of inheritance, only, we could perhaps assert that if X
>     conforms to Y and Y inherits from Z then X conforms to Z... but
>     only if substitutability was taken in a very strong sense to
>     include semantics and pragmatics, which seems doubtful.

As a practical matter, all of the existing interface systems (Zope, 
PyProtocols, and even the defunct Twisted implementation) treat interface 
inheritance as guaranteeing substitutability for the base interface, and do 
so transitively.

However, it seems to me to be a common programming error among people new 
to interfaces to inherit from an interface when they intend to *require* 
the base interface's functionality, rather than *offer* the base 
interface's functionality.  It may be worthwhile to address this issue in 
the design of "standard" interfaces for Python.

This educational issue regarding substitutability is I believe inherent to 
the concept of interfaces, however, and does not go away simply by making 
non-inheritance adaptation non-transitive in the implementation.  It may, 
however, make it take longer for people to encounter the issue, thereby 
slowing their learning process.  ;)



>Backwards Compatibility
>
>     There should be no problem with backwards compatibility unless
>     someone had used the special names __conform__ or __adapt__ in
>     other ways, but this seems unlikely, and, in any case, user code
>     should never use special names for non-standard purposes.

Production implementations of the old version of PEP 246 exist, so the 
changes in semantics you've proposed may introduce backward compatibility 
issues.  More specifically, some field code may not work correctly with 
your proposed reference implementation, in the sense that code that worked 
with Zope or PyProtocols before, may not work with the reference 
implementation's adapt(), resulting in failure of adaptation where success 
occurred before, or in exceptions raised where no exception was raised before.