[Types-sig] QueryProtocol

Clark C. Evans cce@clarkevans.com
Sat, 24 Mar 2001 22:25:30 -0500 (EST)

Below is an updated version of the proposal Ive been 
a champion for.  It incorporates a great deal of feedback
that Ive received.  Changes include:

   * By default, adapt now throws an error again, however
     an alternate object can be provided upon failure much
     like getattr.

   * Now __adapt__ can either return None or throw a TypeError
     with the same effect.

   * This now includes reverse adaptation of the object
     by the protocol via __prodapt__ (better name?), which
     is very similar to Paul Prescods __check__

   * The text has been tightened (hopefully eliminating 
     much superfluous material)

   * The sample implementation code has been cleaned up
     and tested in more detail.

   * Ive received much comment to remove the type-checking
     stuff.  I have a question:  Is type-checking a component
     of adapting?  

I would very much appreciate any comments and feedback.

Especially if the intent/value of this proposal is not
clear as I seem to be having a difficult time with the
expression of this.

Further, I still do not have a PEP number.  Either I have 
not asked correctly, am being stupid, have pissed someone 
off, or am proposing something that has already been 
rejected.  In any case, what do I do?

Kind regards,


P.S.  Id like to thank Alex Martelli for his most helpful
      comments and hope that I have dutifully incorporated
      his suggestions appropriately. 

Title: Protocol Checking and Adaptation
Version: $Revision$
Author: Clark Evans
Python-Version: 2.2
Status: Draft
Type: Standards Track
Created: 21-Mar-2001
Updated: 23-Mar-2001


    This proposal puts forth a light-weight explicit mechanism 
    for the adaptation (including verification) of an object to 
    a context where a specific type, class, interface, or other
    protocol is expected.  This proposal can leverage existing
    protocols such as the type system and class hierarchy and is
    orthogonal, if not complementary to the pending interface
    mechanism [1] and signature based type-checking system [2]

    This proposal allows a standard and extensible method to
    ask two types of questions:  Is a particular object
    compliant with a given protocol? (b) And if not, can the
    object be wrapped so that it is complaint?  This proposal
    does not limit what a protocol is, what compliance to the 
    protocol means, nor what a wrapper constitutes.


    Currently there is no standardized mechanism in Python for
    asking if an object supports a particular protocol. Typically,
    existence of particular methods, particularly those that are
    built-in such as __getitem__, is used as an indicator of
    support for a particular protocol.  This technique works for
    protocols blessed by GvR, such as the new enumerator proposal
    identified by a new built-in __iter__.  However, this technique
    does not admit an infallible way to identify interfaces lacking
    a unique, built-in signature method.

    More so, there is no standardized way to obtain an adapter
    for an object.  Typically, with objects passed to a context
    expecting a particular protocol, either the object knows about
    the context and provides its own wrapper or the context knows
    about the object and wraps it appropriately.  The difficulty 
    with these approaches is that such adaptations are one-offs,
    are not centralized in a single place of the users code, and
    are not executed with a common technique, etc.  This lack of
    standardization increases code duplication with the same
    adapter occurring in more than one place or it encourages
    classes to be re-written instead of adapted.  In either case,
    maintainability suffers.

    It would be very nice to have a standard function that can
    be called upon to verify an objects compliance with a 
    particular protocol and provide for a wrapper if one is
    readily available -- all without having to hunt through 
    a librarys documentation for the appropriate incantation.


    When considering an objects compliance with a protocol, 
    there are several cases to be examined:

     a) When the protocol is a type or class, and the object
        has exactly that type or is a member of the class.
        In this case compliance is automatic.

     b) When the object knows about the protocol and either
        considers itself compliant or knows how to wrap itself

     c) When the protocol knows about the object and either
        it complies or can be wrapped accordingly.

     d) When the protocol is a class, and the object is a
        member of a subclass.  This is distinct from the first
        case (a) above, since inheritance does not necessarily
        imply substitutability and must be handled carefully.

     e) When the context knows about the object and the
        protocol and knows how to adapt the object so that
        the required protocol is satisfied.  This could use an
        adapter registry or similar method.

    For this proposal's requirements, the first case should be
    come for free and the next three cases should be relatively
    relatively easy to accomplish.  This proposal does not address 
    the last case, however it provides a base mechanism upon which 
    such an approach could be developed.  Further, with only minor
    implementation changes, this proposal should be able to 
    incorporate a new interface type or type checking system.

    The fourth case above is subtle. A lack of substitutability can
    occur when a method restricts an argument's domain or raises an 
    exception which a base class does not or extends the co-domain to
    include return values which the base class may never produce.  
    While compliance based on class inheritance should be automatic, 
    this proposal should allow an object to signal that it is not 
    compliant with a base class protocol.


    This proposal introduces a new built-in function, adapt, which
    is the basis for supporting these requirements. 

    The adapt function has four parameters: 1) the object to be
    adapted ("obj"), 2) the protocol requested of the object
    ("protocol"), 3) an optional object to return if the object 
    could not be adapted ("alternate"), and 4) an optional flag 
    which can be used to prevent a wrapper from being generated
    ("can_wrap") if provided.

    A successful result of the adapt function returns either
    the object passed ("obj") if the object is already compliant 
    with the protocol, or a secondary object ("wrapper"), which
    provides a view of the object compliant with the protocol.  
    The definition of wrapper is explicitly vague and a wrapper
    is allowed to be a full object with its own state if necessary.
    A failure to adapt the object to the protocol will raise a
    TypeError unless the alternate parameter is used, in this
    case the alternate argument is returned.

    To enable the first case listed in the requirements, 
    the adapt function first checks to see if the objects 
    type or the objects class are identical to the protocol.
    If so, then the adapt function returns the object directly
    without further ado.

    To enable the second case, when the object knows about the
    protocol, the object must have an __adapt__ method.  This 
    optional method takes three arguments, the object being adapted
    ("self"), the protocol requested ("protocol"), and a flag 
    ("can_wrap").  The object may return itself through this method 
    to indicate compliance.  Alternatively, if the can_wrap flag is
    true the object also has the option of returning a wrapper object
    compliant with the protocol.  Finally, if the object cannot 
    determine its compliance, it should either return None or 
    raise a TypeError to allow other mechanisms to be checked.

    To enable the third case, when the protocol knows about the object,
    the protocol must have a __prodapt__ method.  This optional method
    takes three arguments, the protocol requested ("self"), the
    object being adapted ("obj"), and a flag ("can_wrap").  If the 
    protocol finds the object to be compliant, it can return obj
    directly.  Alternatively, if the can_wrap flag is true, the 
    method may return a wrapper compliant with the protocol.  Finally,
    compliance cannot be determined, this method should either 
    return None or raise a TypeError so other mechanisms can be tried.

    The fourth case, when the objects class is a sub-class of the 
    protocol, is handled by the built-in adapt function.  Under normal
    circumstances, if isinstance(object,protcol) then adapt returns
    the object directly.  However, if the object is not substitutable,
    either the __adapt__ or __prodapt__ methods above may raise an
    adaptForceFailException to prevent this default behavior.

    This proposal could introduce one more built-in function, "isa",
    perhaps implemented as an operator.  If this is deemed useful, the
    function would take two arguments, the first is the object to be
    checked ("obj"), and the second is the protocol to check the
    object against ("protocol").  The return value of the operator
    will be either be the obj or None.  This function would be 
    implemented by calling the adapt function with the can_wrap being
    false and an alternate argument of None.

    Please note two important things.  First, this proposal does not
    preclude the addition of other protocols.  Second, this proposal
    does not preclude other possible cases where adapter pattern may
    hold, such as the context knowing the object and the protocol 
    (the last case in the requirements).  In fact, this proposal opens
    the gate for these other mechanisms to be added; while keeping the
    change in manageable chunks.

Reference Implementation and Example Usage


    import types

    adaptRaiseTypeException = "(raise a type exception on failure)"
    adaptForceFailException = "(forced failure of adapt)"

    def adapt(obj, protocol, alternate = \
                   adaptRaiseTypeException, can_wrap = 1):

        # first check to see if object has the exact protocol
        if type(obj) is types.InstanceType and \
           obj.__class__ is protocol: return obj
        if type(obj) is protocol: return obj

        # procedure to execute on success
        def succeed(retval,obj,protocol,can_wrap):
            if can_wrap: return retval
            return obj

        # procedure to execute on failure
        def fail(obj,protocol,alternate):
            if alternate is adaptRaiseTypeException:
                raise TypeError("%s cannot be adapted to %s" \
                                 % (obj,protocol))
            return alternate

        # try to use the object's adapting mechanism
        adapt = getattr(obj, '__adapt__',None)
        if adapt:
                retval = adapt(protocol,can_wrap)
                if retval: return succeed(retval,obj,protocol,can_wrap)
            except adaptForceFailException:
                return fail(obj,protocol,alternate)
            except TypeError: pass

        # try to use the protocol's adapting mechanism
        adapt = getattr(protocol, '__prodapt__',None)
        if adapt:
                retval = adapt(obj,can_wrap)
                if retval: return succeed(retval,obj,protocol,can_wrap)
            except adaptForceFailException:
                return fail(obj,protocol,alternate)
            except TypeError: pass

        # check to see if the object is an instance
            if isinstance(obj,protocol):
                return obj
        except TypeError: pass

        # no-adaptation-possible case
        return fail(obj,protocol,alternate)

    # imagine binary operator syntax
    def isa(obj,protocol):
        return adapt(obj,protocol,None,0)


    import adapter
    import types
    from adapter import adapt
    from adapter import isa
    from adapter import adaptForceFailException

    class KnightsWhoSayNi: pass

    class Eggs:  # an unrelated class/interface
        def eggs(self): print "eggs!"
        word = "Nee-womm"

    class Ham:  # used as an interface, no inhertance
        def ham(self): pass
        word = "Ping"

    class Spam: # a base class, inheritance used
        def spam(self): print "spam!"

    class EggsSpamAndHam (Spam,KnightsWhoSayNi):
        def ham(self): print "ham!"
        def __adapt__(self,protocol,can_wrap):
            if protocol is Ham:
                # implements Ham's ham, but does not have a word
                return self
            if protocol is KnightsWhoSayNi:
                # we are no longer the Knights who say Ni!
                raise adaptForceFailException
            if protocol is Eggs and can_wrap:
                # Knows how to create the eggs!
                return Eggs()
            raise TypeError("haha")

    class Bing (Ham):
        def __adapt__(self,protcol,can_wrap):
            raise adaptForceFailException

        # in this case, it is the protcol which determines
        # if the class is adapted
    class SacredWord:
        class HasSecredWord:
            def __call__(self, obj, can_wrap):
                if getattr(obj,'word',None): return obj
        __prodapt__= HasSecredWord()

    def test():
        x = EggsSpamAndHam()
        print adapt(Eggs(),SacredWord).word
        print adapt(Ham(),SacredWord).word
        if adapt(x,KnightsWhoSayNi,None): raise "IckyIcky"
        if not isa(x,Spam): raise "Spam"
        if     isa(x,Eggs): raise "Eggs"
        if not isa(x,Ham): raise "Ham"
        if not isa(x,EggsSpamAndHam): raise "EggsAndSpam"
        if     isa(x,KnightsWhoSayNi): raise "NightsWhoSayNi"
        if     isa(x,SacredWord): raise "SacredWord"
        except TypeError: pass
        else: raise "SacredWord"
        except TypeError: print "Ekky-ekky-ekky-ekky-z'Bang, " \
                                + "zoom-Boing, z'nourrrwringmm"
        else: raise "NightsWhoSayNi"
        b = Bing()
        if not isa(b,Bing): raise "Not a Bing"
        if isa(b,Ham): raise "Not a Ham!"
        if isa(1,types.FloatType): raise "Not a float!"
        if isa(b,types.FloatType): raise "Not a float!"
        if isa(1,Ham):             raise "Not a Ham!"
        if not isa(1,types.IntType): raise "Is an Int!"

    Example Run
    >>> import test
    >>> test.test()
    Ekky-ekky-ekky-ekky-z'Bang, zoom-Boing, z'nourrrwringmm

Relationship To Paul Prescod and Tim Hochbergs Type Assertion method

    Paul and Tim had proposed an type checking mechanism, where the
    Interface is passed an object to verify.  The example syntax Paul
    put forth recently [2] was:

        interface Interface
            def __check__(self,obj)

    This could be re-written to avoid the "interface" keyword as:

        class Interface:
            class Checker:
                def __call__(self, obj): pass  #check the object
            __check__= Checker()

    As I understand it, __prodapt__ is essentially __check__, only 
    that the function is also given the ability to substitute a 
    wrapper for the object if can_wrap is set.  Further, __prodapt__
    returns an object instead of true/false, however, this seems
    a trivial difference.  Indeed, a bulk of the idea for this 
    proposal was built directly from Paul and Tims work.

    In short, the work put forth by Paul and company is great, and
    I dont see any problems why these two proposals couldnt work
    together in harmony, if not be completely complementary.

Relationship to Python Interfaces [1] by Michel Pelletier

    The relationship to this proposal to Michels proposal could
    also be complementary.  If approved, the first two paragraphs
    of the built-in adapt function could be changed as follows:

        # first check to see if object has the exact protocol
    >   if type(protocol) is types.InterfaceType and \
    >       instance(obj,protocol): return obj
        if type(obj) is types.InstanceType and \
           obj.__class__ is protocol: return obj
        if type(obj) is protocol: return obj

        # procedure to execute on success
        def succeed(retval,obj,protocol,can_wrap):
    >       if type(protocol) is types.InterfaceType:
    >           if not instance(retval,protocol):
    >               raise "Bad __adapt__ or __prodapt__!"
            if can_wrap: return retval
            return obj

Relationships To Microsofts Query Interface:

    Although this proposal may sounds similar to Microsofts
    QueryInterface, it differs by a number of aspects.  First,
    there is not a special "IUnknown" interface which can be used
    for object identity, although this could be proposed as one
    of those "special" blessed interface protocol identifiers.
    Second, with QueryInterface, once an object supports a particular
    interface it must always there after support this interface;
    this proposal makes no such guarantee, although this may be
    added at a later time. Third, implementations of Microsofts
    QueryInterface must support a kind of equivalence relation.
    By reflexive they mean the querying an interface for itself
    must always succeed.  By symmetrical they mean that if one
    can successfully query an interface IA for a second interface
    IB, then one must also be able to successfully query the
    interface IB for IA.  And finally, by transitive they mean if
    one can successfully query IA for IB and one can successfully
    query IB for IC, then one must be able to successfully query
    IA for IC.  Ability to support this type of equivalence relation
    should be encouraged, but may not be possible.  Further research
    on this topic (by someone familiar with Microsoft COM) would be
    helpful in further determining how compatible this proposal is.

Backwards Compatibility

    There should be no problem with backwards compatibility unless
    someone had used __adapt__ or __prodapt__, but this seems 
    unlikely.  Indeed this proposal, save an built-in adapt() 
    function, could be tested without changes to the interpreter.

Questions and Answers

    Q:  Why was the name changed from __query__ to __adapt__ ?

    A:  It was clear that significant QueryInterface assumptions were
        being laid upon the proposal, when the intent was more of an
        adapter.  Of course, if an object does not need to be adapted
        then it can be used directly and this is the basic premise.

    Q:  Why is the checking mixed with the adapter mechanism?

    A:  Good question.  They could be separated, however, there
        is significant overlap, if you consider the checking
        protocol as returning a compliant object (self) or
        not a compliant object (None).  In this way, adapting
        becomes a special case of checking, via the can_wrap.

        This could be separated out, but the two concepts are 
        very related so much duplicate work would be done, and 
        the overall mechanism would feel quite a bit less unified.

    Q:  I like/dont like the isa function/operator.

    A:  It is separable from this proposal, it just seems like
        a handy short-hand for checking rather than adapting.

    Q:  The name __prodapt__ sucks.

    A:  Suggest a better alternative; I suppose __check__ could
        be used for the protocol side, however I dont want to
        trample on Paul Prescods proposal.


    This proposal was created in large part by the feedback
    of the talented individuals on both the main mailing list
    and also the type signature list.  Specific contributors
    include (sorry if I missed someone).

    This proposal is based largely off the suggestions from
    Alex Martelli and Paul Prescod with significant feedback
    from Robin Thomas and borrowing ideas from Marcin 'Qrczak'
    Kowalczyk.  Other contributors (via comments) include:

        Michel Pelletier, Jeremy Hylton, Carlos Ribeiro,
        Aahz Maruch, Fredrik Lundh, Rainer Deyke,
        Timothy Delaney, and Huaiyu Zhu


    This document has been placed in the public domain.

References and Footnotes

    [1] http://python.sourceforge.net/peps/pep-0245.html
    [2] http://mail.python.org/pipermail/types-sig/2001-March/001223.html
    [3] http://www.zope.org/Members/michel/types-sig/TreasureTrove
    [4] http://mail.python.org/pipermail/types-sig/2001-March/001105.html
    [5] http://mail.python.org/pipermail/types-sig/2001-March/001206.html
    [6] http://mail.python.org/pipermail/types-sig/2001-March/001223.html