Yet Another PEP: Query Protocol Interface or __query__
Clark C. Evans
cce at clarkevans.com
Tue Mar 27 07:49:46 EST 2001
Below is an updated version of the proposal I've been
a champion for. It incorporates a good amount of feedback
that I've received. This proposal is starting to be come
more solid and the implementation is now hosted at
http://sourceforge.net/projects/adapt. Changes include:
* Checking component "can_wrap" and "isa" have been
removed as suggested by Alex Martelli
* Replaced __adapt__ with __conform__ and then __prodapt__
with __adapt__ as suggested by Magnus Lie Hetland
* A _check private function was added to demonstrate how
we work in conjunction with Paul Prescod's checking
proposal and Michel Pelletier's interface proposal.
* Borrowed more descriptive sections from Alex Martelli
for the abstract.
* Updated the title to reflect the removal of
checking from the goal and to reflect the bi-directional
lookup mechanism as proposed; is this a good name?
* Added a section about how this proposal relates to the
automagical proxy construction techniques posted by
Carlos Ribeiro.
Thank you once again for your kind feedback.
Best,
Clark
PEP: XXX
Title: Protocol Conformance and Object Adaptation
Version: $Revision$
Author: Clark Evans
Python-Version: 2.2
Status: Draft
Type: Standards Track
Created: 21-Mar-2001
Updated: 23-Mar-2001
Abstract
This proposal puts forth an extensible mechanism for the
adaptation of an object to a context where a specific type,
class, interface, or other protocol is expected.
This proposal provides a built-in "adapt" function that, for any
object X and protocol Y, can be used to ask the Python environment
for a version of X complaint with Y. Behind scenes the mechanism
asks the object X: "Are you now, or do you know how to wrap
yourself to provide, a supporter of protocol Y?". And, if this
request fails, the function then asks the protocol Y: "Does object
X support you, or do you know how to wrap it to obtain such a
supporter?" This duality is important, because protocols can be
developed after objects are, OR vice-versa, and this PEP lets
either case be supported non-invasively with regard to the
pre-existing component[s].
This proposal does not limit what a protocol is, what
compliance to the protocol means, nor what a wrapper
constitutes. This mechanism leverages existing
protocol categories such as the type system and class
hierarchy and can be expanded to support future protocol
categories such as the pending interface proposal [1]
and signature based type-checking system [2]
Motivation
Currently there is no standardized mechanism in Python for
asking if an object supports a particular protocol. Typically,
existence of particular methods, particularly those that are
built-in such as __getitem__, is used as an indicator of
support for a particular protocol. This technique works for
protocols blessed by GvR, such as the new enumerator proposal
identified by a new built-in __iter__. However, this technique
does not admit an infallible way to identify interfaces lacking
a unique, built-in signature method.
More so, there is no standardized way to obtain an adapter
for an object. Typically, with objects passed to a context
expecting a particular protocol, either the object knows about
the context and provides its own wrapper or the context knows
about the object and wraps it appropriately. The difficulty
with these approaches is that such adaptations are one-offs,
are not centralized in a single place of the users code, and
are not executed with a common technique, etc. This lack of
standardization increases code duplication with the same
adapter occurring in more than one place or it encourages
classes to be re-written instead of adapted. In either case,
maintainability suffers.
It would be very nice to have a standard function that can
be called upon to verify an objects compliance with a
particular protocol and provide for a wrapper if one is
readily available -- all without having to hunt through
a library's documentation for the appropriate incantation.
Requirements
When considering an objects compliance with a protocol,
there are several cases to be examined:
a) When the protocol is a type or class, and the object
has exactly that type or is a member of the class.
In this case compliance is automatic.
b) When the object knows about the protocol and either
considers itself compliant or knows how to wrap itself
appropriately.
c) When the protocol knows about the object and either
the object already complies or can be wrapped accordingly.
d) When the protocol is a class, and the object is a
member of a subclass. This is distinct from the first
case (a) above, since inheritance does not necessarily
imply substitutability and must be handled carefully.
e) When the context knows about the object and the
protocol and knows how to adapt the object so that
the required protocol is satisfied. This could use an
adapter registry or similar method.
For this proposal's requirements, the first case should be
come for free and the next three cases should be relatively
relatively easy to accomplish. This proposal does not address
the last case, however it provides a base mechanism upon which
such an approach could be developed. Further, with only minor
implementation changes, this proposal should be able to
incorporate a new interface type or type checking system.
The fourth case above is subtle. A lack of substitutability can
occur when a method restricts an argument's domain or raises an
exception which a base class does not or extends the co-domain to
include return values which the base class may never produce.
While compliance based on class inheritance should be automatic,
this proposal should allow an object to signal that it is not
compliant with a base class protocol.
Specification
This proposal introduces a new built-in function, adapt, which
is the basis for supporting these requirements.
The adapt function has three parameters: 1) the object to be
adapted ("obj"), 2) the protocol requested of the object
("protocol"), and 3) an optional object to return if the object
could not be adapted ("alternate").
A successful result of the adapt function returns either
the object passed ("obj") if the object is already compliant
with the protocol, or a secondary object ("wrapper"), which
provides a view of the object compliant with the protocol.
The definition of wrapper is explicitly vague and a wrapper
is allowed to be a full object with its own state if necessary.
A failure to adapt the object to the protocol will raise a
TypeError unless the alternate parameter is used, in this
case the alternate argument is returned.
To enable the first case listed in the requirements,
the adapt function first checks to see if the objects
type or the objects class are identical to the protocol.
If so, then the adapt function returns the object directly
without further ado.
To enable the second case, when the object knows about the
protocol, the object must have an __conform__ method. This
optional method takes two arguments, the object being conformed
("self") and the protocol requested ("protocol"). The object
may return itself through this method to indicate compliance.
Alternatively, the object also has the option of returning a
wrapper object compliant with the protocol. Finally, if the
object cannot determine its compliance, it should either return
None or raise a TypeError to enable the remaining mechanisms.
To enable the third case, when the protocol knows about the object,
the protocol must have a __adapt__ method. This optional method
takes two arguments, the protocol requested ("self") and the
object being adapted ("obj"). If the protocol finds the object
to be compliant, it can return obj directly. Alternatively, the
method may return a wrapper compliant with the protocol. Finally,
compliance cannot be determined, this method should either
return None or raise a TypeError so other mechanisms can be tried.
The fourth case, when the object's class is a sub-class of the
protocol, is handled by the built-in adapt function. Under normal
circumstances, if isinstance(object,protcol) then adapt returns
the object directly. However, if the object is not substitutable,
either the __conform__ or __adapt__ methods above may raise an
adaptForceFailException to prevent this default behavior.
Please note two important things. First, this proposal does not
preclude the addition of other protocols. Second, this proposal
does not preclude other possible cases where adapter pattern may
hold, such as the context knowing the object and the protocol
(the last case in the requirements). In fact, this proposal opens
the gate for these other mechanisms to be added.
Reference Implementation and Test Cases
-----------------------------------------------------------------
adapt.py
-----------------------------------------------------------------
import types
adaptRaiseTypeException = "(raise a type exception on failure)"
adaptForceFailException = "(forced failure of adapt)"
# look to see if the object passes other protocols
def _check(obj,protocol,default):
return default
def adapt(obj, protocol, alternate = adaptRaiseTypeException):
# first check to see if object has the exact protocol
if type(obj) is types.InstanceType and \
obj.__class__ is protocol: return obj
if type(obj) is protocol: return obj
# next check other protocols for exact conformance
# before calling __conform__ or __adapt__
if _check(obj,protocol,0):
return obj
# procedure to execute on success
def succeed(obj,retval,protocol,alternate):
if _check(retval,protocol,1):
return retval
else:
return fail(obj,alternate)
# procedure to execute on failure
def fail(obj,protocol,alternate):
if alternate is adaptRaiseTypeException:
raise TypeError("%s cannot be adapted to %s" \
% (obj,protocol))
return alternate
# try to use the object's adapting mechanism
conform = getattr(obj, '__conform__',None)
if conform:
try:
retval = conform(protocol)
if retval:
return succeed(obj,retval,protocol,alternate)
except adaptForceFailException:
return fail(obj,protocol,alternate)
except TypeError: pass
# try to use the protocol's adapting mechanism
adapt = getattr(protocol, '__adapt__',None)
if adapt:
try:
retval = adapt(obj)
if retval:
return succeed(obj,retval,protocol,alternate)
except adaptForceFailException:
return fail(obj,protocol,alternate)
except TypeError: pass
# check to see if the object is an instance
try:
if isinstance(obj,protocol):
return obj
except TypeError: pass
# no-adaptation-possible case
return fail(obj,protocol,alternate)
-----------------------------------------------------------------
test.py
-----------------------------------------------------------------
import types
from adapt import adaptForceFailException
from adapt import adapt
class KnightsWhoSayNi: pass
class Eggs: # an unrelated class/interface
def eggs(self): print "eggs!"
word = "Nee-womm"
class Ham: # used as an interface, no inhertance
def ham(self): pass
word = "Ping"
class Spam: # a base class, inheritance used
def spam(self): print "spam!"
class EggsSpamAndHam (Spam,KnightsWhoSayNi):
def ham(self): print "ham!"
def __conform__(self,protocol):
if protocol is Ham:
# implements Ham's ham, but does not have a word
return self
if protocol is KnightsWhoSayNi:
# we are no longer the Knights who say Ni!
raise adaptForceFailException
if protocol is Eggs:
# Knows how to create the eggs!
return Eggs()
class SacredWord:
class HasSecredWord:
def __call__(self, obj):
if getattr(obj,'word',None): return obj
__adapt__= HasSecredWord()
class Bing (Ham):
def __conform__(self,protcol):
raise adaptForceFailException
def test():
x = EggsSpamAndHam()
adapt(x,Spam).spam()
adapt(x,Eggs).eggs()
adapt(x,Ham).ham()
adapt(x,EggsSpamAndHam).ham()
print adapt(Eggs(),SacredWord).word
print adapt(Ham(),SacredWord).word
pass
if adapt(x,KnightsWhoSayNi,None): raise "IckyIcky"
if not adapt(x,Spam,None): raise "Spam"
if not adapt(x,Eggs,None): raise "Eggs"
if not adapt(x,Ham,None): raise "Ham"
if not adapt(x,EggsSpamAndHam,None): raise "EggsAndSpam"
if adapt(x,KnightsWhoSayNi,None): raise "NightsWhoSayNi"
if adapt(x,SacredWord,None): raise "SacredWord"
try:
adapt(x,SacredWord)
except TypeError: pass
else: raise "SacredWord"
try:
adapt(x,KnightsWhoSayNi)
except TypeError: print "Ekky-ekky-ekky-ekky-z'Bang, " \
+ "zoom-Boing, z'nourrrwringmm"
else: raise "NightsWhoSayNi"
pass
b = Bing()
if not adapt(b,Bing,None): raise "Not a Bing"
if adapt(b,Ham,None): raise "Not a Ham!"
if adapt(1,types.FloatType,None): raise "Not a float!"
if adapt(b,types.FloatType,None): raise "Not a float!"
if adapt(1,Ham,None): raise "Not a Ham!"
if not adapt(1,types.IntType,None): raise "Is an Int!"
-----------------------------------------------------------------
Expected Output
-----------------------------------------------------------------
>>> import test
>>> test.test()
spam!
eggs!
ham!
ham!
Nee-womm
Ping
Ekky-ekky-ekky-ekky-z'Bang, zoom-Boing, z'nourrrwringmm
>>>
Relationship To Paul Prescod and Tim Hochberg's Type Assertion method
Paul and Tim had proposed an type checking mechanism, where the
Interface is passed an object to verify. The example syntax Paul
put forth recently [2] was:
interface Interface
def __check__(self,obj)
For discussion purposes, here would be a protocol with __check__:
class Interface:
class Checker:
def __call__(self, obj): pass #check the object
__check__= Checker()
The built-in adapt function could be augmented to use this
checking mechanism updating the _check method as follows:
# look to see if the object passes other protocols
def _check(obj,protocol,default):
check = getattr(protocol, '__check__',None)
if check:
try:
if check(obj): return 1
except TypeError: pass
return 0
else:
return default
In short, the work put forth by Paul and company is great, and
I don't see any problems why these two proposals couldn't work
together in harmony, if not be completely complementary.
Relationship to Python Interfaces [1] by Michel Pelletier
The relationship to this proposal to Michel's proposal could
also be complementary. Following is how the _check method
would be updated for this mechanism:
# look to see if the object passes other protocols
def _check(obj,protocol,default):
if type(protocol) is types.InterfaceType:
return implements(obj,protcol)
return default
Relationship to Carlos Ribeiro's proxy technique [7] and [8]
Carlos presented a technique where this method could return
a proxy instead of self or a wrapper. The advantage of this
approach is that the internal details of the object are
protected. This is very neat. No changes are necessary to
this proposal to support this usage as a standardized mechanism
to obtain named proxies.
Relationship To Microsoft's Query Interface
Although this proposal may sounds similar to Microsoft's
QueryInterface, it differs by a number of aspects. First,
it is bi-directional allowing the interface to be queried
as well giving more dynamic abilities (more pythonic). Second,
there is not a special "IUnknown" interface which can be used
for object identity, although this could be proposed as one
of those "special" blessed interface protocol identifiers.
Third, with QueryInterface, once an object supports a particular
interface it must always there after support this interface;
this proposal makes no such guarantee, although this may be
added at a later time. Fourth, implementations of Microsoft's
QueryInterface must support a kind of equivalence relation.
By reflexive they mean the querying an interface for itself
must always succeed. By symmetrical they mean that if one
can successfully query an interface IA for a second interface
IB, then one must also be able to successfully query the
interface IB for IA. And finally, by transitive they mean if
one can successfully query IA for IB and one can successfully
query IB for IC, then one must be able to successfully query
IA for IC. Ability to support this type of equivalence relation
should be encouraged, but may not be possible. Further research
on this topic (by someone familiar with Microsoft COM) would be
helpful in further determining how compatible this proposal is.
Question and Answer
Q: What benefit does this provide?
The typical python programmer is an integrator, someone
who is connecting components from various vendors. Often
times the interfaces between these components require
an intermediate adapter. Usually the burden falls upon
the programmer to study the interface exposed by one
component and required by another, determine if they
are directly compatible, or develop an adapter. Sometimes
a vendor may even include the appropriate adapter, but
then searching for the adapter and figuring out how to
deploy the adapter takes time.
This technique enables vendors to work with each other
directly by implementing __conform__ or __adapt__ as
necessary. This frees the integrator from making their
own adapters. In essence, this allows the components
to have a simple dialogue among themselves. The integrator
simply connects one component to another, and if the types
don't automatically match an adapting mechanism is built-in.
For example, consider SAX1 and SAX2 interfaces, there is
an adapter required to switch between them. Normally the
programmer must be aware of this; however, with this
adaptation framework this is no longer the case.
Q: Why does this have to be built-in, can't it be standalone?
Yes, it does work standalone. However, if it is built-in,
it has a greater chance of usage. The value of this proposal
is primarily in standardization. Furthermore:
0. The mechanism is by its very nature a singleton.
1. If used frequently, it will be much faster as a built-in
2. It is extensible and unassuming.
3. A whole-program optimizing compiler could optimize it out
in particular cases (ok, this one is far fetched)
Q: Why the verbs __conform__ and __adapt__?
conform, verb intransitive
1. To correspond in form or character; be similar.
2. To act or be in accord or agreement; comply.
3. To act in accordance with current customs or modes.
adapt, verb transitive
1. To make suitable to or fit for a specific use or
situation.
Source: The American Heritage Dictionary of the English
Language, Third Edition
Backwards Compatibility
There should be no problem with backwards compatibility unless
someone had used __conform__ or __adapt__, but this seems
unlikely. Indeed this proposal, save an built-in adapt()
function, could be tested without changes to the interpreter.
Credits
This proposal was created in large part by the feedback
of the talented individuals on both the main mailing list
and also the type signature list. Specific contributors
include (sorry if I missed someone).
This proposal is based largely off the suggestions from
Alex Martelli and Paul Prescod with significant feedback
from Robin Thomas and borrowing ideas from Marcin 'Qrczak'
Kowalczyk and Carlos Ribeiro. Other contributors (via comments)
include:
Michel Pelletier, Jeremy Hylton, Aahz Maruch,
Fredrik Lundh, Rainer Deyke, Timothy Delaney,
and Huaiyu Zhu
Copyright
This document has been placed in the public domain.
References and Footnotes
[1] http://python.sourceforge.net/peps/pep-0245.html
[2] http://mail.python.org/pipermail/types-sig/2001-March/001223.html
[3] http://www.zope.org/Members/michel/types-sig/TreasureTrove
[4] http://mail.python.org/pipermail/types-sig/2001-March/001105.html
[5] http://mail.python.org/pipermail/types-sig/2001-March/001206.html
[6] http://mail.python.org/pipermail/types-sig/2001-March/001223.html
[7]
http://mail.python.org/pipermail/python-list/2001-March/035136.html
[8]
http://mail.python.org/pipermail/python-list/2001-March/035197.html
More information about the Python-list
mailing list