[Python-Dev] Updating PEP 246 for type/class unification, 2.2+, etc.

[Phillip J. Eby]

I'd like to propose some revisions to PEP 246 based on experience trying to 
implement a prototype of it for use in PEAK and Zope (perhaps Twisted as 
well).  The issues I see are as follows:

1. PEP 246 allows TypeError in __conform__ and __adapt__ methods to pass 
silently.  (After considerable work and thought, I was able to 
reverse-engineer *why*, but that rationale should at least be explicitly 
documented in the PEP, even if the limitation is unavoidable.)

2. The reference implementation in the PEP has fancy extra features that 
are not specified by the main body of the PEP, and in some cases raise more 
questions than they answer about what a valid PEP 246 implementation should 
do.  (adaptRaiseTypeException, adaptForceFailException, _check, etc.)

3. The PEP 246 examples do not illustrate Python 2.2+ idioms for creating 
usable __conform__ and __adapt__ methods.  For example, a class 
instance  with a __call__ method gets stuck in another class in order to 
(presumably) work around the absence of staticmethod or classmethod in 
Python prior to version 2.2.  The reference implementation also uses string 
exceptions, which were a no-no even before version 2.2.

4. PEP 246 does not cover implementation issues for developers in the cases 
where 'obj' is a class or 'protocol' is an instance.  The former is 
particularly important in the context of adapting metaclass instances, and 
the latter is relevant for using Zope 'Interface' objects (for example) as 
protocols.

None of these issues are unresolvable; in fact I have proposals to address 
them all.  If the PEP authors agree with my assessments, perhaps they will 
undertake to update the PEP.  My goal is not to get a PEP 246 'adapt()' 
blessed for the Python core or distro in the immediate future, but rather 
to have a usable reference standard for framework developers to build 
implementations on.  Even more important...  I would like framework users 
to be able to write __conform__ and __adapt__ methods that will be in 
principle usable by any framework that uses PEP 246 as a standard for 
adaptation.  In this sense, we may view the role of PEP 246 as being 
similar to the Python DBAPI.

So, without further ado, my proposals for revisions to PEP 246 are as follows:

Issue #1: My reverse-engineering leads me to the conclusion that PEP 246 
specifies that TypeError be ignored because of issue #4 above: using a 
class as 'obj' or an instance for 'protocol' may lead to a TypeError caused 
by using a class method as an instance method or vice versa.  While the 
creator of the objects being supplied to 'adapt()' can work around these 
issues with descriptors, the casual user should not be expected to.  Thus, 
such TypeErrors should be ignored.

To resolve this dilemna, I propose that 'adapt()' use the following 
pseudocode to verify whether a TypeError has arisen from invocation of a 
method, or the execution of a method:

         try:
             # note: real implementation needs to catch AttributeError!
             result = obj.__conform__(protocol)
             if result is not None:
                 return result

         except TypeError:
             if sys.exc_info()[2].tb_frame is not sys._getframe():
                 raise

In other words, if the exception was raised in the calling frame, it is 
assumed to be an invocation error rather than an execution error, and can 
thus be safely ignored.  The only "exception" to this pattern is if the 
targeted method is written in C and thus does not create a separate frame 
for execution.  (Note that C code generated by Pyrex creates dummy 
execution frames before returning an exception to Python, so this is only 
an issue for hand-written C code.)  The worst case scenario here is that 
authors of '__conform__' and '__adapt__' methods written in C must 1) 
guarantee that TypeError will not be raised, 2) accept silent loss of 
internal TypeErrors, or 3) write code to create a dummy frame when raising 
an error.

As far as Jython impact, the mechanism by which TypeErrors are raised is 
different, so I do not know if it is possible for the Java or Python levels 
to cleanly make this differentiation.  If Jython simulates Python frames 
and tracebacks, including only Python-level frames, then this would work 
more or less directly.  I confess I do not understand enough about Jython's 
implementation at present to know how practical it is under Jython.  An 
alternative might be to recognize the text of the Python exception values 
for unbound methods, missing arguments, etc., applying to the method being 
called.  This might actually be more complex to implement correctly, though.


Issue #2: I propose that the PEP 246 reference implementation be pared down 
to remove extraneous features.  Specifically, I believe that the signature 
of adapt should be:

_marker = object()

def adapt(obj, protocol, default=_marker):

     # ... attempt to return adapted result

     if default is _marker:
         raise NotImplementedError(...)

'adaptForceFailException' looks to me like a YAGNI, since an object 
shouldn't veto its being used for a protocol, if the protocol knows how to 
adapt it.  And the protocol doesn't need to force failure, it can return 
failure.

Rather than raising a TypeError for adaptation failure, and thus "raising" 
even further confusion regarding the proper handling of TypeError.

Finally, the '_check()' function should be dropped.  Its presence simply 
makes it harder to evaluate or consider PEP 246 for inclusion in Python or 
a framework, because it is left unspecified what '_check()' should do.  We 
are given many examples of what it *could* do, but not what it *should* 
do.  In any event, I think it's a YAGNI because if the object claims it can 
conform or the protocol claims it can adapt, then what business is it of 
'adapt()' to question the consent of the objects involved?


Issue #3: Examples should use 'classmethod' for '__adapt__' rather than 
simulated or real 'staticmethod', and include the case where a subclass 
delegates to a superclass '__adapt__' method.  And string exceptions are 
right out.


Issue #4: Illustrate the issues that arise for adapting classes or 
metaclass instances, and using instances rather than types as 
protocols.  Ideally, examples of descriptors that work around the issues 
should be included.  (And as soon as I've figured out how to write them, 
I'll be happy to supply source!)


Thoughts, anyone?