[Python-3000] Draft pre-PEP: function annotations
Phillip J. Eby
pje at telecommunity.com
Sat Aug 12 18:12:26 CEST 2006
At 05:58 PM 8/12/2006 +1000, Nick Coghlan wrote:
>Phillip J. Eby wrote:
>>At 03:39 PM 8/12/2006 -0700, Talin <talin at acm.org> wrote:
>>>So programmer C, who wants to incorporate both A and B's work into his
>>>program, has a dilemma - each has a sharing mechanism, but the sharing
>>>mechanisms are different and incompatible. So he is unable to apply both
>>>A-type and B-type metadata to any given signature.
>>Not at all. A and B need only use overloadable functions,
>Stop right there. "A and B need only use overloadable functions"? That
>sounds an awful lot like placing a constraint on the way annotation
>libraries are implemented in order to facilitate a single program using
>multiple annotation libraries - which is exactly what Talin is saying is
You could perhaps look at it that way. However, I'm simply using
overloadable functions as a trivial example of how easy this is to handle
without specifying a single mechanism. There are numerous overloaded
function implementations available, for example, including ad-hoc
registry-based ones (like the ones used by pickle) and other mechanisms
besides overloaded functions that do the same thing. PEP 246 adaptation,
for example, as used by Twisted and Zope.
My point is that:
1. trivial standard extension mechanisms (that are already in use in
today's Python) allow libraries to offer compatibility between approaches,
without choosing any blessed implementation or even approach to combination
2. there is no need to define a fixed semantic framework for
annotations. Guidelines for combinability (e.g. a standard interpretation
for tuples or lists) might be a good idea, but it isn't *necessary* to
mandate a single interpretation.
>(and using overloaded functions for this strikes me as hitting a very
>small nail with a very large hammer).
Remember: Python is built from the ground up on overloaded
functions. len(), iter(), str(), repr(), hash(), int(), ... You name it
in builtins or operator, it's pretty much an overloaded function.
These functions differ from "full" overloaded functions in only these respects:
1. There is no framework to let you define new ones
2. They are single-dispatch only (except for the binary arithmetic
operators, which have a crude double-dispatching protocol)
3. They do not allow third-party registration; classes must define
__special__ methods to register implementations
(Some other overloaded functions in Python, such as pickle.dump and
copy.copy, *do* allow third-party registrations, but they have ad-hoc
implementations rather than using a common base implementation.)
So, saying that overloaded functions are a large hammer may or may not be
meaningful, but it's certainly true that they are in *enormous* use in
today's Python, even for very small nails like determining the length of an
Indeed, the *default* way of doing almost anything in Python that involves
multiple possible implementations is to define an overloaded function --
regardless of how small the nail might be.
>However, what we're really talking about here is a scenario where you're
>defining your *own* custom annotation processor: you want the first part
>of the tuple in the expression handled by the type processing library, and
>the second part handled by the docstring processing library.
>Which says to me that the right solution is for the annotation to be split
>up into its constituent parts before the libraries ever see it.
>This could be done as Collin suggests by tampering with
>__signature__.annotations before calling each decorator, but I think it is
>cleaner to do it by defining a particular signature for decorators that
>are intended to process annotations.
Now you're embedding a particular implementation again. The way to do this
that imposes the least constraints on users, is to just have an
'iter_annotations()' overloadable function, and let it iterate over lists
and tuples, and yield anything else, e.g.:
for a in annotation:
for aa in iter_annotations(a):
Now, if you have some custom annotation type that contains other
annotations, you need only add a method to iter_annotations, and everything
In contrast, your approach is too limiting because you're *creating a
framework* that then everyone has to conform to. I want annotations to be
framework-free. I don't even think that the stdlib needs to provide an
iter_annotations function, because there's no reason not to just define a
method similar to the above for the specific operations you're doing.
In fact the general rule of overloadable functions is that the closer to
the domain semantics the function is, the better. For example, a
'generateCodeFor(annotation)' overloaded function that can walk annotation
sequences itself is a better idea than writing a non-overloaded function
that uses iter_annotations() and then generates code for individual
annotations, because it allows for better overloads.
For example, if you have a type that contains something that would
ordinarily be considered separate annotation objects, but which the code
generator could combine in some way to produce more optimal code. Walking
the annotations and then generating code would rob you of the opportunity
to define an optimization overload in this case.
And *that* is why I don't think the stdlib should impose any semantics on
annotations -- semantic imposition doesn't *fix* incompatibility, it
How? Because if somebody needs to do something that doesn't fit within the
imposed semantics, they are forced to create their own, and they now must
reinvent everything so it works with their own!
This is the history of Python frameworks in a nutshell, and it's entirely
avoidable. We should leave the semantics open, precisely so that it will
force people to make their code *extensible*. As a side benefit, it
provides a nice example of when and how to use overloaded functions
More information about the Python-3000