[Python-3000] Draft pre-PEP: function annotations

[Talin]

Josiah Carlson wrote:
> "Phillip J. Eby" <pje at telecommunity.com> wrote:
>> At 06:10 AM 8/11/2006 -0700, Talin <talin at acm.org> wrote:
>>> Or to put it another way: If you create a tool, and you assume that tool
>>> will only be used in certain specific ways, but you fail to enforce that
>>> limitation, then your assumption will be dead wrong. The idea that there
>>> will only be a few type annotation providers who will all nicely
>>> cooperate with one another is just as naive as I was in the SysEx debacle.
>> Are you saying that function annotations are a bad idea because we won't be 
>> able to pickle them?
> 
> That is not what I got out of the message at all.
> 
>> If not, your entire argument seems specious.  Actually, even if that *is* 
>> your argument, it's specious, since all that's needed to support pickling 
>> is to support pickling.  All that's needed to support printing is to 
>> support printing (via __str__), and so on.
> 
> I think you misunderstood Talin.  While it was a pain for him to work
> his way through implementing all of the loading/etc. protocols, I
> believe his point was that if we allow any and all arbitrary metadata to
> be placed on arguments to and from functions, then invariably there will
> be multiple methods of doing as much.  That isn't a problem unto itself,
> but when there ends up being multiple metadata formats, with multiple
> interpretations of them, and a user decides that they want to combine
> the functionality of two metadata formats, they may be stuck due to
> incompatibilities, etc.
> 
> I think that it can be fixed by defining a standard mechanism for
> 'metadata chaining', one involving tuples and/or dictionaries.
> 
> Say, for example, we have the following function definition:
>     def foo(argn:meta=dflt):
>         ...
> 
> Since meta can take on the value of a Python expression (executed during
> compile-time), a tuple-based chaining would work like so:
> 
>     @chainmetadatatuple(meta_fcn1, meta_fcn2)
>     def foo(argn:(meta1, meta2)=dflt):
>         ...
> 
> And a dictionary-based chaining would work like so:
>     @chainmetadatadict(m1=meta_fcn1, m2=meta_fcn2)
>     def foo(argn:{'m1'=meta1, 'm2'=meta2}=dflt):
>         ...
> 
> The reason to include the dict-based option is to allow for annotations
> to be optional.
> 
> 
> This method may or may not be good.  But, if we don't define a standard
> method for metadata to be combined from multiple protocols, etc., then
> we could end up with incompatabilities.  However, if we do define a
> standard chaining mechanism, then it can be used, and presumably
> we shouldn't run into problems relating to incompatible annotation, etc.
> 
> 
>  - Josiah

Josiah is essentially correct in his interpretation of my views. I 
really don't understand what Phillip is talking about here.

Say I want to annotate a specific argument with two pieces of 
information, a type and a docstring. I have two metadata interpreters, 
one which uses the type information to restrict the kinds of arguments 
that can be passed in, and another which uses the docstring to enhance 
the generated documentation.

Now, lets say that these two metadata interpreters were written by two 
people, who are not in communication with each other. Each one decides 
that they would like to "play nice" with other competing metadata.

So Author A, who wrote the annotation decorator that looks for 
docstrings, decides that not only will he accept docstring annotations, 
but if the annotation is a tuple, then he will search that tuple for any 
docstrings, skipping over any annotations that he doesn't understand. 
(Although how he is supposed to manage that is unclear - since there 
could also be other annotations that are simple text strings as well.)

Author B, who wrote the type-enforcement module, also wants to play nice 
with others, but since he doesn't know A, comes up with a different 
solution. His idea is to create a system in which annotations 
automatically chain each other - so that each annotation has a "next" 
attribute referring to the next annotation.

So programmer C, who wants to incorporate both A and B's work into his 
program, has a dilemma - each has a sharing mechanism, but the sharing 
mechanisms are different and incompatible. So he is unable to apply both 
A-type and B-type metadata to any given signature.

What happens next is that C complains to both A and B (and in the 
process introducing them to each other.) A and B exchange emails, and 
reach the conclusion that B will modify his library to confirm to the 
sharing mechanism of A.

What this means is that A and B have created a defacto standard. Anyone 
who wants to interoperate with A and B have to write their interpreter 
to conform to the sharing mechanism defined by A and B.

But it also means that anyone outside of the ABC clique will not know 
about A&B's sharing convention, which means that their metadata 
interpreter will not be able to interoperate with A&B-style metadata. So 
in essence, A&B have now "captured" the space of annotations - that is, 
anyone who conforms to the A&B protocol can combine their annotations 
together; Anyone outside that group is excluded from interoperating.

Finally, lets say that A&B eventually become well-known enough that 
their sharing convention becomes the defacto standard. Any metadata that 
wants to interoperate with other metadata-interpretation libraries will 
have to follow the A&B convention. Any metadata library that chooses to 
use a different convention will be at a severe disadvantage, since they 
won't be able to be used together with other metadata interpreters.

What this means is that, despite the statements that annotations have no 
defined format or meaning, the fact is that they now do: The defacto A&B 
sharing convention. The sharing convention tells metadata interpreters 
how to distinguish between metadata that they can interpret, and how to 
skip over other metadata.

So in other words, since the original author of the annotation system 
failed to provide a convention for multiple annotations, they force the 
community to fill in the parts of the standard that they left out.

-- Talin