[Python-3000] Draft pre-PEP: function annotations

[Talin]

Phillip J. Eby wrote:
> At 03:00 PM 8/12/2006 -0700, Talin <talin at acm.org> wrote:
>> Phillip J. Eby wrote:
>> > At 12:38 PM 8/12/2006 -0700, Paul Prescod wrote:
>>
>> > However, if you have:
>> >
>> >    def myfunc( x : doc("The x coordinate"), y : doc("The y 
>> coordinate") )
>> >
>> > There is no ambiguity.  Likewise:
>> >
>> >    def cat( infile:opt("i") = sys.stdin, outfile:opt("o") = 
>> sys.stdout ):
>> >
>> > is unambiguous.  And the interpetation of:
>> >
>> >    def cat(infile: [doc("input stream"), opt("i")] = sys.stdin,
>> >            outfile: [doc("output stream"), opt("o")] = sys.stdout
>> >    ):
>>
>> By doing this, you've already introduced an implicit requirement for
>> annotations: Rather than saying that annotations can be "any format you
>> want", the actual restriction is "any format you want that is
>> distinguishable from other formats."
> 
> And your point is what?
> 

My point is that this statement in the Collin's PEP is wrong:

 > There is no worry that these libraries will assign semantics at
 > random, or that a variety of libraries will appear, each with varying
 > semantics and interpretations of what, say, a tuple of strings
 > means. The difficulty inherent in writing annotation interpreting
 > libraries will keep their number low and their authorship in the
 > hands of people who, frankly, know what they're doing.

The way I read this is "there is no need for annotations to be designed 
so as not to interfere with one another, nor does there need to be any 
mechanism defined in this PEP for resolving such interference". I and 
others have provided extensive use cases to show that unless care is 
taken, different annotations *will* step on each others toes.

>>  More specifically, the rule is that
>> annotations intended for different consumers must be distinguishable
>> from each other via rule. This is in direct contradiction with the
>> statement in the PEP that says that annotations have no predefined
>> syntax or semantics -- they are required to have, at minimum, semantics
>> sufficient to allow rule-based discrimination.
> 
> You've lost me here entirely.  If we didn't want unambiguous semantics, 
> we'd write programs in English, not Python.  :)

Again, look at the language of the PEP.

>> (BTW, I propose the term "Annotation Consumer" to mean a body of code
>> that is intended to process annotations. You can have decorator-based
>> consumers, as well as external consumers that are not part of the
>> decorator stack and which inspect the function signature directly,
>> without invoking the decorators.)
> 
> Um, okay.  I'm not sure what benefit this new term adds over "operation 
> that uses annotations", which is what I've been using, but whatever.
> 
I'm just trying to get a handle on this stuff so that we can *talk* 
about it.

>> Lets use the term 'discriminator' to indicate any means, using function
>> overloading or whatever, of determining which consumers should process
>> which annotations. Lets also define the term 'discriminator protocol' to
>>   mean any input specifications to the discriminator - so in the above
>> example, 'doc()' and 'opt()' are part of the discriminator protocol.
> 
> Um, what?  Why are you adding all this complication to a simple idea?

I'm not adding anything to the concept, I am trying to come up with a 
way to *talk* about the concept. So far the whole conversation has 
gotten very confused because we're dealing with some highly abstract 
stuff here.

> Duck typing is normal, simple, standard Python programming practice.  We 
> use objects with methods all the time, and check for the existence of 
> attributes all the time.
> 
> I don't understand why you insist on making that more complicated than 
> it is.  It's really simple.  Annotations are objects.  Objects can be 
> inspected, or selected by type.  You can do what you want to with them.
> 
> How complex is that?

It gets complex when you have more than one inspector or selector. What 
we are arguing about is how much the various inspectors/selectors need 
to know about each other. And while the answer is hopefully "not much", 
I hope that I have shown that it cannot be "nothing at all". There has 
to be some ground rules for cooperation, or cooperation is impossible, 
that's basic logic.

> (Meanwhile, I'm going to ignore all the red herrings about freedom and 
> commerce and other rigamarole that has absolutely nothing to do with 
> argument annotations.)

Don't think of it as red herrings. Think of it as, um, "highly 
non-linear train of thought". :)

> Going forward, may I suggest you take a look at Java and C# argument 
> annotations before continuing to pursue this spurious line of 
> reasoning?  I'm curious to see what your explanation will be for why 
> these other languages doesn't have the problems that you claim will 
> inevitably occur.

Dude, you don't want to know how many man-years of C# programming I've 
done :)

Lets take C# attributes as an example. C# Attributes have the following 
syntactical/semantic structure:

   1) They must be derived from the base class "Attribute". (This by 
itself is not really significant.)
   2) Attributes are distinguished by type, or in some cases by value.
   3) The types do not overlap.
   4) A given consumer of attributes can always distinguish attributes 
which are relevant to their purposes to attributes which are not, even 
against hypothetical future annotations which have not yet been established.

As a user, when I add an attribute to a method, I know that (a) there is 
a known consumer of that attribute, (b) That it is impossible for an 
attribute which is not intended for that consumer to be confused for one 
that is. if I set [Browseable(false)] on a property, I know exactly how 
that attribute is going to be interpreted, and by what component. If 
someone comes along later and adds a new annotation called 
"SortOfBrowseable", which has many of the same attributes as Browseable, 
there will never be the possibility that there annotation and mine can 
get confused with each other. (As opposed to Python, where it's 
relatively easy to have classes that masquerade as one another.)

The Annotation PEP, on the other hand, makes none of these guarantees, 
because it tries hard not to guarantee anything. It doesn't specify the 
mechanism by which one annotation is distinguished from another; Unlike 
the C# attributes which are organized into a tree of types, the 
annotations have no organization and no categorization defined. Because 
there is no prohibition against category overlap, that means that the 
annotations that I write today might one day in the future match against 
a newly-created category, with results that I can't predict.

I also want to point out that C# attributes are very different from 
Python decorators, so you can't use analogies between them. Decorators 
are active agents - that is, they hook into the process of defining a 
method. Because of this, decorators have the option of having all of 
their semantic meaning buried within the decorator itself. In essence, 
the rule by which decorators "play nice" with each other is already 
defined - each gets a shot at modifying the function object, and each 
receives the result of the previous decorator.

C# attributes and function annotations, on the other hand, are purely 
passive - they have no knowledge of what they are attached to, and their 
only meaning is derived from external use. They themselves don't have to 
play nice with each other, but the interpreters / inspectors / consumers do.

> Meanwhile, if library authors write bad code because they don't 
> understand basic OO concepts like duck typing and "tell, don't ask", 
> then their users will educate them when they complain about not being 
> able to use multiple annotation types.
> 
> Providing good examples and recommending best practices is one thing, 
> but mandating a particular semantics is another.