[Python-ideas] Optional static typing -- the crossroads

[Terry Reedy]

On 8/15/2014 5:48 AM, Nick Coghlan wrote:
> On 15 August 2014 19:38, Terry Reedy <tjreedy at udel.edu> wrote:
>> On 8/15/2014 12:40 AM, Nick Coghlan wrote:
>>>
>>> On 15 August 2014 09:56, Guido van Rossum <guido at python.org> wrote:
>>>>
>>>>
>>>> I don't buy the argument that PEP 3107 promises that annotations are
>>>> completely free of inherent semantics.
>>>
>>>
>>> It's also worth noting the corresponding bullet point in PEP 3100
>>> (under http://www.python.org/dev/peps/pep-3100/#core-language):
>>>
>>> * Add optional declarations for static typing [45] [10] [done]
>>
>> ...
>>
>>> Linters/checkers may also want to provide a configurable way to say
>>> "the presence of decorator <X> means the annotations on that function
>>> aren't type markers". That ties in with the recommendation we added to
>>> PEP 8 a while back: "It is recommended that third party experiments
>>> with annotations use an associated decorator to indicate how the
>>> annotation should be interpreted."

I claim that the mere presence of a decorator in the *source* is not 
enough. The decorator for non-type markers should do something with 
.__annotations__ -- in particular, remove the non-type markers.  The 
presence of a decorator in the source does not necessarily leave a trace 
on the returned function object for runtime detection.

>> Depending on the checker, this suggests that non-type-check annotations need
>> not be deprecated. If a decorator wraps a function with an unannotated
>> wrapper, then the checker should see the result as unannotated, rather than
>> looking for a wrapped attribute. Also, a decorator can remove non-type
>> annotations and act on them, store them in a closure variable, or store them
>> on the function in a different name.

"Depending on the checker" alludes to the fact that there are two types 
of annotation consumers: those that read the source or parsed source 
(ast) and the annotations therein and those that look at the function 
.__annotations__ attribute after compilation.  Inspect.signature is an 
example of the latter. (If there were none, there would be no purpose to 
.__annotations__!)

Suppose an integer square root function were annotated with with type 
and non-type information.

 >>> def nsqrt(n:(int, 'random non-type info'))->(int, 'more non-type 
info'): pass

To me, inspect.signature already assumes that annotations are about 
type, which means that this horse has already left barn. In 3.4.1:

 >>> from inspect import signature as sig
 >>> str(sig(nsqrt))
"(n:(<class 'int'>, 'random non-type info')) -> (<class 'int'>, 'more 
non-type info')"

Typing 'nsqrt(' in Idle and pausing a fraction of a second brings up a 
calltip with the same string (without the outer quotes). To me, having 
random non-type info in the signature and calltip is noise and therefore 
wrong. So I agree that the non-standard annotation should be signaled by 
a decorator *and* suggest that the decorator should remove the 
non-standard annotation, which is easily done, so that the signature 
string for the above would be

"(n:<class 'int') -> <class 'int'>"

I summarized the above, perhaps not the best I could have, with

"Given these possibilities, all that is needs be said is "After a 
function is post-processed by decorators, any remaining annotations 
should be for type-checking or documentation."

> No, many (most?)

The future relative proportion of pre- and post-compile annotation 
consumers is not relevant to my argument. The stdlib already has a 
important post-compile consumer that is already somewhat broken by 
non-type info remaining in .__annotations__.

 > linters and IDEs will run off the AST without
> actually executing the code, so they'll see the annotations, even if
> they get stripped by the decorator at runtime.

Being aware of this, I concluded the post with the following that 
already said this.

"For checkers that do look at the source, or the AST before compiling,"

*and* I went on to suggest a solution.

"the rule could be to ignore string annotations. Decorators can always 
eval, or perhaps safe_eval, strings."

In other words, if type annotations were to be classes, as proposed, 
then non-type annotations should not be classes, so that pre-compile 
annotation consumers could easily ignore them. In particular, I 
suggested literal strings, which are easily recognized in source, as 
well as in asts.


To put this all another way --

The new-in-3.0 annotation feature has two components: a python function 
source syntax, and a function compile behavior of adding a dict 
attribute as .__annotations__.  If I understand correctly, Argument 
Clinic piggybacks on this by adding a mechanism to produce 
.__annotations__ from C source -- mainly for use by .signature, but also 
by another other .__annotations__ users. The two components -- source 
annotations and .__annotations__ dict -- each have their consumers.

Currently, annotation values are untyped (or AnyTyped).  Guido has 
proposed favoring type annotations. I support that and suggest that such 
favoritism is already needed for the annotation dict. So I would 
strengthen the PEP 8 recommendation.

Guido has also suggested 'deprecating' non-type annotations.  That would 
literally mean raising an error either when source is compiled or when 
def statements are executed.  I think deprecation in this sense is both 
unwise, since non-type annotation were explicitly invited, and 
unnecessary for the purpose of favoring type annotations. The point of 
my previous post was to explore what restrictions *are* necessary.  In 
summary, I suggest 1. use distinct syntax (this depends on what is 
adopted for type annotations); 2. decorate (as already suggested in PEP 
8); 3. clean .__annotations__ (which should also be suggested in PEP 8).

-- 
Terry Jan Reedy