[Python-ideas] Optional static typing -- the crossroads

[Steven D'Aprano]

On Fri, Aug 15, 2014 at 05:18:11PM -0400, Terry Reedy wrote:
[...]
> "Depending on the checker" alludes to the fact that there are two types 
> of annotation consumers: those that read the source or parsed source 
> (ast) and the annotations therein and those that look at the function 
> .__annotations__ attribute after compilation.  Inspect.signature is an 
> example of the latter. (If there were none, there would be no purpose to 
> .__annotations__!)

Agreed.


> Suppose an integer square root function were annotated with with type 
> and non-type information.
> 
> >>> def nsqrt(n:(int, 'random non-type info'))->(int, 'more non-type 
> info'): pass

I don't think it is reasonable to expect arbitrary annotation tools to 
interoperate, unless they are specifically designed to interoperate. If 
tool A expects annotations to be a certain thing, and tool B expects 
them to be a different thing, they are going to confuse each other. We 
need a convention so that tools which expect to work with annotations 
can identify which annotations are aimed at them.

E.g. tool A might decorate the function with a marker that says "A", so 
that tool B knows to skip those functions. And vice versa. In the 
absence of any such marker, annotations can be assumed to be standard 
mypy-style type annotations.

(The nature of this marker probably should be standardized. I suggest a 
key/item in __annotations__, where the key cannot clash with parameter 
names.)

The easiest way to apply that marker is with a decorator: perhaps the 
typing module could provide a standard decorator that all annotation 
tools can recognise at compile-time:

@register_annotations(A.marker)  # for example
def function(x:"something understandable by A"):
    ...

# inside module A
marker = object()
def introspect(func):
    if magic(func) is marker:
        # okay to operate on func


> To me, inspect.signature already assumes that annotations are about 
> type, which means that this horse has already left barn. In 3.4.1:

I don't see how that follows. Your example demonstrates that inspect 
treats annotations as arbitrary Python expressions (which is what they 
are). You annotate the n parameter with the expression (int, "random 
non-type info"), which is a tuple of two objects. And signature 
dutifully reports that:

> >>> from inspect import signature as sig
> >>> str(sig(nsqrt))
> "(n:(<class 'int'>, 'random non-type info')) -> (<class 'int'>, 'more 
> non-type info')"
> 
> Typing 'nsqrt(' in Idle and pausing a fraction of a second brings up a 
> calltip with the same string (without the outer quotes). To me, having 
> random non-type info in the signature and calltip is noise and therefore 
> wrong.

Then don't put random info in the annotations :-)

If Idle has documented that the calltip is *always* type information, 
then Idle is wrong. Currently, there is no standard interpretation of 
function annotations, and if Idle documentation says otherwise, it is 
the documentation which is wrong.

In the future, I can see that Idle might want to only display calltips 
that it knows contain type information, or perhaps show them slightly 
differently if they are not type annotations. Or perhaps not... see 
below.


> So I agree that the non-standard annotation should be signaled by 
> a decorator 

I agree whole-heartedly to this.


> *and* suggest that the decorator should remove the 
> non-standard annotation, which is easily done, so that the signature 
> string for the above would be

But I disagree equally as strongly to this. Other uses of annotations 
are just as valid as static typing, and may equally want to be available 
for runtime introspection.

All we need is some sort of standardised runtime marker whereby tools 
can decide whether or not they should use the annotations.


> The future relative proportion of pre- and post-compile annotation 
> consumers is not relevant to my argument. The stdlib already has a 
> important post-compile consumer that is already somewhat broken by 
> non-type info remaining in .__annotations__.

I think it is only broken if you treat Idle calltips as displaying 
*types*. If you treat Idle calltips as displaying *annotations* no 
matter what the nature of those annotations, then it is not broken in 
the least. You yourself call them *call* tips, not "type tips", so there 
could be useful information provided other than the types of arguments. 

Consider two (imaginary) annotations in a graphics library:

def move(x: int, y:int): ...

def move(x: "distance along the left-right axis", 
         y: "distance along the up-down axis"): ...

I think that the second would be far more useful in a library 
aimed at beginners.


[...]
> Guido has also suggested 'deprecating' non-type annotations.  That would 
> literally mean raising an error either when source is compiled or when 
> def statements are executed.

Not necessarily. It could mean just documenting that we shouldn't use 
function annotations for anything other than specifying types. The usual 
procedure for deprecations is:

* for at least one release, tell people not to do this, but don't 
  raise a warning or an exception;
* for at least one release, raise a warning but not an exception;
* for at least one release, raise an exception

"For at least one release" might mean "until Python 5000".


> I think deprecation in this sense is both 
> unwise, since non-type annotation were explicitly invited, and 
> unnecessary for the purpose of favoring type annotations.

I whole-heartedly agree with this part!



-- 
Steven