[Python-ideas] Proposal: Use mypy syntax for function annotations

[Guido van Rossum]

On Mon, Aug 25, 2014 at 6:57 AM, Antoine Pitrou <antoine at python.org> wrote:

>
> Le 24/08/2014 23:12, Guido van Rossum a écrit :
>
> > As I have said several times now, it fulfills exactly the original
> > (pre-PEP 3107) goal I had in mind for them.
>
> But were you envisioning back then that said annotations would be
> looked up without the regular execution environment?
>
> PEP 3107 doesn't say anything about that (or rather, it says that
> annotations can be looked up on the function, but function objects only
> exist at run-time). Actually, the bytecode isn't very practical to work
> with to extract annotations, it seems:
>
> >>> def g():
> ...   def f(a: "foo") -> "bar": pass
> ...
> >>> dis.dis(g)
>   2           0 LOAD_CONST               1 ('foo')
>               3 LOAD_CONST               2 ('bar')
>               6 LOAD_CONST               3 (('a', 'return'))
>               9 LOAD_CONST               4 (<code object f at
> 0x7f17339e2580, file "<stdin>", line 2>)
>              12 LOAD_CONST               5 ('g.<locals>.f')
>              15 EXTENDED_ARG             3
>              18 MAKE_FUNCTION        196608
>              21 STORE_FAST               0 (f)
>              24 LOAD_CONST               0 (None)
>              27 RETURN_VALUE
>
> ... so I suppose people would want to run an AST pass instead?
> (and then evaluate the "annotations" parts of the AST by hand... uh)
>

Actually mypy uses an entirely different parser. This is mostly for
historical reasons: it started out as a compiler/checker/runtime for a
different, somewhat-Python-like statically-typed language. But I assume
that static checkers often have no choice in the matter and must write
their own parser -- e.g. PyCharm is itself written in Java and its parser
must work even if the code is not syntactically correct.

Another reason to use a different parser is that mypy needs to see the
comments so it can look for magic comments starting with "#type:". This is
typical for linters.

In general a type checker should not attempt to cope with clever things
(decorators or otherwise) that modify __annotations__; it should take the
expression tree of the annotation at face value. (It should of course know
about certain decorators like @property and other Python ideosyncracies
like self-passing for methods, and I think there should be a way to teach
it about new decorators. But executing Python code should not be part of
that.)


> >     Your proposal seems determined by the fact that mypy has much
> >     grander ambitions (and therefore requires its insertion into regular
> >     Python), but it doesn't make use of that power at all, worse, it
> >     forbids others to use it.
> >
> > This remark about mypy's ambitions sounds delusional and paranoid.
>
> It is entirely uncritical about mypy. It's fine to have other Python
> (or Python-like) implementations, even with deliberately different
> semantics. Such experimentations actually make the community much livelier
> than, say, PHP's or Ruby's.
>

OK, thanks for clarifying that. I took "grand ambitions" as a
(sarcastically) pejorative term, but you're right that it doesn't need to
be read like that.


> I don't know at which point mypy changed goals (if it has), but there are
> still signs in the website of the goal of building a separate runtime (not
> necessarily a separate syntax), e.g.
>
> """Also some language features that are evaluated at runtime in Python
> may happen during compilation in mypy when using the native semantics.
> For example, mypy base classes may be bound during compilation (or
> program loading, before evaluation), unlike Python."""
>

I have talked to Jukka about this, and he is definitely on board with
reducing mypy's functionality to that of a linter; I think there's even an
issue in mypy's tracker about removing the ability to execute the code
(currently there's a flag you must pass to prevent it from trying to run
the code). Updating the website is an ongoing low-priority project (you can
probably send him pull requests for it).


> Also the fact that mypy supports constructs such as "List[int]()". This
> is a set of design constraints that you're not bound to.
>

Yeah, I'm not crazy about that syntax (I think nobody is). It mostly exists
for the common use case where you have a function that is declared to
return e.g. a list of ints and you want to start off with an empty list;
the current mypy implementation complains about such code. Example:

def squares(xs: Sequence[float]) -> List[float]:
    sqs = []
    for x in xs:
        sqs.append(x**2)
    return sqs

mypy complains about the unannotated initialization of sqs. The two ways to
currently address this are

    sqs = []  # type: List[float]

or

    sqs = List[float]()

Neither is very attractive; mypy should just infer the type of sqs.

Nevertheless, I don't want to use call syntax to set parameters for generic
types, since a generic type still "feels" sufficiently like a class that
calling it is easily confused with instantiation -- even though ABCs are
typically not instantiable. (There's no hard rule for that though -- it is
merely the result of typical ABCs having at least one abstract method.)

-- 
--Guido van Rossum (python.org/~guido)
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140825/9fdcc983/attachment.html>