[Python-Dev] PEP 563: Postponed Evaluation of Annotations

INADA Naoki songofacandy at gmail.com
Thu Nov 2 20:55:39 EDT 2017


I'm 100% agree with Łukasz and Brett.
+1 and thanks for writing this PEP.
INADA Naoki  <songofacandy at gmail.com>


On Fri, Nov 3, 2017 at 2:00 AM, Brett Cannon <brett at python.org> wrote:
>
>
> On Thu, 2 Nov 2017 at 08:46 Steven D'Aprano <steve at pearwood.info> wrote:
>>
>> On Wed, Nov 01, 2017 at 03:48:00PM -0700, Lukasz Langa wrote:
>>
>> > PEP: 563
>> > Title: Postponed Evaluation of Annotations
>>
>> > This PEP proposes changing function annotations and variable annotations
>> > so that they are no longer evaluated at function definition time.
>> > Instead, they are preserved in ``__annotations__`` in string form.
>>
>> This means that now *all* annotations, not just forward references, are
>> no longer validated at runtime and will allow arbitrary typos and
>> errors:
>>
>> def spam(n:itn):  # now valid
>>     ...
>>
>> Up to now, it has been only forward references that were vulnerable to
>> that sort of thing. Of course running a type checker should pick those
>> errors up, but the evaluation of annotations ensures that they are
>> actually valid (not necessarily correct, but at least a valid name),
>> even if you happen to not be running a type checker. That's useful.
>>
>> Are we happy to live with that change?
>
>
> I would say "yes" for two reasons. One, if you're bothering to provide type
> hints then you should be testing those type hints. So as you pointed out,
> Steve, that will be caught at that point.
>
> Two, code editors with auto-completion will help prevent this kind of typo.
> Now I would never suggest that we design Python with expectations of what
> sort of tooling people have available, but in this instance it will help. It
> also feeds into a question you ask below...
>
>>
>>
>>
>> > Rationale and Goals
>> > ===================
>> >
>> > PEP 3107 added support for arbitrary annotations on parts of a function
>> > definition.  Just like default values, annotations are evaluated at
>> > function definition time.  This creates a number of issues for the type
>> > hinting use case:
>> >
>> > * forward references: when a type hint contains names that have not been
>> >   defined yet, that definition needs to be expressed as a string
>> >   literal;
>>
>> After all the discussion, I still don't see why this is an issue.
>> Strings makes perfectly fine forward references. What is the problem
>> that needs solving? Is this about people not wanting to type the leading
>> and trailing ' around forward references?
>
>
> I think it's mainly about the next point you ask about...
>
>>
>>
>>
>> > * type hints are executed at module import time, which is not
>> >   computationally free.
>>
>> True; but is that really a performance bottleneck? If it is, that should
>> be stated in the PEP, and state what typical performance improvement
>> this change should give.
>>
>> After all, if we're going to break people's code in order to improve
>> performance, we should at least be sure that it improves performance :-)
>
>
> The cost of constructing some of the objects used as type hints can be very
> expensive and make importing really expensive (this has been pointed out by
> Lukasz previously as well as Inada-san). By making Python itself not have to
> construct objects from e.g. the 'typing' module at runtime, you then don't
> pay a runtime penalty for something you're almost never going to use at
> runtime anyway.
>
>>
>>
>>
>> > Postponing the evaluation of annotations solves both problems.
>>
>> Actually it doesn't. As your PEP says later:
>>
>> > This PEP is meant to solve the problem of forward references in type
>> > annotations.  There are still cases outside of annotations where
>> > forward references will require usage of string literals.  Those are
>> > listed in a later section of this document.
>>
>> So the primary problem this PEP is designed to solve, isn't actually
>> solved by this PEP.
>
>
> I think the performance bit is really the big deal here.
>
> And as I mentioned earlier, if you turn all of your type hints into strings,
> you lose auto-completion/intellisense which is a shame.
>
> I think there's also a benefit here of promoting the fact that type hints
> are not a runtime thing, they are a static analysis thing. By requiring the
> extra step to convert from a string to an actual object, it helps get the
> point across that type hints are just bits of metadata for tooling and not
> something you're expected really interact with at runtime unless you have a
> really good reason to.
>
> So I'm +1 on the idea, but the __future__ statement is a bit too generic for
> me. I would prefer something like `from __future__ import
> annotation_strings` or `annotations_as_strings`.
>
> -Brett
>
>>
>>
>> (See Guido's comments, quoted later.)
>>
>>
>>
>> > Implementation
>> > ==============
>> >
>> > In Python 4.0, function and variable annotations will no longer be
>> > evaluated at definition time.  Instead, a string form will be preserved
>> > in the respective ``__annotations__`` dictionary.  Static type checkers
>> > will see no difference in behavior,
>>
>> Static checkers don't see __annotations__ at all, since that's not
>> available at edit/compile time. Static checkers see only the source
>> code. The checker (and the human reader!) will no longer have the useful
>> clue that something is a forward reference:
>>
>>     # before
>>     class C:
>>         def method(self, other:'C'):
>>             ...
>>
>> since the quotes around C will be redundant and almost certainly left
>> out. And if they aren't left out, then what are we to make of the
>> annotation? Will the quotes be stripped out, or left in?
>>
>> In other words, will method's __annotations__ contain 'C' or "'C'"? That
>> will make a difference when the type hint is eval'ed.
>>
>>
>> > If an annotation was already a string, this string is preserved
>> > verbatim.
>>
>> That's ambiguous. See above.
>>
>>
>> > Annotations can only use names present in the module scope as postponed
>> > evaluation using local names is not reliable (with the sole exception of
>> > class-level names resolved by ``typing.get_type_hints()``).
>>
>> Even if you call get_type_hints from inside the function defining the
>> local names?
>>
>> def function():
>>     A = something()
>>     def inner(x:A)->int:
>>         ...
>>     d = typing.get_type_hints(inner)
>>     return (d, inner)
>>
>> I would expect that should work. Will it?
>>
>>
>> > For code which uses annotations for other purposes, a regular
>> > ``eval(ann, globals, locals)`` call is enough to resolve the
>> > annotation.
>>
>> Let's just hope nobody doing that has allowed any tainted strings to
>> be stuffed into __annotations__.
>>
>>
>> > * modules should use their own ``__dict__``.
>>
>> Which is better written as ``vars()`` with no argument, I believe. Or
>> possibly ``globals()``.
>>
>>
>> > If a function generates a class or a function with annotations that
>> > have to use local variables, it can populate the given generated
>> > object's ``__annotations__`` dictionary directly, without relying on
>> > the compiler.
>>
>> I don't understand this paragraph.
>>
>>
>> > The biggest controversy on the issue was Guido van Rossum's concern
>> > that untokenizing annotation expressions back to their string form has
>> > no precedent in the Python programming language and feels like a hacky
>> > workaround.  He said:
>> >
>> >     One thing that comes to mind is that it's a very random change to
>> >     the language.  It might be useful to have a more compact way to
>> >     indicate deferred execution of expressions (using less syntax than
>> >     ``lambda:``).  But why would the use case of type annotations be so
>> >     all-important to change the language to do it there first (rather
>> >     than proposing a more general solution), given that there's already
>> >     a solution for this particular use case that requires very minimal
>> >     syntax?
>>
>> I agree with Guido's concern here. A more general solution would
>> (hopefully!) be like a thunk, and might allow some interesting
>> techniques unrelated to type checking. Just off the top of my head, say,
>> late binding of default values (without the "if arg is None: arg = []"
>> trick).
>>
>>
>> > A few people voiced concerns that there are libraries using annotations
>> > for non-typing purposes.  However, none of the named libraries would be
>> > invalidated by this PEP.  They do require adapting to the new
>> > requirement to call ``eval()`` on the annotation with the correct
>> > ``globals`` and ``locals`` set.
>>
>> Since this is likely to be a common task for such libraries, can we have
>> a evaluate_annotations() function to do this, rather than have everyone
>> reinvent the wheel?
>>
>> def func(arg:int):
>>     ...
>>
>> evaluate_annotations(func)
>> assert func.__annotations__['arg'] is int
>>
>> It could be a decorator, as well as modifying __annotations__ in place.
>>
>> I imagine something with a signature like this:
>>
>> def evaluate_annotations(
>>         obj:Union[Function, Class],
>>         globals:Dict=None,
>>         locals:Dict=None
>>         )->Union[Function, Class]:
>>     """Evaluate the __annotations__ of a function, or recursively
>>     a class and all its methods. Replace the __annotations__ in
>>     place. Returns the modified argument, making this suitable as
>>     a decorator.
>>
>>     If globals is not given, it is taken from the function.__globals__
>>     or class.__module__ if available. If locals is not given, it
>>     defaults to the current locals.
>>     """
>>
>>
>> --
>> Steve
>> _______________________________________________
>> Python-Dev mailing list
>> Python-Dev at python.org
>> https://mail.python.org/mailman/listinfo/python-dev
>> Unsubscribe:
>> https://mail.python.org/mailman/options/python-dev/brett%40python.org
>
>
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> https://mail.python.org/mailman/options/python-dev/songofacandy%40gmail.com
>


More information about the Python-Dev mailing list