<div dir="ltr">I like it. For previous discussion of this idea see here:<div><a href="https://mail.python.org/pipermail/python-ideas/2016-September/042527.html">https://mail.python.org/pipermail/python-ideas/2016-September/042527.html</a> </div><div><br></div><div>I don't see this mentioned in the PEP, but it will also allow (easy) description of contracts and dependent types.</div><div><br></div><div>Elazar</div></div><br><div class="gmail_quote"><div dir="ltr">On Mon, Sep 11, 2017 at 6:59 PM Lukasz Langa <<a href="mailto:lukasz@langa.pl">lukasz@langa.pl</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">PEP: 563<br>
Title: Postponed Evaluation of Annotations<br>
Version: $Revision$<br>
Last-Modified: $Date$<br>
Author: Łukasz Langa <<a href="mailto:lukasz@langa.pl" target="_blank">lukasz@langa.pl</a>><br>
Discussions-To: Python-Dev <<a href="mailto:python-dev@python.org" target="_blank">python-dev@python.org</a>><br>
Status: Draft<br>
Type: Standards Track<br>
Content-Type: text/x-rst<br>
Created: 8-Sep-2017<br>
Python-Version: 3.7<br>
Post-History:<br>
Resolution:<br>
<br>
<br>
Abstract<br>
========<br>
<br>
PEP 3107 introduced syntax for function annotations, but the semantics<br>
were deliberately left undefined. PEP 484 introduced a standard meaning<br>
to annotations: type hints. PEP 526 defined variable annotations,<br>
explicitly tying them with the type hinting use case.<br>
<br>
This PEP proposes changing function annotations and variable annotations<br>
so that they are no longer evaluated at function definition time.<br>
Instead, they are preserved in ``__annotations__`` in string form.<br>
<br>
This change is going to be introduced gradually, starting with a new<br>
``__future__`` import in Python 3.7.<br>
<br>
<br>
Rationale and Goals<br>
===================<br>
<br>
PEP 3107 added support for arbitrary annotations on parts of a function<br>
definition. Just like default values, annotations are evaluated at<br>
function definition time. This creates a number of issues for the type<br>
hinting use case:<br>
<br>
* forward references: when a type hint contains names that have not been<br>
defined yet, that definition needs to be expressed as a string<br>
literal;<br>
<br>
* type hints are executed at module import time, which is not<br>
computationally free.<br>
<br>
Postponing the evaluation of annotations solves both problems.<br>
<br>
Non-goals<br>
---------<br>
<br>
Just like in PEP 484 and PEP 526, it should be emphasized that **Python<br>
will remain a dynamically typed language, and the authors have no desire<br>
to ever make type hints mandatory, even by convention.**<br>
<br>
Annotations are still available for arbitrary use besides type checking.<br>
Using ``@typing.no_type_hints`` in this case is recommended to<br>
disambiguate the use case.<br>
<br>
<br>
Implementation<br>
==============<br>
<br>
In a future version of Python, function and variable annotations will no<br>
longer be evaluated at definition time. Instead, a string form will be<br>
preserved in the respective ``__annotations__`` dictionary. Static type<br>
checkers will see no difference in behavior, whereas tools using<br>
annotations at runtime will have to perform postponed evaluation.<br>
<br>
If an annotation was already a string, this string is preserved<br>
verbatim. In other cases, the string form is obtained from the AST<br>
during the compilation step, which means that the string form preserved<br>
might not preserve the exact formatting of the source.<br>
<br>
Annotations need to be syntactically valid Python expressions, also when<br>
passed as literal strings (i.e. ``compile(literal, '', 'eval')``).<br>
Annotations can only use names present in the module scope as postponed<br>
evaluation using local names is not reliable.<br>
<br>
Note that as per PEP 526, local variable annotations are not evaluated<br>
at all since they are not accessible outside of the function's closure.<br>
<br>
Enabling the future behavior in Python 3.7<br>
------------------------------------------<br>
<br>
The functionality described above can be enabled starting from Python<br>
3.7 using the following special import::<br>
<br>
from __future__ import annotations<br>
<br>
<br>
Resolving Type Hints at Runtime<br>
===============================<br>
<br>
To resolve an annotation at runtime from its string form to the result<br>
of the enclosed expression, user code needs to evaluate the string.<br>
<br>
For code that uses type hints, the ``typing.get_type_hints()`` function<br>
correctly evaluates expressions back from its string form. Note that<br>
all valid code currently using ``__annotations__`` should already be<br>
doing that since a type annotation can be expressed as a string literal.<br>
<br>
For code which uses annotations for other purposes, a regular<br>
``eval(ann, globals, locals)`` call is enough to resolve the<br>
annotation. The trick here is to get the correct value for globals.<br>
Fortunately, in the case of functions, they hold a reference to globals<br>
in an attribute called ``__globals__``. To get the correct module-level<br>
context to resolve class variables, use::<br>
<br>
cls_globals = sys.modules[SomeClass.__module__].__dict__<br>
<br>
Runtime annotation resolution and class decorators<br>
--------------------------------------------------<br>
<br>
Metaclasses and class decorators that need to resolve annotations for<br>
the current class will fail for annotations that use the name of the<br>
current class. Example::<br>
<br>
def class_decorator(cls):<br>
annotations = get_type_hints(cls) # raises NameError on 'C'<br>
print(f'Annotations for {cls}: {annotations}')<br>
return cls<br>
<br>
@class_decorator<br>
class C:<br>
singleton: 'C' = None<br>
<br>
This was already true before this PEP. The class decorator acts on<br>
the class before it's assigned a name in the current definition scope.<br>
<br>
The situation is made somewhat stricter when class-level variables are<br>
considered. Previously, when the string form wasn't used in annotations,<br>
a class decorator would be able to cover situations like::<br>
<br>
@class_decorator<br>
class Restaurant:<br>
class MenuOption(Enum):<br>
SPAM = 1<br>
EGGS = 2<br>
<br>
default_menu: List[MenuOption] = []<br>
<br>
This is no longer possible.<br>
<br>
Runtime annotation resolution and ``TYPE_CHECKING``<br>
---------------------------------------------------<br>
<br>
Sometimes there's code that must be seen by a type checker but should<br>
not be executed. For such situations the ``typing`` module defines a<br>
constant, ``TYPE_CHECKING``, that is considered ``True`` during type<br>
checking but ``False`` at runtime. Example::<br>
<br>
import typing<br>
<br>
if typing.TYPE_CHECKING:<br>
import expensive_mod<br>
<br>
def a_func(arg: expensive_mod.SomeClass) -> None:<br>
a_var: expensive_mod.SomeClass = arg<br>
...<br>
<br>
This approach is also useful when handling import cycles.<br>
<br>
Trying to resolve annotations of ``a_func`` at runtime using<br>
``typing.get_type_hints()`` will fail since the name ``expensive_mod``<br>
is not defined (``TYPE_CHECKING`` variable being ``False`` at runtime).<br>
This was already true before this PEP.<br>
<br>
<br>
Backwards Compatibility<br>
=======================<br>
<br>
This is a backwards incompatible change. Applications depending on<br>
arbitrary objects to be directly present in annotations will break<br>
if they are not using ``typing.get_type_hints()`` or ``eval()``.<br>
<br>
Annotations that depend on locals at the time of the function/class<br>
definition are now invalid. Example::<br>
<br>
def generate_class():<br>
some_local = datetime.datetime.now()<br>
class C:<br>
field: some_local = 1 # NOTE: INVALID ANNOTATION<br>
def method(self, arg: some_local.day) -> None: # NOTE: INVALID ANNOTATION<br>
...<br>
<br>
Annotations using nested classes and their respective state are still<br>
valid, provided they use the fully qualified name. Example::<br>
<br>
class C:<br>
field = 'c_field'<br>
def method(self, arg: C.field) -> None: # this is OK<br>
...<br>
<br>
class D:<br>
field2 = 'd_field'<br>
def method(self, arg: C.field -> C.D.field2: # this is OK<br>
...<br>
<br>
In the presence of an annotation that cannot be resolved using the<br>
current module's globals, a NameError is raised at compile time.<br>
<br>
<br>
Deprecation policy<br>
------------------<br>
<br>
In Python 3.7, a ``__future__`` import is required to use the described<br>
functionality and a ``PendingDeprecationWarning`` is raised by the<br>
compiler in the presence of type annotations in modules without the<br>
``__future__`` import. In Python 3.8 the warning becomes a<br>
``DeprecationWarning``. In the next version this will become the<br>
default behavior.<br>
<br>
<br>
Rejected Ideas<br>
==============<br>
<br>
Keep the ability to use local state when defining annotations<br>
-------------------------------------------------------------<br>
<br>
With postponed evaluation, this is impossible for function locals. For<br>
classes, it would be possible to keep the ability to define annotations<br>
using the local scope. However, when using ``eval()`` to perform the<br>
postponed evaluation, we need to provide the correct globals and locals<br>
to the ``eval()`` call. In the face of nested classes, the routine to<br>
get the effective "globals" at definition time would have to look<br>
something like this::<br>
<br>
def get_class_globals(cls):<br>
result = {}<br>
result.update(sys.modules[cls.__module__].__dict__)<br>
for child in cls.__qualname__.split('.'):<br>
result.update(result[child].__dict__)<br>
return result<br>
<br>
This is brittle and doesn't even cover slots. Requiring the use of<br>
module-level names simplifies runtime evaluation and provides the<br>
"one obvious way" to read annotations. It's the equivalent of absolute<br>
imports.<br>
<br>
<br>
Acknowledgements<br>
================<br>
<br>
This document could not be completed without valuable input,<br>
encouragement and advice from Guido van Rossum, Jukka Lehtosalo, and<br>
Ivan Levkivskyi.<br>
<br>
<br>
Copyright<br>
=========<br>
<br>
This document has been placed in the public domain.<br>
<br>
<br>
<br>
..<br>
Local Variables:<br>
mode: indented-text<br>
indent-tabs-mode: nil<br>
sentence-end-double-space: t<br>
fill-column: 70<br>
coding: utf-8<br>
End:<br>
<br>
_______________________________________________<br>
Python-ideas mailing list<br>
<a href="mailto:Python-ideas@python.org" target="_blank">Python-ideas@python.org</a><br>
<a href="https://mail.python.org/mailman/listinfo/python-ideas" rel="noreferrer" target="_blank">https://mail.python.org/mailman/listinfo/python-ideas</a><br>
Code of Conduct: <a href="http://python.org/psf/codeofconduct/" rel="noreferrer" target="_blank">http://python.org/psf/codeofconduct/</a><br>
</blockquote></div>