[Python-ideas] PEP 563: Postponed Evaluation of Annotations, first draft

[Lukasz Langa]

Thanks for your detailed review!


> On Sep 11, 2017, at 9:45 PM, Steven D'Aprano <steve at pearwood.info> wrote:
> 
> Regarding forward references: I see no problem with quoting forward
> references. Some people complain about the quotation marks, but frankly
> I don't think that's a major imposition.

Are you a user of type annotations? In the introduction of typing at Facebook this is the single most annoying thing people point out. The reason is that it's not obvious and the workaround is ugly. Learning about this quirk adds to an already significant body of knowledge new typing users have to familiarize themselves with. It's also aesthetically jarring, the original PEP 484 admits this is a clunky solution.


> Regarding the execution time at runtime: this sounds like premature
> optimization. If it is not, if you have profiled some applications found
> that there is more than a trivial amount of time used by generating the
> annotations, you should say so.

I used hand-wavy language because I didn't really check before. This time around I'm coming back prepared. Instagram has roughly 10k functions annotated at this point. Using tracemalloc I tested how much memory it takes to import modules with those functions. Then I compared how much memory it takes to import modules with those functions when annotations are stripped from the entire module (incl. module-level variable annotations, aliases, class variable annotations, etc.).

The difference in allocated memory is over 22 MB. The import time with annotations is over 2s longer. The problem with those numbers that we still have 80% functions to cover.


> In my opinion, a less disruptive solution to the execution time
> (supposed?) problem is a switch to disable annotations altogether,
> similar to the existing -O optimize switch, turning them into no-ops.
> That won't make any difference to static type-checkers.
> 
> Some people will probably want such a switch anyway, if they care about
> the memory used by the __annotations__ dictionaries.

Yes, this is a possibility that I should address in the PEP explicitly. There are two reasons this is not satisfying:

1. This only addresses runtime cost, not forward references, those still cannot be safely used in source code. Even if a single one was used in a single module, the entire program now has to be executed with this new hypothetical -O switch. Nobody would agree to dependencies like that.

2. This throws the baby out with the bath water. Now *no* runtime annotation use can be performed. There's work on new tools that use PEP 484-compliant type annotations at runtime (Larry is reviving his dryparse library, Eric Smith is working on data classes). Those would not work with this hypothetical -O switch but are fine with the string form since they already need to use `typing.get_type_hints()`.


>> If an annotation was already a string, this string is preserved
>> verbatim.  In other cases, the string form is obtained from the AST
>> during the compilation step, which means that the string form preserved
>> might not preserve the exact formatting of the source.
> 
> Can you give an example of how the AST may distort the source?
> 
> My knee-jerk reaction is that anything which causes the annotation to
> differ from the source is likely to cause confusion.

Anything not explicitly stored in the AST will require a normalized untokenization. This covers whitespace, punctuation, and brackets. An example:

def fun(a: Dict[
  str,
  str,
]) -> None: ...

would become something like:

{'a': 'Dict[(str, str)]', 'return': 'None'}


>> Annotations need to be syntactically valid Python expressions, also when
>> passed as literal strings (i.e. ``compile(literal, '', 'eval')``).
>> Annotations can only use names present in the module scope as postponed
>> evaluation using local names is not reliable.
> 
> And that's a problem. Forcing the use of global variables seems harmful.
> Preventing the use of locals seems awful. Can't we use some sort of
> closure-like mechanism?

As Nick measured, currently closures would add to the performance problem, not solve it. What is an actual use case that this would prevent?


> This restriction is going to break, or prevent, situations like this:
> 
> def decorator(func):
>    kind = ... # something generated at decorator call time
>    @functools.wraps(func)
>    def inner(arg: kind):
>        ...
>    return inner
> 
> 
> Even if static typecheckers have no clue what the annotation on the
> inner function is, it is still useful for introspection and
> documentation.

Do you have a specific use case in mind? If you need to put magic dynamic annotations for runtime introspection, the decorator can apply those directly via calling

inner.__annotations__ = {'arg': 'kind'}

This is what the "attrs" library is considering doing for the magic __init__ method. Having annotation literals in the source not validated would not hinder that at all.

That's it for introspection. As for documentation, I don't really understand that comment.


>> Resolving Type Hints at Runtime
>> ===============================
> [...]
>> To get the correct module-level
>> context to resolve class variables, use::
>> 
>>    cls_globals = sys.modules[SomeClass.__module__].__dict__
> 
> A small style issue: I think that's better written as:
> 
>    cls_globals = vars(sys.modules[SomeClass.__module__])
> 
> We should avoid directly accessing dunders unless necessary, and vars()
> exists specifically for the purpose of returning object's __dict__.

Right! Would be great if it composed a dictionary of slots for us, too ;-)


>> Runtime annotation resolution and ``TYPE_CHECKING``
>> ---------------------------------------------------
>> 
>> Sometimes there's code that must be seen by a type checker but should
>> not be executed.  For such situations the ``typing`` module defines a
>> constant, ``TYPE_CHECKING``, that is considered ``True`` during type
>> checking but ``False`` at runtime.  Example::
>> 
>>  import typing
>> 
>>  if typing.TYPE_CHECKING:
>>      import expensive_mod
>> 
>>  def a_func(arg: expensive_mod.SomeClass) -> None:
>>      a_var: expensive_mod.SomeClass = arg
>>      ...
> 
> I don't know whether this is important, but for the record the current
> documentation shows expensive_mod.SomeClass quoted.

Yes, I wanted to explicitly illustrate that now you can use the annotations directly without quoting and it won't fail at import time.  If you tried to evaluate them with `typing.get_type_hints()` that would fail, just like trying to evaluate the string form today.


>> Backwards Compatibility
>> =======================
> [...]
>> Annotations that depend on locals at the time of the function/class
>> definition are now invalid.  Example::
> 
> As mentioned above, I think this is a bad loss of existing
> functionality.

I don't quite see it.

If somebody badly needs to store arbitrary data as annotations in generated functions, they still can directly write to `__annotations__`. In every other case, specifically in static typing (the overwhelming use case for annotations), this makes human readers happier and static analysis tools are indifferent.

In fact, if runtime type hint evaluation is necessary, then even if a library is already using `typing.get_type_hints()` (which it should!), so far I haven't seen much use of `globalns` and `localns` arguments to this function which suggests that the forward references we can support are already constrained to globals anyway.


>> In the presence of an annotation that cannot be resolved using the
>> current module's globals, a NameError is raised at compile time.
> 
> It is not clear what this means, or how it will work, especially given
> that the point of this is to delay evaluating annotations. How will the
> compiler know that an annotation cannot be resolved if it doesn't try to
> evaluate it?

The idea would be to validate the expressions after the entire module is compiled, something like what the flake8-pyi plugin is doing today for .pyi files. Guido pointed out that it's not trivial since the compiler doesn't keep a symbol table around. But I'd invest time in this since I agree with your point that we should raise errors as early as possible.


>> Rejected Ideas
>> ==============
>> 
>> Keep the ability to use local state when defining annotations
>> -------------------------------------------------------------
>> 
>> With postponed evaluation, this is impossible for function locals.
> 
> Impossible seems a bit strong. Can you elaborate?

Unless we stored the closure, it's not possible.


> [...]
>> This is brittle and doesn't even cover slots.  Requiring the use of
>> module-level names simplifies runtime evaluation and provides the
>> "one obvious way" to read annotations.  It's the equivalent of absolute
>> imports.
> 
> I hardly think that "simplifies runtime evaluation" is true. At the
> moment annotations are already evaluated. *Anything* that you have to do
> by hand (like call eval) cannot be simpler than "do nothing".

This section covers the rejected idea of allowing local access on classes specifically.  As I mentioned earlier, preserving local function scope access is not something we can do without bending ourselves backwards.  Constraining ourselves to only global scope simplifies runtime evaluation compared to that use case.  The alternative is not "do nothing".


> I don't think the analogy with absolute imports is even close to useful,
> and far from being "one obvious way", this is a surprising, non-obvious,
> seemingly-arbitrary restriction on annotations.

That's a bit harsh. I probably didn't voice my idea clearly enough here so let me elaborate. Annotations inside nested classes which are using local scope currently have to use the local names directly instead of using the qualified name. This has similar issues to relative imports:

class Menu(UIComponent):
    ...

class Restaurant:
    class Menu(Enum):
        SPAM = 1
        EGGS = 2

    def generate_menu_component(self) -> Menu:
        ...

It is impossible today to use the global "Menu" type in the annotation of the example method. This PEP is proposing to use qualified names in this case which disambiguates between the global "Menu" and "Restaurant.Menu". In this sense it felt similar to absolute imports to me.


> Given that for *six versions* of Python 3 annotations could be locals,
> there is nothing obvious about restricting them to globals.

I don't understand the argument of status quo. In those six versions, the first two were not usable in production and the following two only slowly started getting adoption. It wasn't until Python 3.5 that we started seeing significant adoption figures. This is incidentally also the first release with PEP 484.


Wow, this took quite some time to respond to, sorry it took so long! I had to do additional research and clarify some of my own understanding while doing this. It was very valuable, thanks again for your feedback!

- Ł
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 842 bytes
Desc: Message signed with OpenPGP
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20170913/a77fcf94/attachment.sig>