Guido, thanks for taking the time to write up a response! Daniel already covered the two main problems we’re hoping to solve with explicit aliases above.

 

To clarify some other potential points of confusion:

We’re not actually concerned about knowing what `C` is when running into `from m` import C`, since Pyre builds out an environment before type checking each function and has no trouble identifying `C` as a class or a type. You’re right that resolving imports isn’t a problem unique to type aliases. As Daniel mentioned, the only times it becomes ambiguous whether an assignment is an untyped global expression or if it is a type alias are (1) when using quoted forward references, or (2) when there’s been a “mistake” in the alias.

 

The first problem is pretty straightforward, but the second can start getting confusing because type with a mistake in it can still be a valid expression. For example, `x = List[some_invalid_type]` is a valid expression that evaluates to `Any` via the `typing.TypeAlias.__getitem__` method. It becomes impossible to soundly throw any errors on the definition of the alias itself without knowing the user’s intent for this assignment, even for the most common mistakes made when declaring aliases. We’ll wind up throwing redundant and confusing errors in various downstream parts of the program that don’t really identify what the source of the problem is – this gets especially hard to track down when aliases are composed of other aliases and you have to guess where the problem started, etc.

 

Re: the different syntactic approaches to type aliases, I don’t have too much context on the pros and cons of runtime effects vs. backwards compatibility, etc. Looks like the proposed syntax options so far include

 

 

Note: If `Any` is not actually intended to be acceptable as a valid type alias, that simplifies some of the complexity we were running into. However, I believe the remaining issues still present a case for making type aliases explicit. Thoughts?

 

 

From: Daniel Moisset <dfmoisset@gmail.com>
Date: Monday, July 29, 2019 at 3:15 AM
To: Guido van Rossum <guido@python.org>
Cc: Shannon Zhu <szhu@fb.com>, "typing-sig@python.org" <typing-sig@python.org>
Subject: [Typing-sig] Re: Explicit Type Aliases

 

I'm not the OP, but I'll add some comments on what I see are some problems to be solved by explicit type aliases

  1. Be able to have forward references (because type-aliases are evaluated at runtime, they don't currently get the benefit of __future__.anotations , so string quoting is still required in those).
  2. Improve error messages when mistakes are made in the RHS of a type alias. An example

MyAlias = MyGeneric(int)  # I actually meant MyGeneric[int] here. Now this looks like an expression

...

def f() -> MyAlias: ...

 

here I will get an error on the "MyAlias =" line saying:  «Need type annotation for 'MyAlias'» (what?)

And another error on the function definition saying: «Variable "MyAlias" is not valid as a type» (why?)

 

The first error could be much better, and the second removed (it's only a consequence of the first) if the intent fo creating an alias were explicit.

 

Based on these, I'm inclined towards some sort of annotation-only (no runtime effect) solution, i.e. "Indices: Alias(List[int])" 

 

I'm not sure if Łukasz Langa is here, but he mentioned similar concerns in his draft https://www.python.org/dev/peps/pep-0585/#moving-the-remaining-runtime-syntax-for-typing-related-functionality-to-annotations 

 

Speaking in much more general terms, for me most changes that separate annotations from runtime effects are good, because that's the boundary where a lot of the problems have happened (forward annotations, circular imports, having to add new dunder methods, performance issues on module loading time, ...)

 

On Mon, 29 Jul 2019 at 01:58, Guido van Rossum <guido@python.org> wrote:

Hi Shannon,

 

Sorry that it's so quiet here. The mypy team members have been alternatingly super busy with Dropbox stuff and on vacation (I'm just back and in a week I'm going again for another week; Ivan is on vacation until the end of next week; Jukka is on vacation intermittently for the next few weeks).

 

That said I'd like to provide some feedback on your proposal.

 

First, the `z: Any` situation looks like a bug or accidental feature to me. This is definitely meant (and works) as a variable declaration; that it also allows using `z` as a type seems wrong. I can't find any evidence in PEP 484 that this was intended; in mypy it's likely the accidental result of other design choices meant to shut up errors about `Any`.

 

Next, it would be good to be clear about the reason why you want type aliases to be explicitly marked as such. I *think* that it is so that a type checker can know whether something is a type alias even if it hasn't seen the definition of the type to which it is aliased, right? And this would be handy for parallelization of checking large collections of files. I think I might have head you explain this in person before, but I'm not sure if I can do the argument justice. For my own edification here's some thinking aloud about this.

 

- For simple aliases involving builtin types I don't think this is an issue. E.g. if a type checker encounters `T = int` it should have no problem knowing that `int` is a type and hence T is, too. Ditto for types defined in `typing` such as `List` or `Mapping`.

 

- If the type is defined explicitly earlier in the file it also shouldn't be an issue. E.g. if a type checker sees `class C: ...` and then later in the same file `T = C` (or `T = C[xxx]` if C is generic) it should be totally clear that T is a type.

 

- So it looks like this really is only an issue when we have two modules, m1 and m2, where in m1 we define `class C: ...`, and in m2 we have `from m1 import C` followed by `T = C` or `T = C[<something>]`. At this point we don't know whether T is a type or not until we have processed m1 to the point where we know that C is a class. Under your new proposal we'll know that T is a type (alias) as soon as we read `T = TypeAlias[<whatever>]` in m2, regardless of whether we have processed any of its imports.

 

- I guess there's an additional issue when the type checker cannot find the source code for module m1 -- in this case we will eventually come to the conclusion that C has type `Any`, and then T also has type `Any`. (Maybe this is the reason for the `z: Any` corner case -- we can't exclude the possibility that C is a type, hence we can't exclude that T is a type, but if it is, it must be `Any`.) But this seems a corner case and I doubt that it is part of your motivation.

 

- Now an open question (for me) is why it is important that we know whether T is a type or not. If the checker sees a use of a variable X in a type context (e.g. in an annotation or cast) it might as well assume that X is a type for the time being -- an error can come later once the definition of X is clearer. (Even if it is known that X is a type, there could still be errors, e.g. it could be a non-generic type in a generic position.)

 

- And doesn't the same reasoning apply to simply importing a class? If we see `from m1 import C` we don't know whether C is a type until we've processed m1 to the point where it's clear that C is a class (or a type alias). For a few seconds I thought it was about chains of imports, where `from m1 import T` might need to process m1 to see a type alias definiton of T, which might in turn depend on something that m1 imports from elsewhere, but that same reasoning would apply to chains of imports passing a class definition along.

 

- So I'm still stumped -- what is special about type aliases?

 

Finally I fear that type aliases are so pervasive that changing all of them to the explicit notation in any significant code base is going to be prohibitive -- even if you had a perfect tool to find type aliases (e.g. some custom code in mypy or Pyre) this would create a gigantic diff, causing large numbers of merge conflicts, blame noise, and some misfires that could cause hours of debugging or even production failures.

 

Last and (for now) least, I can't recall if there is a good reason to prefer `x: TypeAlias = int` over `x = TypeAlias[int]` (or even `x: TypeAlias[int]`?).

 

If you only have a little time for a response, please focus on answering the question "what problem is this solving", since knowing the problem may cause some creative minds to produce alternative solutions (or at least help us estimate the magnitude of the issue and perhaps provide preferences for one syntax or another).

 

--Guido

 

 

 

On Thu, Jul 18, 2019 at 2:10 PM Shannon Zhu <szhu@fb.com> wrote:

Hi everyone,

 

I’d like to continue the discussion from our last typing summit on explicit type aliases. A quick summary of the current state and the proposal --

 

Current state:

```

from typing import List

 

x = List[int]  # considered a type alias

y : Type[List[int]] = List[int]  # considered an expression

z : Any  # considered a type alias

 

z : x = []  # fine

z : y = []  # invalid type error

```

 

Proposal (with explicit aliases):

```

from typing import List, TypeAlias

 

x = TypeAlias[List[int]]    # considered a type alias

y = List[int]  # considered an expression

z : Any  # considered an expression

z = TypeAlias[Any]  # considered a type alias

 

reveal_type(x)  # Type[List[int]]

reveal_type(y)  # Any  (return type of __getitem__)

 

z : x  = []  # fine

z : y = []  # invalid type error

```

 

Some of the benefits we’re hoping to gain from explicit type aliases:

  • Clearly distinguish between an unannotated global assignment and a type alias, especially when parsing forward-referencing string annotations.
  • Avoid the confusing case of type aliases in which some part of the type is invalid or undefined. This would facilitate warnings on invalid types at the alias definition rather than later on when the alias is used.
  • Make valid and invalid types more intuitive to Python programmers by shifting from delineation of non-aliases with a meta annotation toward delineation of aliases with `TypeAlias`.
  • Remove special treatment of values annotated as `Any`, which currently break all type aliasing rules. This will also allow us to clean up some of the distinctions we maintain between unannotated values and explicit Anys.

 

Note: We also considered denoting the alias as an annotation (e.g. `x: TypeAlias = int`) as an alternative to the syntax above.

 

I’m interested to hear your thoughts and suggestions on making type aliasing explicit going forward – we’d like to start implementing a version of this in Pyre soon!

 

Shannon

_______________________________________________
Typing-sig mailing list -- typing-sig@python.org
To unsubscribe send an email to typing-sig-leave@python.org
https://mail.python.org/mailman3/lists/typing-sig.python.org/



--

--Guido van Rossum (python.org/~guido)

Pronouns: he/him/his (why is my pronoun here?)

_______________________________________________
Typing-sig mailing list -- typing-sig@python.org
To unsubscribe send an email to typing-sig-leave@python.org
https://mail.python.org/mailman3/lists/typing-sig.python.org/