Explicit Type Aliases
Hi everyone, I’d like to continue the discussion from our last typing summit on explicit type aliases. A quick summary of the current state and the proposal -- Current state: ``` from typing import List x = List[int] # considered a type alias y : Type[List[int]] = List[int] # considered an expression z : Any # considered a type alias z : x = [] # fine z : y = [] # invalid type error ``` Proposal (with explicit aliases): ``` from typing import List, TypeAlias x = TypeAlias[List[int]] # considered a type alias y = List[int] # considered an expression z : Any # considered an expression z = TypeAlias[Any] # considered a type alias reveal_type(x) # Type[List[int]] reveal_type(y) # Any (return type of __getitem__) z : x = [] # fine z : y = [] # invalid type error ``` Some of the benefits we’re hoping to gain from explicit type aliases: * Clearly distinguish between an unannotated global assignment and a type alias, especially when parsing forward-referencing string annotations. * Avoid the confusing case of type aliases in which some part of the type is invalid or undefined. This would facilitate warnings on invalid types at the alias definition rather than later on when the alias is used. * Make valid and invalid types more intuitive to Python programmers by shifting from delineation of non-aliases with a meta annotation toward delineation of aliases with `TypeAlias`. * Remove special treatment of values annotated as `Any`, which currently break all type aliasing rules. This will also allow us to clean up some of the distinctions we maintain between unannotated values and explicit Anys. Note: We also considered denoting the alias as an annotation (e.g. `x: TypeAlias = int`) as an alternative to the syntax above. I’m interested to hear your thoughts and suggestions on making type aliasing explicit going forward – we’d like to start implementing a version of this in Pyre soon! Shannon
Hi Shannon, Sorry that it's so quiet here. The mypy team members have been alternatingly super busy with Dropbox stuff and on vacation (I'm just back and in a week I'm going again for another week; Ivan is on vacation until the end of next week; Jukka is on vacation intermittently for the next few weeks). That said I'd like to provide some feedback on your proposal. First, the `z: Any` situation looks like a bug or accidental feature to me. This is definitely meant (and works) as a variable declaration; that it also allows using `z` as a type seems wrong. I can't find any evidence in PEP 484 that this was intended; in mypy it's likely the accidental result of other design choices meant to shut up errors about `Any`. Next, it would be good to be clear about the reason why you want type aliases to be explicitly marked as such. I *think* that it is so that a type checker can know whether something is a type alias even if it hasn't seen the definition of the type to which it is aliased, right? And this would be handy for parallelization of checking large collections of files. I think I might have head you explain this in person before, but I'm not sure if I can do the argument justice. For my own edification here's some thinking aloud about this. - For simple aliases involving builtin types I don't think this is an issue. E.g. if a type checker encounters `T = int` it should have no problem knowing that `int` is a type and hence T is, too. Ditto for types defined in `typing` such as `List` or `Mapping`. - If the type is defined explicitly earlier in the file it also shouldn't be an issue. E.g. if a type checker sees `class C: ...` and then later in the same file `T = C` (or `T = C[xxx]` if C is generic) it should be totally clear that T is a type. - So it looks like this really is only an issue when we have two modules, m1 and m2, where in m1 we define `class C: ...`, and in m2 we have `from m1 import C` followed by `T = C` or `T = C[<something>]`. At this point we don't know whether T is a type or not until we have processed m1 to the point where we know that C is a class. Under your new proposal we'll know that T is a type (alias) as soon as we read `T = TypeAlias[<whatever>]` in m2, regardless of whether we have processed any of its imports. - I guess there's an additional issue when the type checker cannot find the source code for module m1 -- in this case we will eventually come to the conclusion that C has type `Any`, and then T also has type `Any`. (Maybe this is the reason for the `z: Any` corner case -- we can't exclude the possibility that C is a type, hence we can't exclude that T is a type, but if it is, it must be `Any`.) But this seems a corner case and I doubt that it is part of your motivation. - Now an open question (for me) is why it is important that we know whether T is a type or not. If the checker sees a use of a variable X in a type context (e.g. in an annotation or cast) it might as well assume that X is a type for the time being -- an error can come later once the definition of X is clearer. (Even if it is known that X is a type, there could still be errors, e.g. it could be a non-generic type in a generic position.) - And doesn't the same reasoning apply to simply importing a class? If we see `from m1 import C` we don't know whether C is a type until we've processed m1 to the point where it's clear that C is a class (or a type alias). For a few seconds I thought it was about chains of imports, where `from m1 import T` might need to process m1 to see a type alias definiton of T, which might in turn depend on something that m1 imports from elsewhere, but that same reasoning would apply to chains of imports passing a class definition along. - So I'm still stumped -- what is special about type aliases? Finally I fear that type aliases are so pervasive that changing all of them to the explicit notation in any significant code base is going to be prohibitive -- even if you had a perfect tool to find type aliases (e.g. some custom code in mypy or Pyre) this would create a gigantic diff, causing large numbers of merge conflicts, blame noise, and some misfires that could cause hours of debugging or even production failures. Last and (for now) least, I can't recall if there is a good reason to prefer `x: TypeAlias = int` over `x = TypeAlias[int]` (or even `x: TypeAlias[int]`?). If you only have a little time for a response, please focus on answering the question "what problem is this solving", since knowing the problem may cause some creative minds to produce alternative solutions (or at least help us estimate the magnitude of the issue and perhaps provide preferences for one syntax or another). --Guido On Thu, Jul 18, 2019 at 2:10 PM Shannon Zhu <szhu@fb.com> wrote:
Hi everyone,
I’d like to continue the discussion from our last typing summit on explicit type aliases. A quick summary of the current state and the proposal --
*Current state:*
```
from typing import List
x = List[int] # considered a type alias
y : Type[List[int]] = List[int] # considered an expression
z : Any # considered a type alias
z : x = [] # fine
z : y = [] # invalid type error
```
*Proposal (with explicit aliases):*
```
from typing import List, TypeAlias
x = TypeAlias[List[int]] # considered a type alias
y = List[int] # considered an expression
z : Any # considered an expression
z = TypeAlias[Any] # considered a type alias
reveal_type(x) # Type[List[int]]
reveal_type(y) # Any (return type of __getitem__)
z : x = [] # fine
z : y = [] # invalid type error
```
Some of the benefits we’re hoping to gain from explicit type aliases:
- Clearly distinguish between an unannotated global assignment and a type alias, especially when parsing forward-referencing string annotations. - Avoid the confusing case of type aliases in which some part of the type is invalid or undefined. This would facilitate warnings on invalid types at the alias definition rather than later on when the alias is used. - Make valid and invalid types more intuitive to Python programmers by shifting from delineation of non-aliases with a meta annotation toward delineation of aliases with `TypeAlias`. - Remove special treatment of values annotated as `Any`, which currently break all type aliasing rules. This will also allow us to clean up some of the distinctions we maintain between unannotated values and explicit Anys.
Note: We also considered denoting the alias as an annotation (e.g. `x: TypeAlias = int`) as an alternative to the syntax above.
I’m interested to hear your thoughts and suggestions on making type aliasing explicit going forward – we’d like to start implementing a version of this in Pyre soon!
Shannon _______________________________________________ Typing-sig mailing list -- typing-sig@python.org To unsubscribe send an email to typing-sig-leave@python.org https://mail.python.org/mailman3/lists/typing-sig.python.org/
-- --Guido van Rossum (python.org/~guido) *Pronouns: he/him/his **(why is my pronoun here?)* <http://feministing.com/2015/02/03/how-using-they-as-a-singular-pronoun-can-change-the-world/>
I'm not the OP, but I'll add some comments on what I see are some problems to be solved by explicit type aliases 1. Be able to have forward references (because type-aliases are evaluated at runtime, they don't currently get the benefit of __future__.anotations , so string quoting is still required in those). 2. Improve error messages when mistakes are made in the RHS of a type alias. An example MyAlias = MyGeneric(int) # I actually meant MyGeneric[int] here. Now this looks like an expression ... def f() -> MyAlias: ... here I will get an error on the "MyAlias =" line saying: «Need type annotation for 'MyAlias'» (what?) And another error on the function definition saying: «Variable "MyAlias" is not valid as a type» (why?) The first error could be much better, and the second removed (it's only a consequence of the first) if the intent fo creating an alias were explicit. Based on these, I'm inclined towards some sort of annotation-only (no runtime effect) solution, i.e. "Indices: Alias(List[int])" I'm not sure if Łukasz Langa is here, but he mentioned similar concerns in his draft https://www.python.org/dev/peps/pep-0585/#moving-the-remaining-runtime-synta... Speaking in much more general terms, for me most changes that separate annotations from runtime effects are good, because that's the boundary where a lot of the problems have happened (forward annotations, circular imports, having to add new dunder methods, performance issues on module loading time, ...) On Mon, 29 Jul 2019 at 01:58, Guido van Rossum <guido@python.org> wrote:
Hi Shannon,
Sorry that it's so quiet here. The mypy team members have been alternatingly super busy with Dropbox stuff and on vacation (I'm just back and in a week I'm going again for another week; Ivan is on vacation until the end of next week; Jukka is on vacation intermittently for the next few weeks).
That said I'd like to provide some feedback on your proposal.
First, the `z: Any` situation looks like a bug or accidental feature to me. This is definitely meant (and works) as a variable declaration; that it also allows using `z` as a type seems wrong. I can't find any evidence in PEP 484 that this was intended; in mypy it's likely the accidental result of other design choices meant to shut up errors about `Any`.
Next, it would be good to be clear about the reason why you want type aliases to be explicitly marked as such. I *think* that it is so that a type checker can know whether something is a type alias even if it hasn't seen the definition of the type to which it is aliased, right? And this would be handy for parallelization of checking large collections of files. I think I might have head you explain this in person before, but I'm not sure if I can do the argument justice. For my own edification here's some thinking aloud about this.
- For simple aliases involving builtin types I don't think this is an issue. E.g. if a type checker encounters `T = int` it should have no problem knowing that `int` is a type and hence T is, too. Ditto for types defined in `typing` such as `List` or `Mapping`.
- If the type is defined explicitly earlier in the file it also shouldn't be an issue. E.g. if a type checker sees `class C: ...` and then later in the same file `T = C` (or `T = C[xxx]` if C is generic) it should be totally clear that T is a type.
- So it looks like this really is only an issue when we have two modules, m1 and m2, where in m1 we define `class C: ...`, and in m2 we have `from m1 import C` followed by `T = C` or `T = C[<something>]`. At this point we don't know whether T is a type or not until we have processed m1 to the point where we know that C is a class. Under your new proposal we'll know that T is a type (alias) as soon as we read `T = TypeAlias[<whatever>]` in m2, regardless of whether we have processed any of its imports.
- I guess there's an additional issue when the type checker cannot find the source code for module m1 -- in this case we will eventually come to the conclusion that C has type `Any`, and then T also has type `Any`. (Maybe this is the reason for the `z: Any` corner case -- we can't exclude the possibility that C is a type, hence we can't exclude that T is a type, but if it is, it must be `Any`.) But this seems a corner case and I doubt that it is part of your motivation.
- Now an open question (for me) is why it is important that we know whether T is a type or not. If the checker sees a use of a variable X in a type context (e.g. in an annotation or cast) it might as well assume that X is a type for the time being -- an error can come later once the definition of X is clearer. (Even if it is known that X is a type, there could still be errors, e.g. it could be a non-generic type in a generic position.)
- And doesn't the same reasoning apply to simply importing a class? If we see `from m1 import C` we don't know whether C is a type until we've processed m1 to the point where it's clear that C is a class (or a type alias). For a few seconds I thought it was about chains of imports, where `from m1 import T` might need to process m1 to see a type alias definiton of T, which might in turn depend on something that m1 imports from elsewhere, but that same reasoning would apply to chains of imports passing a class definition along.
- So I'm still stumped -- what is special about type aliases?
Finally I fear that type aliases are so pervasive that changing all of them to the explicit notation in any significant code base is going to be prohibitive -- even if you had a perfect tool to find type aliases (e.g. some custom code in mypy or Pyre) this would create a gigantic diff, causing large numbers of merge conflicts, blame noise, and some misfires that could cause hours of debugging or even production failures.
Last and (for now) least, I can't recall if there is a good reason to prefer `x: TypeAlias = int` over `x = TypeAlias[int]` (or even `x: TypeAlias[int]`?).
If you only have a little time for a response, please focus on answering the question "what problem is this solving", since knowing the problem may cause some creative minds to produce alternative solutions (or at least help us estimate the magnitude of the issue and perhaps provide preferences for one syntax or another).
--Guido
On Thu, Jul 18, 2019 at 2:10 PM Shannon Zhu <szhu@fb.com> wrote:
Hi everyone,
I’d like to continue the discussion from our last typing summit on explicit type aliases. A quick summary of the current state and the proposal --
*Current state:*
```
from typing import List
x = List[int] # considered a type alias
y : Type[List[int]] = List[int] # considered an expression
z : Any # considered a type alias
z : x = [] # fine
z : y = [] # invalid type error
```
*Proposal (with explicit aliases):*
```
from typing import List, TypeAlias
x = TypeAlias[List[int]] # considered a type alias
y = List[int] # considered an expression
z : Any # considered an expression
z = TypeAlias[Any] # considered a type alias
reveal_type(x) # Type[List[int]]
reveal_type(y) # Any (return type of __getitem__)
z : x = [] # fine
z : y = [] # invalid type error
```
Some of the benefits we’re hoping to gain from explicit type aliases:
- Clearly distinguish between an unannotated global assignment and a type alias, especially when parsing forward-referencing string annotations. - Avoid the confusing case of type aliases in which some part of the type is invalid or undefined. This would facilitate warnings on invalid types at the alias definition rather than later on when the alias is used. - Make valid and invalid types more intuitive to Python programmers by shifting from delineation of non-aliases with a meta annotation toward delineation of aliases with `TypeAlias`. - Remove special treatment of values annotated as `Any`, which currently break all type aliasing rules. This will also allow us to clean up some of the distinctions we maintain between unannotated values and explicit Anys.
Note: We also considered denoting the alias as an annotation (e.g. `x: TypeAlias = int`) as an alternative to the syntax above.
I’m interested to hear your thoughts and suggestions on making type aliasing explicit going forward – we’d like to start implementing a version of this in Pyre soon!
Shannon _______________________________________________ Typing-sig mailing list -- typing-sig@python.org To unsubscribe send an email to typing-sig-leave@python.org https://mail.python.org/mailman3/lists/typing-sig.python.org/
-- --Guido van Rossum (python.org/~guido) *Pronouns: he/him/his **(why is my pronoun here?)* <http://feministing.com/2015/02/03/how-using-they-as-a-singular-pronoun-can-change-the-world/> _______________________________________________ Typing-sig mailing list -- typing-sig@python.org To unsubscribe send an email to typing-sig-leave@python.org https://mail.python.org/mailman3/lists/typing-sig.python.org/
Guido, thanks for taking the time to write up a response! Daniel already covered the two main problems we’re hoping to solve with explicit aliases above. To clarify some other potential points of confusion: We’re not actually concerned about knowing what `C` is when running into `from m` import C`, since Pyre builds out an environment before type checking each function and has no trouble identifying `C` as a class or a type. You’re right that resolving imports isn’t a problem unique to type aliases. As Daniel mentioned, the only times it becomes ambiguous whether an assignment is an untyped global expression or if it is a type alias are (1) when using quoted forward references, or (2) when there’s been a “mistake” in the alias. The first problem is pretty straightforward, but the second can start getting confusing because type with a mistake in it can still be a valid expression. For example, `x = List[some_invalid_type]` is a valid expression that evaluates to `Any` via the `typing.TypeAlias.__getitem__` method. It becomes impossible to soundly throw any errors on the definition of the alias itself without knowing the user’s intent for this assignment, even for the most common mistakes made when declaring aliases. We’ll wind up throwing redundant and confusing errors in various downstream parts of the program that don’t really identify what the source of the problem is – this gets especially hard to track down when aliases are composed of other aliases and you have to guess where the problem started, etc. Re: the different syntactic approaches to type aliases, I don’t have too much context on the pros and cons of runtime effects vs. backwards compatibility, etc. Looks like the proposed syntax options so far include * MyAlias = TypeAlias[int] * MyAlias: TypeAlias[int] * MyAlias: TypeAlias = int Note: If `Any` is not actually intended to be acceptable as a valid type alias, that simplifies some of the complexity we were running into. However, I believe the remaining issues still present a case for making type aliases explicit. Thoughts? From: Daniel Moisset <dfmoisset@gmail.com> Date: Monday, July 29, 2019 at 3:15 AM To: Guido van Rossum <guido@python.org> Cc: Shannon Zhu <szhu@fb.com>, "typing-sig@python.org" <typing-sig@python.org> Subject: [Typing-sig] Re: Explicit Type Aliases I'm not the OP, but I'll add some comments on what I see are some problems to be solved by explicit type aliases 1. Be able to have forward references (because type-aliases are evaluated at runtime, they don't currently get the benefit of __future__.anotations , so string quoting is still required in those). 2. Improve error messages when mistakes are made in the RHS of a type alias. An example MyAlias = MyGeneric(int) # I actually meant MyGeneric[int] here. Now this looks like an expression ... def f() -> MyAlias: ... here I will get an error on the "MyAlias =" line saying: «Need type annotation for 'MyAlias'» (what?) And another error on the function definition saying: «Variable "MyAlias" is not valid as a type» (why?) The first error could be much better, and the second removed (it's only a consequence of the first) if the intent fo creating an alias were explicit. Based on these, I'm inclined towards some sort of annotation-only (no runtime effect) solution, i.e. "Indices: Alias(List[int])" I'm not sure if Łukasz Langa is here, but he mentioned similar concerns in his draft https://www.python.org/dev/peps/pep-0585/#moving-the-remaining-runtime-syntax-for-typing-related-functionality-to-annotations<https://urldefense.proofpoint.com/v2/url?u=https-3A__www.python.org_dev_peps_pep-2D0585_-23moving-2Dthe-2Dremaining-2Druntime-2Dsyntax-2Dfor-2Dtyping-2Drelated-2Dfunctionality-2Dto-2Dannotations&d=DwMFaQ&c=5VD0RTtNlTh3ycd41b3MUw&r=j6-YRB8UY9YIqi5vSSyJ-A&m=_vNiOsqcc4GV-6IijISXsfTtzzJJRYnZsvNgSP4YM2g&s=rMR2DVsMikPc6vCo0bhHa8kodTPfYTe3Yiz3cHgBJDk&e=> Speaking in much more general terms, for me most changes that separate annotations from runtime effects are good, because that's the boundary where a lot of the problems have happened (forward annotations, circular imports, having to add new dunder methods, performance issues on module loading time, ...) On Mon, 29 Jul 2019 at 01:58, Guido van Rossum <guido@python.org<mailto:guido@python.org>> wrote: Hi Shannon, Sorry that it's so quiet here. The mypy team members have been alternatingly super busy with Dropbox stuff and on vacation (I'm just back and in a week I'm going again for another week; Ivan is on vacation until the end of next week; Jukka is on vacation intermittently for the next few weeks). That said I'd like to provide some feedback on your proposal. First, the `z: Any` situation looks like a bug or accidental feature to me. This is definitely meant (and works) as a variable declaration; that it also allows using `z` as a type seems wrong. I can't find any evidence in PEP 484 that this was intended; in mypy it's likely the accidental result of other design choices meant to shut up errors about `Any`. Next, it would be good to be clear about the reason why you want type aliases to be explicitly marked as such. I *think* that it is so that a type checker can know whether something is a type alias even if it hasn't seen the definition of the type to which it is aliased, right? And this would be handy for parallelization of checking large collections of files. I think I might have head you explain this in person before, but I'm not sure if I can do the argument justice. For my own edification here's some thinking aloud about this. - For simple aliases involving builtin types I don't think this is an issue. E.g. if a type checker encounters `T = int` it should have no problem knowing that `int` is a type and hence T is, too. Ditto for types defined in `typing` such as `List` or `Mapping`. - If the type is defined explicitly earlier in the file it also shouldn't be an issue. E.g. if a type checker sees `class C: ...` and then later in the same file `T = C` (or `T = C[xxx]` if C is generic) it should be totally clear that T is a type. - So it looks like this really is only an issue when we have two modules, m1 and m2, where in m1 we define `class C: ...`, and in m2 we have `from m1 import C` followed by `T = C` or `T = C[<something>]`. At this point we don't know whether T is a type or not until we have processed m1 to the point where we know that C is a class. Under your new proposal we'll know that T is a type (alias) as soon as we read `T = TypeAlias[<whatever>]` in m2, regardless of whether we have processed any of its imports. - I guess there's an additional issue when the type checker cannot find the source code for module m1 -- in this case we will eventually come to the conclusion that C has type `Any`, and then T also has type `Any`. (Maybe this is the reason for the `z: Any` corner case -- we can't exclude the possibility that C is a type, hence we can't exclude that T is a type, but if it is, it must be `Any`.) But this seems a corner case and I doubt that it is part of your motivation. - Now an open question (for me) is why it is important that we know whether T is a type or not. If the checker sees a use of a variable X in a type context (e.g. in an annotation or cast) it might as well assume that X is a type for the time being -- an error can come later once the definition of X is clearer. (Even if it is known that X is a type, there could still be errors, e.g. it could be a non-generic type in a generic position.) - And doesn't the same reasoning apply to simply importing a class? If we see `from m1 import C` we don't know whether C is a type until we've processed m1 to the point where it's clear that C is a class (or a type alias). For a few seconds I thought it was about chains of imports, where `from m1 import T` might need to process m1 to see a type alias definiton of T, which might in turn depend on something that m1 imports from elsewhere, but that same reasoning would apply to chains of imports passing a class definition along. - So I'm still stumped -- what is special about type aliases? Finally I fear that type aliases are so pervasive that changing all of them to the explicit notation in any significant code base is going to be prohibitive -- even if you had a perfect tool to find type aliases (e.g. some custom code in mypy or Pyre) this would create a gigantic diff, causing large numbers of merge conflicts, blame noise, and some misfires that could cause hours of debugging or even production failures. Last and (for now) least, I can't recall if there is a good reason to prefer `x: TypeAlias = int` over `x = TypeAlias[int]` (or even `x: TypeAlias[int]`?). If you only have a little time for a response, please focus on answering the question "what problem is this solving", since knowing the problem may cause some creative minds to produce alternative solutions (or at least help us estimate the magnitude of the issue and perhaps provide preferences for one syntax or another). --Guido On Thu, Jul 18, 2019 at 2:10 PM Shannon Zhu <szhu@fb.com<mailto:szhu@fb.com>> wrote: Hi everyone, I’d like to continue the discussion from our last typing summit on explicit type aliases. A quick summary of the current state and the proposal -- Current state: ``` from typing import List x = List[int] # considered a type alias y : Type[List[int]] = List[int] # considered an expression z : Any # considered a type alias z : x = [] # fine z : y = [] # invalid type error ``` Proposal (with explicit aliases): ``` from typing import List, TypeAlias x = TypeAlias[List[int]] # considered a type alias y = List[int] # considered an expression z : Any # considered an expression z = TypeAlias[Any] # considered a type alias reveal_type(x) # Type[List[int]] reveal_type(y) # Any (return type of __getitem__) z : x = [] # fine z : y = [] # invalid type error ``` Some of the benefits we’re hoping to gain from explicit type aliases: * Clearly distinguish between an unannotated global assignment and a type alias, especially when parsing forward-referencing string annotations. * Avoid the confusing case of type aliases in which some part of the type is invalid or undefined. This would facilitate warnings on invalid types at the alias definition rather than later on when the alias is used. * Make valid and invalid types more intuitive to Python programmers by shifting from delineation of non-aliases with a meta annotation toward delineation of aliases with `TypeAlias`. * Remove special treatment of values annotated as `Any`, which currently break all type aliasing rules. This will also allow us to clean up some of the distinctions we maintain between unannotated values and explicit Anys. Note: We also considered denoting the alias as an annotation (e.g. `x: TypeAlias = int`) as an alternative to the syntax above. I’m interested to hear your thoughts and suggestions on making type aliasing explicit going forward – we’d like to start implementing a version of this in Pyre soon! Shannon _______________________________________________ Typing-sig mailing list -- typing-sig@python.org<mailto:typing-sig@python.org> To unsubscribe send an email to typing-sig-leave@python.org<mailto:typing-sig-leave@python.org> https://mail.python.org/mailman3/lists/typing-sig.python.org/<https://urldefense.proofpoint.com/v2/url?u=https-3A__mail.python.org_mailman3_lists_typing-2Dsig.python.org_&d=DwMFaQ&c=5VD0RTtNlTh3ycd41b3MUw&r=j6-YRB8UY9YIqi5vSSyJ-A&m=_vNiOsqcc4GV-6IijISXsfTtzzJJRYnZsvNgSP4YM2g&s=trUSLH7QzQTtMsS1Y1zVB1FsWwK-gQARqUT3SRmO7-k&e=> -- --Guido van Rossum (python.org/~guido<https://urldefense.proofpoint.com/v2/url?u=http-3A__python.org_-7Eguido&d=DwMFaQ&c=5VD0RTtNlTh3ycd41b3MUw&r=j6-YRB8UY9YIqi5vSSyJ-A&m=_vNiOsqcc4GV-6IijISXsfTtzzJJRYnZsvNgSP4YM2g&s=IjZSTqW4t_x6om1w14BB2OQ_Dk9U-VPpLcbyu7nEhNc&e=>) Pronouns: he/him/his (why is my pronoun here?)<https://urldefense.proofpoint.com/v2/url?u=http-3A__feministing.com_2015_02_03_how-2Dusing-2Dthey-2Das-2Da-2Dsingular-2Dpronoun-2Dcan-2Dchange-2Dthe-2Dworld_&d=DwMFaQ&c=5VD0RTtNlTh3ycd41b3MUw&r=j6-YRB8UY9YIqi5vSSyJ-A&m=_vNiOsqcc4GV-6IijISXsfTtzzJJRYnZsvNgSP4YM2g&s=2WXG-ArskpJawndW02FrDICehCBQ0rKZ2qwyqbSjrQo&e=> _______________________________________________ Typing-sig mailing list -- typing-sig@python.org<mailto:typing-sig@python.org> To unsubscribe send an email to typing-sig-leave@python.org<mailto:typing-sig-leave@python.org> https://mail.python.org/mailman3/lists/typing-sig.python.org/<https://urldefense.proofpoint.com/v2/url?u=https-3A__mail.python.org_mailman3_lists_typing-2Dsig.python.org_&d=DwMFaQ&c=5VD0RTtNlTh3ycd41b3MUw&r=j6-YRB8UY9YIqi5vSSyJ-A&m=_vNiOsqcc4GV-6IijISXsfTtzzJJRYnZsvNgSP4YM2g&s=trUSLH7QzQTtMsS1Y1zVB1FsWwK-gQARqUT3SRmO7-k&e=>
Hi Shannon, Perhaps we can compromise and add an *optional* new syntax for type aliases that allows forward references? You would have to support both styles (old and new) but you could campaign with your users to switch to the new style and you could offer forward references in aliases and better diagnostics as a carrot (and maybe use a linter as a stick :-). An optional new way of doing things is a much easier sell to people who already have millions of lines of annotated code than a required change. In terms of syntax, here are my thoughts: MyAlias = TypeAlias[int] This is okay, it gets the job done. MyAlias: TypeAlias[int] This is my least favorite. Pro: with PEP 563 (from __future__ import annotations) this lets you write forward references without quotes. Con: it looks too much like an uninitialized variable. MyAlias: TypeAlias = int This is nice because it's a simple transformation from old style to new style: just like annotating a variable with a type, here we are annotating an alias (a variable at the meta-level) with a type (a meta-type). That makes it my favorite, with the first one (MyAlias = TypeAlias[int]) as the runner-up. --Guido On Mon, Jul 29, 2019 at 10:46 AM Shannon Zhu <szhu@fb.com> wrote:
Guido, thanks for taking the time to write up a response! Daniel already covered the two main problems we’re hoping to solve with explicit aliases above.
To clarify some other potential points of confusion:
We’re not actually concerned about knowing what `C` is when running into `from m` import C`, since Pyre builds out an environment before type checking each function and has no trouble identifying `C` as a class or a type. You’re right that resolving imports isn’t a problem unique to type aliases. As Daniel mentioned, the only times it becomes ambiguous whether an assignment is an untyped global expression or if it is a type alias are (1) when using quoted forward references, or (2) when there’s been a “mistake” in the alias.
The first problem is pretty straightforward, but the second can start getting confusing because type with a mistake in it can still be a valid expression. For example, `x = List[some_invalid_type]` is a valid expression that evaluates to `Any` via the `typing.TypeAlias.__getitem__` method. It becomes impossible to soundly throw any errors on the definition of the alias itself without knowing the user’s intent for this assignment, even for the most common mistakes made when declaring aliases. We’ll wind up throwing redundant and confusing errors in various downstream parts of the program that don’t really identify what the source of the problem is – this gets especially hard to track down when aliases are composed of other aliases and you have to guess where the problem started, etc.
Re: the different syntactic approaches to type aliases, I don’t have too much context on the pros and cons of runtime effects vs. backwards compatibility, etc. Looks like the proposed syntax options so far include
- MyAlias = TypeAlias[int] - MyAlias: TypeAlias[int] - MyAlias: TypeAlias = int
Note: If `Any` is not actually intended to be acceptable as a valid type alias, that simplifies some of the complexity we were running into. However, I believe the remaining issues still present a case for making type aliases explicit. Thoughts?
*From: *Daniel Moisset <dfmoisset@gmail.com> *Date: *Monday, July 29, 2019 at 3:15 AM *To: *Guido van Rossum <guido@python.org> *Cc: *Shannon Zhu <szhu@fb.com>, "typing-sig@python.org" < typing-sig@python.org> *Subject: *[Typing-sig] Re: Explicit Type Aliases
I'm not the OP, but I'll add some comments on what I see are some problems to be solved by explicit type aliases
1. Be able to have forward references (because type-aliases are evaluated at runtime, they don't currently get the benefit of __future__.anotations , so string quoting is still required in those). 2. Improve error messages when mistakes are made in the RHS of a type alias. An example
MyAlias = MyGeneric(int) # I actually meant MyGeneric[int] here. Now this looks like an expression
...
def f() -> MyAlias: ...
here I will get an error on the "MyAlias =" line saying: «Need type annotation for 'MyAlias'» (what?)
And another error on the function definition saying: «Variable "MyAlias" is not valid as a type» (why?)
The first error could be much better, and the second removed (it's only a consequence of the first) if the intent fo creating an alias were explicit.
Based on these, I'm inclined towards some sort of annotation-only (no runtime effect) solution, i.e. "Indices: Alias(List[int])"
I'm not sure if Łukasz Langa is here, but he mentioned similar concerns in his draft https://www.python.org/dev/peps/pep-0585/#moving-the-remaining-runtime-synta... <https://urldefense.proofpoint.com/v2/url?u=https-3A__www.python.org_dev_peps_pep-2D0585_-23moving-2Dthe-2Dremaining-2Druntime-2Dsyntax-2Dfor-2Dtyping-2Drelated-2Dfunctionality-2Dto-2Dannotations&d=DwMFaQ&c=5VD0RTtNlTh3ycd41b3MUw&r=j6-YRB8UY9YIqi5vSSyJ-A&m=_vNiOsqcc4GV-6IijISXsfTtzzJJRYnZsvNgSP4YM2g&s=rMR2DVsMikPc6vCo0bhHa8kodTPfYTe3Yiz3cHgBJDk&e=>
Speaking in much more general terms, for me most changes that separate annotations from runtime effects are good, because that's the boundary where a lot of the problems have happened (forward annotations, circular imports, having to add new dunder methods, performance issues on module loading time, ...)
On Mon, 29 Jul 2019 at 01:58, Guido van Rossum <guido@python.org> wrote:
Hi Shannon,
Sorry that it's so quiet here. The mypy team members have been alternatingly super busy with Dropbox stuff and on vacation (I'm just back and in a week I'm going again for another week; Ivan is on vacation until the end of next week; Jukka is on vacation intermittently for the next few weeks).
That said I'd like to provide some feedback on your proposal.
First, the `z: Any` situation looks like a bug or accidental feature to me. This is definitely meant (and works) as a variable declaration; that it also allows using `z` as a type seems wrong. I can't find any evidence in PEP 484 that this was intended; in mypy it's likely the accidental result of other design choices meant to shut up errors about `Any`.
Next, it would be good to be clear about the reason why you want type aliases to be explicitly marked as such. I *think* that it is so that a type checker can know whether something is a type alias even if it hasn't seen the definition of the type to which it is aliased, right? And this would be handy for parallelization of checking large collections of files. I think I might have head you explain this in person before, but I'm not sure if I can do the argument justice. For my own edification here's some thinking aloud about this.
- For simple aliases involving builtin types I don't think this is an issue. E.g. if a type checker encounters `T = int` it should have no problem knowing that `int` is a type and hence T is, too. Ditto for types defined in `typing` such as `List` or `Mapping`.
- If the type is defined explicitly earlier in the file it also shouldn't be an issue. E.g. if a type checker sees `class C: ...` and then later in the same file `T = C` (or `T = C[xxx]` if C is generic) it should be totally clear that T is a type.
- So it looks like this really is only an issue when we have two modules, m1 and m2, where in m1 we define `class C: ...`, and in m2 we have `from m1 import C` followed by `T = C` or `T = C[<something>]`. At this point we don't know whether T is a type or not until we have processed m1 to the point where we know that C is a class. Under your new proposal we'll know that T is a type (alias) as soon as we read `T = TypeAlias[<whatever>]` in m2, regardless of whether we have processed any of its imports.
- I guess there's an additional issue when the type checker cannot find the source code for module m1 -- in this case we will eventually come to the conclusion that C has type `Any`, and then T also has type `Any`. (Maybe this is the reason for the `z: Any` corner case -- we can't exclude the possibility that C is a type, hence we can't exclude that T is a type, but if it is, it must be `Any`.) But this seems a corner case and I doubt that it is part of your motivation.
- Now an open question (for me) is why it is important that we know whether T is a type or not. If the checker sees a use of a variable X in a type context (e.g. in an annotation or cast) it might as well assume that X is a type for the time being -- an error can come later once the definition of X is clearer. (Even if it is known that X is a type, there could still be errors, e.g. it could be a non-generic type in a generic position.)
- And doesn't the same reasoning apply to simply importing a class? If we see `from m1 import C` we don't know whether C is a type until we've processed m1 to the point where it's clear that C is a class (or a type alias). For a few seconds I thought it was about chains of imports, where `from m1 import T` might need to process m1 to see a type alias definiton of T, which might in turn depend on something that m1 imports from elsewhere, but that same reasoning would apply to chains of imports passing a class definition along.
- So I'm still stumped -- what is special about type aliases?
Finally I fear that type aliases are so pervasive that changing all of them to the explicit notation in any significant code base is going to be prohibitive -- even if you had a perfect tool to find type aliases (e.g. some custom code in mypy or Pyre) this would create a gigantic diff, causing large numbers of merge conflicts, blame noise, and some misfires that could cause hours of debugging or even production failures.
Last and (for now) least, I can't recall if there is a good reason to prefer `x: TypeAlias = int` over `x = TypeAlias[int]` (or even `x: TypeAlias[int]`?).
If you only have a little time for a response, please focus on answering the question "what problem is this solving", since knowing the problem may cause some creative minds to produce alternative solutions (or at least help us estimate the magnitude of the issue and perhaps provide preferences for one syntax or another).
--Guido
On Thu, Jul 18, 2019 at 2:10 PM Shannon Zhu <szhu@fb.com> wrote:
Hi everyone,
I’d like to continue the discussion from our last typing summit on explicit type aliases. A quick summary of the current state and the proposal --
*Current state:*
```
from typing import List
x = List[int] # considered a type alias
y : Type[List[int]] = List[int] # considered an expression
z : Any # considered a type alias
z : x = [] # fine
z : y = [] # invalid type error
```
*Proposal (with explicit aliases):*
```
from typing import List, TypeAlias
x = TypeAlias[List[int]] # considered a type alias
y = List[int] # considered an expression
z : Any # considered an expression
z = TypeAlias[Any] # considered a type alias
reveal_type(x) # Type[List[int]]
reveal_type(y) # Any (return type of __getitem__)
z : x = [] # fine
z : y = [] # invalid type error
```
Some of the benefits we’re hoping to gain from explicit type aliases:
- Clearly distinguish between an unannotated global assignment and a type alias, especially when parsing forward-referencing string annotations. - Avoid the confusing case of type aliases in which some part of the type is invalid or undefined. This would facilitate warnings on invalid types at the alias definition rather than later on when the alias is used. - Make valid and invalid types more intuitive to Python programmers by shifting from delineation of non-aliases with a meta annotation toward delineation of aliases with `TypeAlias`. - Remove special treatment of values annotated as `Any`, which currently break all type aliasing rules. This will also allow us to clean up some of the distinctions we maintain between unannotated values and explicit Anys.
Note: We also considered denoting the alias as an annotation (e.g. `x: TypeAlias = int`) as an alternative to the syntax above.
I’m interested to hear your thoughts and suggestions on making type aliasing explicit going forward – we’d like to start implementing a version of this in Pyre soon!
Shannon
_______________________________________________ Typing-sig mailing list -- typing-sig@python.org To unsubscribe send an email to typing-sig-leave@python.org https://mail.python.org/mailman3/lists/typing-sig.python.org/ <https://urldefense.proofpoint.com/v2/url?u=https-3A__mail.python.org_mailman3_lists_typing-2Dsig.python.org_&d=DwMFaQ&c=5VD0RTtNlTh3ycd41b3MUw&r=j6-YRB8UY9YIqi5vSSyJ-A&m=_vNiOsqcc4GV-6IijISXsfTtzzJJRYnZsvNgSP4YM2g&s=trUSLH7QzQTtMsS1Y1zVB1FsWwK-gQARqUT3SRmO7-k&e=>
--
--Guido van Rossum (python.org/~guido <https://urldefense.proofpoint.com/v2/url?u=http-3A__python.org_-7Eguido&d=DwMFaQ&c=5VD0RTtNlTh3ycd41b3MUw&r=j6-YRB8UY9YIqi5vSSyJ-A&m=_vNiOsqcc4GV-6IijISXsfTtzzJJRYnZsvNgSP4YM2g&s=IjZSTqW4t_x6om1w14BB2OQ_Dk9U-VPpLcbyu7nEhNc&e=> )
*Pronouns: he/him/his **(why is my pronoun here?)* <https://urldefense.proofpoint.com/v2/url?u=http-3A__feministing.com_2015_02_03_how-2Dusing-2Dthey-2Das-2Da-2Dsingular-2Dpronoun-2Dcan-2Dchange-2Dthe-2Dworld_&d=DwMFaQ&c=5VD0RTtNlTh3ycd41b3MUw&r=j6-YRB8UY9YIqi5vSSyJ-A&m=_vNiOsqcc4GV-6IijISXsfTtzzJJRYnZsvNgSP4YM2g&s=2WXG-ArskpJawndW02FrDICehCBQ0rKZ2qwyqbSjrQo&e=>
_______________________________________________ Typing-sig mailing list -- typing-sig@python.org To unsubscribe send an email to typing-sig-leave@python.org https://mail.python.org/mailman3/lists/typing-sig.python.org/ <https://urldefense.proofpoint.com/v2/url?u=https-3A__mail.python.org_mailman3_lists_typing-2Dsig.python.org_&d=DwMFaQ&c=5VD0RTtNlTh3ycd41b3MUw&r=j6-YRB8UY9YIqi5vSSyJ-A&m=_vNiOsqcc4GV-6IijISXsfTtzzJJRYnZsvNgSP4YM2g&s=trUSLH7QzQTtMsS1Y1zVB1FsWwK-gQARqUT3SRmO7-k&e=>
-- --Guido van Rossum (python.org/~guido) *Pronouns: he/him/his **(why is my pronoun here?)* <http://feministing.com/2015/02/03/how-using-they-as-a-singular-pronoun-can-change-the-world/>
MyAlias: TypeAlias = int
Looks like a typed global from https://www.python.org/dev/peps/pep-0526/#global-and-local-variable-annotati... Which has the upside in this example "MyAlias is int" but I wonder if there's a downside somewhere, for example, presently this statement is evaluated at runtime, and ought to fail should `int` not be defined?
On Tue, Jul 30, 2019 at 6:23 PM Dima Tisnek <dimaqq@gmail.com> wrote:
MyAlias: TypeAlias = int
Looks like a typed global from
https://www.python.org/dev/peps/pep-0526/#global-and-local-variable-annotati...
Which has the upside in this example "MyAlias is int" but I wonder if there's a downside somewhere, for example, presently this statement is evaluated at runtime, and ought to fail should `int` not be defined?
Yeah, that's intentional. If you need to reference a type that's not yet defined (but will be later in the file) you put it in quotes, like any other forward reference. It's the same for `MyAlias = TypeAlias[int]`. The only form that doesn't have this is `MyAlias: TypeAlias[int]` but that has the opposite problem -- the alias itself is not defined at runtime and that means it cannot be used in other forms (e.g. TypeDef and base classes). This question is definitely not settled! -- --Guido van Rossum (python.org/~guido) *Pronouns: he/him/his **(why is my pronoun here?)* <http://feministing.com/2015/02/03/how-using-they-as-a-singular-pronoun-can-change-the-world/>
On Tue, 30 Jul 2019 at 17:44, Guido van Rossum <guido@python.org> wrote:
[...] MyAlias: TypeAlias = int This is nice because it's a simple transformation from old style to new style: just like annotating a variable with a type, here we are annotating an alias (a variable at the meta-level) with a type (a meta-type). That makes it my favorite, with the first one (MyAlias = TypeAlias[int]) as the runner-up.
I don't have any preference between x: TypeAlias = int and x = TypeAlias[int]. I like the general idea of allowing the explicit form in parallel to the existing implicit syntax. The problem that currently exists is that there are few corner cases where mypy decides between a variable of type Type[C] and an alias to C (moreover the rules are still undocumented, see https://github.com/python/mypy/issues/3494). For example: class C: ... x = C # this is a type alias from mypy point of view class D: x = C # this is a variable with type Type[C] This default behaviour is kind of a heuristics that matches most typical use in each case, but I think it would be nice to have a simple way to be explicit either way: x: TypeAlias = C y: Type[C] = C -- Ivan
**Enhancement Proposal** I propose adding `TypeAlias[T]` as a valid type. This is *in addition to* the plain `typeAlias` in PEP 613. It can be used as `MyType: TypeAlias[T]` (no assignment) instead of `MyType: TypeAlias = T` or `MyType: TypeAlias = "T"` This was a rejected alternative syntax in PEP 613, saying that it looks like an uninitialized variable, as though "alternative" meant "instead of `MyType: TypeAlias = T`. I am saying that **both** forms are allowed. And with my proposal, `MyType` *is* in fact an unitialized variable. I further propose that `TypeAlias` should be considered as `Final`. The program should have no other assignments to the variable in the assignment form, or no assignments at all in the non-assignment form. The advantages are: 1. It is faster, as there is no assignment at runtime. Only the entry in __annotations__["MyType"]. 2. `T` can be a forward reference, without quotes, when using future annotations import. 3. It could be used in a function scope, possibly. I don't understand why `MyType: TypeAlias = T` was not allowed in a function. The objection might possibly not apply to `MyType: TypeAlias[T]`. Additional requirements for a type checker: The type checker treats a `MyType: TypeAlias[T]` statement as exactly equivalent to `MyType: TypeAlias = "T"`. That is, `MyType` is an alias for the type `"T"`.
I propose allowing `MyType: TypeAlias[T]` *in addition to* the existing `MyType: TypeAlias = T` or `MyType: TypeAlias = "T"` I understand that this is listed under Rejected Ideas, but I assume this was rejected as a syntax instead of the PEP syntax. I am suggesting that *both* forms be allowed. It will be faster at runtime because `"MyType"` will not be added to the current namespace. It will be stored in `__annotations__` in the current namespace in either case. It allows `T` to be a forward reference, when using future annotations import. I don't understand why TypeAlias cannot be used in a function scope. If it has to do with the value assigned to the alias variable, this would not apply with the new form, because there is no assignment. The type alias is just an unitialized variable. A type checker will treat the non-assignment form as equivalent to the assignment form. That is, `MyType` is considered to be an alias for `"T"` or `T`. The name `MyType` should be able to be imported to another module (but with a forward reference evaluated in the current module, not the importing module), or used as a type or part of a type in other contexts. To avoid the objection that it "looks like an uninitialized variable," I say that it *is* in fact an uninitialized variable syntactically, but semantically it is a type alias. The docs should make this clear. To avoid ambiguities with further assignments to `MyType`, the type checker should consider `TypeAlias` as Final. Thus, for `MyType` in a given scope, there will be exactly one of the above forms of alias declaration. When a type checker examines the name `MyType` it should ignore its being uninitialized when the type is a `TypeAlias[T]` and evaluate it as `T`.
The Python type system has advanced in many ways since 2019 when this thread was created. PEP 695 (currently in draft form) proposes a new way to declare generic type aliases. It also proposes to deprecate the older `TypeAlias` annotation, which would no longer have any utility. Here's the link to the draft PEP 695: https://peps.python.org/pep-0695/ Here's the link to the typing-sig discussion: https://mail.python.org/archives/list/typing-sig@python.org/thread/BB2BGYJY2... Feedback and comments are welcome. -- Eric Traut Contributor to pyright & pylance Microsoft
participants (7)
-
Daniel Moisset -
Dima Tisnek -
Eric Traut -
Guido van Rossum -
Ivan Levkivskyi -
m＠rolle.name -
Shannon Zhu