Defining special forms in terms of `Annotated`
`Annotated` [1] was introduced to standardize annotations (beyond just types) and make it possible for static analyses to be written more independently of each other. However I see new type annotations [2] [3] that don't use the `Annotated` mechanism and so would need to be handled as special cases by anyone writing a static analysis, as is currently the case with Final, ClassVar, NewType, Any [4]. We should try to keep special forms to a minimum, if possible, by defining/rewriting them in terms of `Annotated`. For example: ``` Final[T] := Annotated[T, Final] ClassVar[T] := Annotated[T, ClassVar] Any := Annotated[object, AllowEverything] # that's more or less what `Any` means (UserId = NewType('UserId', int)) := (UserId: Annotated[..., NewType('UserId')] = int) TypeAlias := Annotated[..., TypeAlias] Required[T] := Annotated[T, Required] ``` Shortcuts for common patterns can still be provided (using `Annotated` + generics), but the goal is that consumers of annotations should only have to deal with `Annotated[<type>, <list of other annotations>]` instead of `Some[Nested[Form[<type>]]]`. [1] https://python.org/dev/peps/pep-0593 [2] https://python.org/dev/peps/pep-0613 [3] https://python.org/dev/peps/pep-0655 [4] While going through the docs to make this list I found `NoReturn` as well, which should be defined as `Union[]` (aka "empty" or "bottom"), but that's not related to `Annotated`.
I'm a bit lukewarm towards this proposal. My read of PEP 593 is that `Annotated` is a tool for the runtime, for downstream static analyses, and for individual type checkers to experiment on new features. As a result, interpretation of those annotations is never intended to be standardized -- a type checker that simply drops all annotations should still be functional and standard-conforming. Assigning special meanings to `Annotated` in the standard would immediately break this assumption.
We should try to keep special forms to a minimum, if possible, by defining/rewriting them in terms of `Annotated`
I might be missing something -- is there a more concrete value proposal, beyond just the (arguably subjective) belief that it's cleaner to "keep special forms to a minimum"? From the perspective of the type checker, things like `Final`, `NewType`, `TypeAlias`, etc. has to be special-cased anyway in the backend regardless of how one defines them (since they indeed represent different concepts from user-defined generic classes). So this is at most a cosmetic addon which, although not too costly to implement, brings little benefit to the table. That said, perhaps the story is different at runtime? If this kind of normalization/cleanup is really desirable, I'd prefer we rely on something that behaves in the same way as `Annotated` but has a different name, just to keep the pre-existing semantics of `Annotated` intact. - Jia
`Annotated` was introduced to support use cases other than standardized static type checking use cases described in PEP 484. Static type checkers like mypy and pyright necessarily ignore any information provided in an `Annotated` annotation beyond the first type argument. You mentioned that PEP 613 and 655 introduced new forms. Those are related to static type checking, so it wouldn't be appropriate for these to use `Annotated`. This proposal would require static type checkers to sometimes evaluate the second type argument but only in some cases. As Jia said, static type checkers already need to special-case handling of these special forms. Introducing another form using `Annotated` adds complexity and ambiguity. So I'm very negative on this proposal. -Eric -- Eric Traut Contributor to pyright & pylance Microsoft
participants (3)
-
Eric Traut -
Jia Chen -
val tron