I have drafted an initial PEP for TypeForm, a way to spell the type of
typing special forms (like Union[int, str], Literal['foo'], etc). Please
see the following commentable document, or the copy at the bottom of
this email:
https://docs.google.com/document/d/18UF8V00EVU1-h-BtiVFhXoJkvfL4rHp4ORaenMQL...
Please leave your comments either on the document itself or by responses
to this email on typing-sig. I will integrate feedback periodically.
For further background on why this feature is being introduced and prior
discussions, please see:
(1) the originating thread on typing-sig at:
https://mail.python.org/archives/list/typing-sig@python.org/thread/I5ZOQICTJ...
(2) the discussion about TypeForm on the mypy issue tracker at:
https://github.com/python/mypy/issues/9773
--
David Foster | Seattle, WA, USA
Contributor to TypedDict support for mypy
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
PEP: 9999
Title: TypeForm: Type Hints for Typing Special Forms and Regular Types
Author: David Foster
Sponsor: TODO
Discussions-To: typing-sig@python.org
Status: Draft
Type: Standards Track
Content-Type: text/x-rst
Requires: 647 (TypeGuard)
Created: 21-Dec-2020
Python-Version: 3.10
Post-History: 24-Jan-2021
Abstract
========
PEP 484 [^type-c] defines the type `Type[C]` where `C` is a class, to
refer to a class object that is a subtype of `C`. It explicitly does not
allow `Type[C]` to refer to typing special forms such as the runtime
object `Optional[str]` even if `C` is an unbounded `TypeVar`. This PEP
proposes a new type `TypeForm` that allows referring to both any class
object and *also* to any runtime object that is a typing special form,
such as `Optional[str]`, `Union[int, str]`, or `MyTypedDict`.
[^type-c]:
https://www.python.org/dev/peps/pep-0484/#the-type-of-class-objects
Motivation
==========
The introduction of `TypeForm` allows new kinds of metaprogramming
functions that operate on typing special forms to be type-annotated.
For example, here is a function that checks whether a value is
assignable to a variable of a particular type, and if so returns the
original value:
```
T = TypeVar('T')
def trycast(form: TypeForm[T], value: object) -> Optional[T]: ...
```
And here is another function that checks whether a value is assignable
to a variable of a particular type, and if so returns `True`:
```
def isassignable(value: object, form: TypeForm) -> bool: ...
```
With the introduction of `TypeGuard` in PEP 647[^TypeGuardPep] the above
function can be enhanced to return a `TypeGuard` instead of a regular
`bool`:
[^TypeGuardPep]: https://www.python.org/dev/peps/pep-0647/
```
def isassignable(value: object, form: TypeForm[T]) -> TypeGuard[T]: ...
```
Without `TypeForm` the next-best type that could be given to the `form`
parameter in the above examples would be `object` which would improperly
allow values like `1` to be passed in. More importantly, there would be
no way to express the relationship between the parameter type of `form`
and the function's return type.
NB: The preceding example functions implement the kinds of enhanced
`isinstance` checks that were omitted in PEP
589[^typeddict-no-isinstance] which are very useful for, among other
things, [checking whether a value decoded from JSON conforms to a
particular structure] of nested `TypedDict`s, `List`s, `Optional`s,
`Literal`s, and other types.
[^typeddict-no-isinstance]:
https://www.python.org/dev/peps/pep-0589/#using-typeddict-types
[checking whether a value decoded from JSON conforms to a particular
structure]:
https://mail.python.org/archives/list/typing-sig@python.org/thread/I5ZOQICTJ...
Specification
=============
A type-form type represents a `type` object or a special typing form
such as `Optional[str]`, `Union[int, str]`, or `MyTypedDict`. A
type-form type can be written as either `TypeForm[T]` where `T` is a
type variable or as `TypeForm` with no argument.
The `T` in `TypeForm[T]` is always an invariant type variable with no bound:
* Attempting to use a type variable `T` declared with a `bound=...` or
marked with `covariant=True` or `contravariant=True` in a `TypeForm[T]`
should be rejected by typecheckers.
* Attempting to use a literal type-form `*form*` as an argument to
`TypeForm[*form*]`, such as `TypeForm[int]`, should be rejected by
typecheckers. (See §"TypeForm[*form*] or Literal[*form*]" in Rejected
Ideas for rationale.)
The syntax `TypeForm` alone, without a type argument, is equivalent to
`TypeForm[_T]` where `_T` is a freshly created type variable.
```
T = TypeVar('T')
def trycast(form: TypeForm[T], value: object) -> Optional[T]: ...
def isassignable(value: object, form: TypeForm) -> bool: ...
def is_type(form: TypeForm) -> bool: ...
```
Using TypeForm Types
--------------------
Type-form types may be used as function parameter types, return types,
and variable types:
```
def is_type(form: TypeForm) -> bool: ... # parameter type
```
```
S = TypeVar('S')
T = TypeVar('T')
U = TypeVar('U')
def meet_types(s: TypeForm[S], t: TypeForm[T]) \
-> Union[TypeForm[S], TypeForm[T], TypeForm[U]]: ... # return types
```
```
NULL_TYPE: TypeForm # variable type
NULL_TYPE = type(None)
```
```
NULL_TYPE: TypeForm = type(None) # variable type
```
Note however that a typechecker won't automatically infer a TypeForm
type for an unannotated variable assignment that contains a special
typing form on the right-hand-side because PEP 484 [^type-alias-syntax]
reserves that syntax for defining type aliases:
[^type-alias-syntax]: https://www.python.org/dev/peps/pep-0484/#type-aliases
```
NULL_TYPE = type(None) # OOPS; treated as a type alias!
```
If you want a typechecker to infer a TypeForm variable type for a bare
assignment, use the walrus operator instead of the assignment operator:
```
NULL_TYPE := type(None) # infers NULL_TYPE as having type TypeForm
```
Or explicitly declare the assignment-target as having `TypeForm` type:
```
NULL_TYPE: TypeForm = type(None)
```
```
NULL_TYPE = type(None) # type: TypeForm # the type comment is significant
```
```
NULL_TYPE: TypeForm
NULL_TYPE = type(None)
```
Values of type TypeForm
-----------------------
A particular literal value is said to *inhabit* a type if it is
described by that type and can be assigned to a variable of that type.
The type `TypeForm` is inhabited by exactly those runtime objects that
are valid on the right-hand-side of a variable declaration,
```
value: *form*
```
the right-hand-side of a parameter declaration,
```
def some_func(value: *form*):
```
*and* as the return type of a function:
```
def some_func() -> *form*:
```
Any runtime object that is not valid in one of the above locations is
not an inhabitant of `TypeForm`. For example the special forms `Final`,
`Final[int]`, `NoReturn`, and `ClassVar[int]` are not inhabitants.
Incomplete forms like a bare `Optional` or `Union` are also not inhabitants.
Example of inhabitants include:
* type objects like `int`, `str`, `object`, and `FooClass`
* generic collections like `List`, `List[int]`, `Dict`, or `Dict[K, V]`
* callables like `Callable`, `Callable[[Arg1Type, Arg2Type],
ReturnType]`, `Callable[..., ReturnType]`
* union forms like `Optional[str]` or `Union[int, str]`
* literal forms like `Literal['r', 'rb', 'w', 'wb']`
* type variables like `T` or `AnyStr`
* annotated types like `Annotated[int, ValueRange(-10, 5)]`
* type aliases like `Vector` (where `Vector = list[float]`)
* the `Any` form
* the `Type` and `Type[C]` forms
* the `TypeForm` and `TypeForm[T]` forms
A few peculiar cases permitted by the above include:
```
STR_FORM: TypeForm = str
STR_OR_SUBCLASS_FORM: TypeForm = Type[str]
SOME_FORM: TypeForm = TypeForm
```
Forward References
''''''''''''''''''
Runtime objects for forms which contain forward references such as
`Union['Triangle', 'Shape']` are normalized at runtime to
`Union[ForwardRef('Triangle'), ForwardRef('Shape')]`. The latter are
inhabitants of `TypeForm`.
Additionally the "top-level" form for a particular variable type can
itself be a forward reference such as in:
```
class Node:
value: object
next: 'Node' # 'Node' is normalized to ForwardRef('Node') at runtime
```
Code that manipulates TypeForm values at runtime should be prepared to
deal with ForwardRef(...) subcomponents.
Type Consistency
----------------
Informally speaking, *type consistency* is a generalization of the
is-subtype-of relation to support the `Any` type. It is defined more
formally in PEP 483 [^type-consistency]. This section introduces the new
rules needed to support type consistency for TypeForm types:
[^type-consistency]:
https://www.python.org/dev/peps/pep-0483/#summary-of-gradual-typing
* `Type[C]` is consistent with `TypeForm`. `TypeForm` is consistent with
`object`.
* `TypeForm[S]` is consistent with `TypeForm[T]` iff `S` and `T` are the
same type variable.
* `Any` is consistent with `TypeForm`. `TypeForm` is consistent with `Any`.
Overloading
-----------
`TypeForm` can be used in overloads. For example:
```
T = TypeVar('T')
F = TypeVar('F')
@overload
def trycast(form: TypeForm[T], value: object) -> Optional[T]: ...
@overload
def trycast(form: TypeForm[T], value: object, failure: F) -> Union[T,
F]: ...
```
Interactions with Type[C]
-------------------------
A `TypeForm` and a `Type` cannot simultaneously constrain the same type
variable within a generic function definition:
```
def bad_func(form: TypeForm[T], tp: Type[T]): ... # invalid
```
A group of type variables where some variables are constrained by
`TypeForm`s, some are constrained by `Type`s, and others are
unconstrained may appear together as arguments to the same type
constructor (i.e. `Union[...]`, `Dict[K, V]`, etc):
```
F = TypeVar('F')
C = TypeVar('C')
V = TypeVar('V')
def pick_random(form: TypeForm[F], tp: Type[C], value: V) -> Union[F, C,
V]: ...
```
Interactions with cast
----------------------
It is possible to cast a value to a `TypeForm` or `TypeForm[T]` type:
```
OPTIONAL_STR_OBJECT: object = Optional[str]
OPTIONAL_STR = cast(TypeForm, OPTIONAL_STR_OBJECT)
```
Type Inference
--------------
A runtime object corresponding to a special form should be inferred as
having type `TypeForm` unless it can instead be inferred to have type
`Type[C]` for some `C`:
```
T = TypeVar('T')
def identity(value: T) -> T:
return value
A_TYPE = identity(str) # inferred type is: Type[str]
A_FORM = identity(Optional[str]) # inferred type is: TypeForm
```
Backwards Compatibility
=======================
Previously the type `object` would be inferred for runtime objects
corresponding to a special form like `Optional[str]` rather than
`TypeForm`. But since `TypeForm` is consistent with `object` no
unexpected typechecking failures will be introduced by this change.
Ongoing Maintenance
===================
As new kinds of typing special forms are introduced by upcoming PEPs
(usually to the `typing` module), it will be necessary for typecheckers
to define whether or not the new forms should be considered inhabitants
of `TypeForm` or not, based on the general rules in §"Values of type
TypeForm" above. Therefore it is recommended that any new PEP
introducing a new typing special form should define whether it inhabits
`TypeForm` or not.
Reference Implementation
========================
The following will be true when
[mypy#9773](https://github.com/python/mypy/issues/9773) is implemented:
The mypy type checker supports `TypeForm` types. A reference
implementation of the runtime component is provided in the
`typing_extensions` module.
In addition an implementation of the `trycast` and `isassignable`
functions mentioned in various examples above is being implemented in
the [trycast PyPI module](https://github.com/davidfstr/trycast).
Rejected Ideas
==============
TypeForm[*form*] or Literal[*form*]
-----------------------------------
This PEP only defines syntax for a bare `TypeForm` and a `TypeForm[T]`
parameterized by a type variable. In particular it does not define
syntax like that allows referring to a specific known type-form value
via a syntax like `TypeForm[*form*]`:
```
NULL_TYPE := type(None) # has type TypeForm[_T] rather than
TypeForm[type(None)]
UNION_STR_TYPE := Union[Str] # has type TypeForm[_T] rather than
TypeForm[Union[str]]
```
In addition similar syntax like `Literal[*form*]` also does not work.
Neither of those syntaxes seem to be especially useful; type forms are
not typically used as sentinel values. It is possible that a future PEP
may revisit such syntaxes.
Open Issues
===========
[Any points that are still being decided/discussed.]
References
==========
[A collection of URLs used as references through the PEP.]
Copyright
=========
This document is placed in the public domain or under the
CC0-1.0-Universal license, whichever is more permissive.
..
Local Variables:
mode: indented-text
indent-tabs-mode: nil
sentence-end-double-space: t
fill-column: 70
coding: utf-8
End:
<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<