First impressions:

I noticed that none of your examples actually show how you would write the body of a function like isassignable(). Are you assuming that those implementations just introspect the runtime representation?

I also saw this:

If you want a typechecker to infer a TypeForm variable type for a bare assignment, use the walrus operator instead of the assignment operator:


```

NULL_TYPE := type(None)  # infers NULL_TYPE as having type TypeForm

```


The walrus operator syntactically cannot be used at the top level, so this clever hack won't work.

All in all I feel pretty lukewarm about this proposal (but then again, static type checking satisfies all my needs).

--Guido

On Sun, Jan 24, 2021 at 10:07 PM David Foster <davidfstr@gmail.com> wrote:
I have drafted an initial PEP for TypeForm, a way to spell the type of
typing special forms (like Union[int, str], Literal['foo'], etc). Please
see the following commentable document, or the copy at the bottom of
this email:
https://docs.google.com/document/d/18UF8V00EVU1-h-BtiVFhXoJkvfL4rHp4ORaenMQL-Zo/edit?usp=sharing

Please leave your comments either on the document itself or by responses
to this email on typing-sig. I will integrate feedback periodically.

For further background on why this feature is being introduced and prior
discussions, please see:

(1) the originating thread on typing-sig at:
https://mail.python.org/archives/list/typing-sig@python.org/thread/I5ZOQICTJCENTCDPHLZR7NT42QJ43GP4/

(2) the discussion about TypeForm on the mypy issue tracker at:
https://github.com/python/mypy/issues/9773

--
David Foster | Seattle, WA, USA
Contributor to TypedDict support for mypy


 >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>

PEP: 9999
Title: TypeForm: Type Hints for Typing Special Forms and Regular Types
Author: David Foster <david@dafoster.net>
Sponsor: TODO
Discussions-To: typing-sig@python.org
Status: Draft
Type: Standards Track
Content-Type: text/x-rst
Requires: 647 (TypeGuard)
Created: 21-Dec-2020
Python-Version: 3.10
Post-History: 24-Jan-2021


Abstract
========

PEP 484 [^type-c] defines the type `Type[C]` where `C` is a class, to
refer to a class object that is a subtype of `C`. It explicitly does not
allow `Type[C]` to refer to typing special forms such as the runtime
object `Optional[str]` even if `C` is an unbounded `TypeVar`. This PEP
proposes a new type `TypeForm` that allows referring to both any class
object and *also* to any runtime object that is a typing special form,
such as `Optional[str]`, `Union[int, str]`, or `MyTypedDict`.

[^type-c]:
https://www.python.org/dev/peps/pep-0484/#the-type-of-class-objects


Motivation
==========

The introduction of `TypeForm` allows new kinds of metaprogramming
functions that operate on typing special forms to be type-annotated.

For example, here is a function that checks whether a value is
assignable to a variable of a particular type, and if so returns the
original value:

```
T = TypeVar('T')

def trycast(form: TypeForm[T], value: object) -> Optional[T]: ...
```

And here is another function that checks whether a value is assignable
to a variable of a particular type, and if so returns `True`:

```
def isassignable(value: object, form: TypeForm) -> bool: ...
```

With the introduction of `TypeGuard` in PEP 647[^TypeGuardPep] the above
function can be enhanced to return a `TypeGuard` instead of a regular
`bool`:

[^TypeGuardPep]: https://www.python.org/dev/peps/pep-0647/

```
def isassignable(value: object, form: TypeForm[T]) -> TypeGuard[T]: ...
```

Without `TypeForm` the next-best type that could be given to the `form`
parameter in the above examples would be `object` which would improperly
allow values like `1` to be passed in. More importantly, there would be
no way to express the relationship between the parameter type of `form`
and the function's return type.

NB: The preceding example functions implement the kinds of enhanced
`isinstance` checks that were omitted in PEP
589[^typeddict-no-isinstance] which are very useful for, among other
things, [checking whether a value decoded from JSON conforms to a
particular structure] of nested `TypedDict`s, `List`s, `Optional`s,
`Literal`s, and other types.

[^typeddict-no-isinstance]:
https://www.python.org/dev/peps/pep-0589/#using-typeddict-types

[checking whether a value decoded from JSON conforms to a particular
structure]:
https://mail.python.org/archives/list/typing-sig@python.org/thread/I5ZOQICTJCENTCDPHLZR7NT42QJ43GP4/


Specification
=============

A type-form type represents a `type` object or a special typing form
such as `Optional[str]`, `Union[int, str]`, or `MyTypedDict`. A
type-form type can be written as either `TypeForm[T]` where `T` is a
type variable or as `TypeForm` with no argument.

The `T` in `TypeForm[T]` is always an invariant type variable with no bound:

* Attempting to use a type variable `T` declared with a `bound=...` or
marked with `covariant=True` or `contravariant=True` in a `TypeForm[T]`
should be rejected by typecheckers.

* Attempting to use a literal type-form `*form*` as an argument to
`TypeForm[*form*]`, such as `TypeForm[int]`, should be rejected by
typecheckers. (See §"TypeForm[*form*] or Literal[*form*]" in Rejected
Ideas for rationale.)

The syntax `TypeForm` alone, without a type argument, is equivalent to
`TypeForm[_T]` where `_T` is a freshly created type variable.

```
T = TypeVar('T')

def trycast(form: TypeForm[T], value: object) -> Optional[T]: ...

def isassignable(value: object, form: TypeForm) -> bool: ...

def is_type(form: TypeForm) -> bool: ...
```


Using TypeForm Types
--------------------

Type-form types may be used as function parameter types, return types,
and variable types:

```
def is_type(form: TypeForm) -> bool: ...  # parameter type
```

```
S = TypeVar('S')
T = TypeVar('T')
U = TypeVar('U')

def meet_types(s: TypeForm[S], t: TypeForm[T]) \
     -> Union[TypeForm[S], TypeForm[T], TypeForm[U]]: ...  # return types
```

```
NULL_TYPE: TypeForm  # variable type
NULL_TYPE = type(None)
```

```
NULL_TYPE: TypeForm = type(None)  # variable type
```

Note however that a typechecker won't automatically infer a TypeForm
type for an unannotated variable assignment that contains a special
typing form on the right-hand-side because PEP 484 [^type-alias-syntax]
reserves that syntax for defining type aliases:

[^type-alias-syntax]: https://www.python.org/dev/peps/pep-0484/#type-aliases

```
NULL_TYPE = type(None)  # OOPS; treated as a type alias!
```

If you want a typechecker to infer a TypeForm variable type for a bare
assignment, use the walrus operator instead of the assignment operator:

```
NULL_TYPE := type(None)  # infers NULL_TYPE as having type TypeForm
```

Or explicitly declare the assignment-target as having `TypeForm` type:

```
NULL_TYPE: TypeForm = type(None)
```

```
NULL_TYPE = type(None)  # type: TypeForm  # the type comment is significant
```

```
NULL_TYPE: TypeForm
NULL_TYPE = type(None)
```


Values of type TypeForm
-----------------------

A particular literal value is said to *inhabit* a type if it is
described by that type and can be assigned to a variable of that type.

The type `TypeForm` is inhabited by exactly those runtime objects that
are valid on the right-hand-side of a variable declaration,

```
value: *form*
```

the right-hand-side of a parameter declaration,

```
def some_func(value: *form*):
```

*and* as the return type of a function:

```
def some_func() -> *form*:
```

Any runtime object that is not valid in one of the above locations is
not an inhabitant of `TypeForm`. For example the special forms `Final`,
`Final[int]`, `NoReturn`, and `ClassVar[int]` are not inhabitants.
Incomplete forms like a bare `Optional` or `Union` are also not inhabitants.

Example of inhabitants include:

* type objects like `int`, `str`, `object`, and `FooClass`
* generic collections like `List`, `List[int]`, `Dict`, or `Dict[K, V]`
* callables like `Callable`, `Callable[[Arg1Type, Arg2Type],
ReturnType]`, `Callable[..., ReturnType]`
* union forms like `Optional[str]` or `Union[int, str]`
* literal forms like `Literal['r', 'rb', 'w', 'wb']`
* type variables like `T` or `AnyStr`
* annotated types like `Annotated[int, ValueRange(-10, 5)]`
* type aliases like `Vector` (where `Vector = list[float]`)
* the `Any` form
* the `Type` and `Type[C]` forms
* the `TypeForm` and `TypeForm[T]` forms

A few peculiar cases permitted by the above include:

```
STR_FORM: TypeForm = str
STR_OR_SUBCLASS_FORM: TypeForm = Type[str]
SOME_FORM: TypeForm = TypeForm
```


Forward References
''''''''''''''''''

Runtime objects for forms which contain forward references such as
`Union['Triangle', 'Shape']` are normalized at runtime to
`Union[ForwardRef('Triangle'), ForwardRef('Shape')]`. The latter are
inhabitants of `TypeForm`.

Additionally the "top-level" form for a particular variable type can
itself be a forward reference such as in:

```
class Node:
     value: object
     next: 'Node'  # 'Node' is normalized to ForwardRef('Node') at runtime
```

Code that manipulates TypeForm values at runtime should be prepared to
deal with ForwardRef(...) subcomponents.


Type Consistency
----------------

Informally speaking, *type consistency* is a generalization of the
is-subtype-of relation to support the `Any` type. It is defined more
formally in PEP 483 [^type-consistency]. This section introduces the new
rules needed to support type consistency for TypeForm types:

[^type-consistency]:
https://www.python.org/dev/peps/pep-0483/#summary-of-gradual-typing

* `Type[C]` is consistent with `TypeForm`. `TypeForm` is consistent with
`object`.

* `TypeForm[S]` is consistent with `TypeForm[T]` iff `S` and `T` are the
same type variable.

* `Any` is consistent with `TypeForm`. `TypeForm` is consistent with `Any`.


Overloading
-----------

`TypeForm` can be used in overloads. For example:

```
T = TypeVar('T')
F = TypeVar('F')

@overload
def trycast(form: TypeForm[T], value: object) -> Optional[T]: ...
@overload
def trycast(form: TypeForm[T], value: object, failure: F) -> Union[T,
F]: ...
```


Interactions with Type[C]
-------------------------

A `TypeForm` and a `Type` cannot simultaneously constrain the same type
variable within a generic function definition:

```
def bad_func(form: TypeForm[T], tp: Type[T]): ...  # invalid
```

A group of type variables where some variables are constrained by
`TypeForm`s, some are constrained by `Type`s, and others are
unconstrained may appear together as arguments to the same type
constructor (i.e. `Union[...]`, `Dict[K, V]`, etc):

```
F = TypeVar('F')
C = TypeVar('C')
V = TypeVar('V')

def pick_random(form: TypeForm[F], tp: Type[C], value: V) -> Union[F, C,
V]: ...
```


Interactions with cast
----------------------

It is possible to cast a value to a `TypeForm` or `TypeForm[T]` type:

```
OPTIONAL_STR_OBJECT: object = Optional[str]
OPTIONAL_STR = cast(TypeForm, OPTIONAL_STR_OBJECT)
```

Type Inference
--------------

A runtime object corresponding to a special form should be inferred as
having type `TypeForm` unless it can instead be inferred to have type
`Type[C]` for some `C`:

```
T = TypeVar('T')

def identity(value: T) -> T:
     return value

A_TYPE = identity(str)  # inferred type is: Type[str]
A_FORM = identity(Optional[str])  # inferred type is: TypeForm
```


Backwards Compatibility
=======================

Previously the type `object` would be inferred for runtime objects
corresponding to a special form like `Optional[str]` rather than
`TypeForm`. But since `TypeForm` is consistent with `object` no
unexpected typechecking failures will be introduced by this change.


Ongoing Maintenance
===================

As new kinds of typing special forms are introduced by upcoming PEPs
(usually to the `typing` module), it will be necessary for typecheckers
to define whether or not the new forms should be considered inhabitants
of `TypeForm` or not, based on the general rules in §"Values of type
TypeForm" above. Therefore it is recommended that any new PEP
introducing a new typing special form should define whether it inhabits
`TypeForm` or not.


Reference Implementation
========================

The following will be true when
[mypy#9773](https://github.com/python/mypy/issues/9773) is implemented:

     The mypy type checker supports `TypeForm` types. A reference
implementation of the runtime component is provided in the
`typing_extensions` module.

In addition an implementation of the `trycast` and `isassignable`
functions mentioned in various examples above is being implemented in
the [trycast PyPI module](https://github.com/davidfstr/trycast).


Rejected Ideas
==============

TypeForm[*form*] or Literal[*form*]
-----------------------------------

This PEP only defines syntax for a bare `TypeForm` and a `TypeForm[T]`
parameterized by a type variable. In particular it does not define
syntax like that allows referring to a specific known type-form value
via a syntax like `TypeForm[*form*]`:

```
NULL_TYPE := type(None)       # has type TypeForm[_T] rather than
TypeForm[type(None)]
UNION_STR_TYPE := Union[Str]  # has type TypeForm[_T] rather than
TypeForm[Union[str]]
```

In addition similar syntax like `Literal[*form*]` also does not work.

Neither of those syntaxes seem to be especially useful; type forms are
not typically used as sentinel values. It is possible that a future PEP
may revisit such syntaxes.


Open Issues
===========

[Any points that are still being decided/discussed.]


References
==========

[A collection of URLs used as references through the PEP.]


Copyright
=========

This document is placed in the public domain or under the
CC0-1.0-Universal license, whichever is more permissive.



..
    Local Variables:
    mode: indented-text
    indent-tabs-mode: nil
    sentence-end-double-space: t
    fill-column: 70
    coding: utf-8
    End:

<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
_______________________________________________
Typing-sig mailing list -- typing-sig@python.org
To unsubscribe send an email to typing-sig-leave@python.org
https://mail.python.org/mailman3/lists/typing-sig.python.org/
Member address: guido@python.org


--
--Guido van Rossum (python.org/~guido)
Pronouns: he/him (why is my pronoun here?)