Eric's post is quite good, but we should adjust the terminology. Type variables are bound *by* (not *to*) a class or function definition. They occur bound *in* a scope like a class's bases and body, or a function's signature and body. Saying that they are bound *to* the class or function invites confusion. One might think that the type variable _T of list is bound to int in list[int]. Next, I don't think there is any actual ambiguity in def get_identity_func() -> Callable[[T], T]: ... though it's not explicit enough in the PEP for my tastes. get_identity_func is a generic function parameterized over the free type variables in its signature. T is implicitly bound by the function definition. There is not yet any syntax for generic function types though they exist, so Callable[[T], T] must be some instantiation. There is also not yet any syntax for explicitly instantiating a generic function (like get_identity_func[int]). It would be great to have syntax for both of those. There is a way to solve for T, it's just underconstrained. Every type variable is constrained to be a subtype of its bound. It's just in this case because T appears both co- and contravariantly, there is no least (according to the subtype relation) solution. If the bound were float, both Callable[[int], int] and Callable[[float], float] are solutions and neither one is a subtype of the other. Otherwise, the rest is good. We want: * type variables bound by class definitions, bound in the bases and the body * type variables bound by function definitions, bound in the signature and the body * type variables bound by type aliases, bound in the right-hand side * type variables bound by generic function types (here a shorthand syntax for function signatures would have really helped us) I prefer square brackets over angle brackets, because then the binding and the instantiation use the same syntax. (Just like parentheses are used for both bindings introduced by parameter lists and instantiations introduced by argument lists.) But writing them out with angle brackets is instructive because it focuses on what's new.
# Type Alias??? ListOrDict<T1, T2> = List<T2> | Dict<T1, T2>
Surely that has to be ListOrDict<T1, T2> = list[T2] | dict[T1, T2] The left-hand side is a binding of T1 and T2. The right-hand side is an instantiation of list and dict. On Mon, Mar 14, 2022 at 9:09 PM Eric Traut <eric@traut.com> wrote:
I'm glad to see interest in improving this aspect of the type system. The current mechanism for defining a TypeVar (and ParamSpec and TypeVarTuple) in Python is very confusing. It affects readability as well as understanding of type variable scoping.
Before we get too deep into a discussion about syntax options, I think we should establish some principles about what we are trying to achieve and the use cases it would cover. This aligns with Guido's suggestion that we enumerate the various ways that TypeVars can be used.
Let's first think about TypeVar scopes. PEP 484 provides some guidance about TypeVar scoping rules, but it is incomplete. As a precursor to this effort, I think we need to address some of the existing holes in the specification regarding TypeVar scoping.
PEP 484 talks about the "binding" of a TypeVar to a class or a function and describes rules in TypeVars of the same name can and cannot be used within nested contexts. It does not say anything about type variables that are bound in the context of a generic type alias. It also doesn't discuss ambiguous cases like the following:
```python def get_identity_func() -> Callable[[T], T]: ... ``` Is `T` bound to `get_identity_func`, or is it bound to the returned callable? If it's bound to the former, then there is no way to "solve for T" based on a call to `get_identity_func`. For that reason, mypy and pyright (and perhaps other type checkers — I'm not sure) treat `T` as though it's bound to the returned callable.
So I think we should consider the following use cases: 1. A TypeVarLike bound to a class 2. A TypeVarLike bound to a function/method 3. A TypeVarLike bound to a type alias And possibly: 4. A TypeVarLike bound to a callable
(Note: I'm using TypeVarLike above to refer to TypeVar, ParamSpec and TypeVarTuple.)
I don't see any way for a decorator approach to work for cases 3 and 4.
I'm not sure how to make the decorator approach work in practice, even for functions and classes. The problem is that decorators are evaluated after the statement they are decorating. For example, a decorator applied to a function is evaluated after the `def` statement is evaluated and a function object is constructed. If a `@typevar("T")` decorator is responsible for declaring a symbol `T` that is needed for the evaluation of the `def` statement, that won't work.
There's no precedent in the language for a symbol being defined within a scope through the use of a string literal. For this reason, `@typevar("T")` doesn't strike me as a good way to inject the symbol `T` into a decorated function or class scope.
For all of the above reasons, I don't think it's feasible to use decorators here. Unless someone has suggestions for overcoming all of the above problems, I think we can eliminate decorators from the list.
The square bracket and angle bracket proposals are similar. Let's consider what those might look like for the four use cases I listed above. I've used angle brackets here, but these could easily be swapped for square brackets.
```python # Class class Child<T1, T2>(Parent1[T1], Parent2[T2]): ...
# Function def foo<T1, T2>(x: T1, y: T2) -> List[T1 | T2]: ...
# Type Alias??? ListOrDict<T1, T2> = List<T2> | Dict<T1, T2>
# (with TypeAlias annotation) ListOrDict: TypeAlias<T1, T2> = List<T2> | Dict<T1, T2>
# (with function call) ListOrDict: TypeAlias = type_alias<T1, T2>(List<T2> | Dict<T1, T2>)
# (with a new keyword) alias ListOrDict<T1, T2> = List<T2> | Dict<T1, T2>
# Callable??? def get_identity_func() -> Callable<T>[[T], T]: ... ```
Another scope-related consideration... The signature of a function (the `def` statement) is evaluated outside of the scope of the function body. Therefore, by normal runtime rules, the symbol `T` must be available in the scope that contains the `def` statement. Can this be implemented such that a TypeVar `T` that is bound to one function doesn't "leak" into the scope that contains that function?
```python def func<T>(x: T) -> T: ...
# Is "T" accessible at this point in the code? ```
One solution to this scoping problem is to introduce another runtime scope for each function and class so there is a private symbol table that holds any type variables introduced into the scope. But this would represent a pretty significant runtime change (and potentially a perf and memory impact). I think we'd get quite a bit of pushback.
Maybe it's better to say that `T` can "leak" at runtime, but type checkers would generate an error if this leaked value was used beyond the context in which it was intended to be used.
Another option is to follow the precedent for the "except ... as" statement where the runtime explicitly deletes the symbol after the context in which it meant to be used. That way, any subsequent use of the symbol would generate a runtime exception.
```python def func<T>(x: T) -> T: ...
# translates to the following at runtime T = TypeVar("T") def func(x: T) -> T: ... del T ```
-Eric
-- Eric Traut Contributor to Pyright & Pylance Microsoft _______________________________________________ Typing-sig mailing list -- typing-sig@python.org To unsubscribe send an email to typing-sig-leave@python.org https://mail.python.org/mailman3/lists/typing-sig.python.org/ Member address: kmillikin@google.com