Am 14.03.22 um 15:51 schrieb Sergei Lebedev:

...

class C[T, S]:      def foo[R=int](self) -> R: pass

Could you clarify what the "=int" part stands for?

A type var default (not currently supported by type vars).

...

Some ideas how we could express upper bounds, constraints, and variance: [...]

Wdyt about <= for defining an upper bound and +/- for variance annotations? IIRC both are used at least in Scala and OCaml has +/- for specifying variance and </> for polymorphic variants <https://ocaml.org/manual/polyvariant.html>.

This could work, but I'm not too fond of using too many punctuation characters. Python has always been the antithesis to "line-noise" Perl, and I think "key words" (not necessarily "keywords" from a parser perspective) are more readable and obvious.

* Could we make it work for ParamSpecs?

This is one more reason a decorator syntax could be better: @paramspec("P") def foo(*args: P.args, **kwargs: P.kwargs) -> None: ...  - Sebastian

I'm excited for this since I've always found the TypeVar approach a little cumbersome. I would only ask for the runtime aspect to also be considered. For example, cattrs can already handle creating MyClass[int] from json, so it would be great to keep that functionality. Also, the approach with less importing is probably going to be more pleasant to use. On Mon, Mar 14, 2022 at 4:39 PM Guido van Rossum <guido@python.org> wrote:

I think it would be useful to compile a table of various ways one can use typevars to show the analogous syntax using each of the proposals. E.g.

# Classic T = TypeVar("T") class C(Generic[T]): def f(self, arg: T) -> T: ...

# Angle brackets class C<T>: def f(self, arg: T) -> T: ...

# Square brackets class C[T]: def f(self, arg: T) -> T: ...

# Decorator @typevar("T") class C: def f(self, arg: T) -> T: ...

Include examples using variance (could we use +T, -T for covariant, contravariant?), bounds and constraints. Show generic functions as well as classes. Go wild.

When using the decorator syntax, I have a question: how would the implementation of `typevar` insert the name "T" into the surrounding scope so that it can be used inside the class or function? sys._getframe()? (Urgh.)

On Mon, Mar 14, 2022 at 8:05 AM Sebastian Rittau <srittau@rittau.biz> wrote:

...

Am 14.03.22 um 15:51 schrieb Sergei Lebedev:

...

class C[T, S]: def foo[R=int](self) -> R: pass

Could you clarify what the "=int" part stands for?

A type var default (not currently supported by type vars).

...

Some ideas how we could express upper bounds, constraints, and variance: [...]

Wdyt about <= for defining an upper bound and +/- for variance annotations? IIRC both are used at least in Scala and OCaml has +/- for specifying variance and </> for polymorphic variants <https://ocaml.org/manual/polyvariant.html>.

This could work, but I'm not too fond of using too many punctuation characters. Python has always been the antithesis to "line-noise" Perl, and I think "key words" (not necessarily "keywords" from a parser perspective) are more readable and obvious.

* Could we make it work for ParamSpecs?

This is one more reason a decorator syntax could be better:

@paramspec("P") def foo(*args: P.args, **kwargs: P.kwargs) -> None: ... - Sebastian _______________________________________________ Typing-sig mailing list -- typing-sig@python.org To unsubscribe send an email to typing-sig-leave@python.org https://mail.python.org/mailman3/lists/typing-sig.python.org/ Member address: guido@python.org

-- --Guido van Rossum (python.org/~guido) *Pronouns: he/him **(why is my pronoun here?)* <http://feministing.com/2015/02/03/how-using-they-as-a-singular-pronoun-can-c...> _______________________________________________ Typing-sig mailing list -- typing-sig@python.org To unsubscribe send an email to typing-sig-leave@python.org https://mail.python.org/mailman3/lists/typing-sig.python.org/ Member address: tinchester@gmail.com

Eric's post is quite good, but we should adjust the terminology. Type variables are bound *by* (not *to*) a class or function definition. They occur bound *in* a scope like a class's bases and body, or a function's signature and body. Saying that they are bound *to* the class or function invites confusion. One might think that the type variable _T of list is bound to int in list[int]. Next, I don't think there is any actual ambiguity in def get_identity_func() -> Callable[[T], T]: ... though it's not explicit enough in the PEP for my tastes. get_identity_func is a generic function parameterized over the free type variables in its signature. T is implicitly bound by the function definition. There is not yet any syntax for generic function types though they exist, so Callable[[T], T] must be some instantiation. There is also not yet any syntax for explicitly instantiating a generic function (like get_identity_func[int]). It would be great to have syntax for both of those. There is a way to solve for T, it's just underconstrained. Every type variable is constrained to be a subtype of its bound. It's just in this case because T appears both co- and contravariantly, there is no least (according to the subtype relation) solution. If the bound were float, both Callable[[int], int] and Callable[[float], float] are solutions and neither one is a subtype of the other. Otherwise, the rest is good. We want: * type variables bound by class definitions, bound in the bases and the body * type variables bound by function definitions, bound in the signature and the body * type variables bound by type aliases, bound in the right-hand side * type variables bound by generic function types (here a shorthand syntax for function signatures would have really helped us) I prefer square brackets over angle brackets, because then the binding and the instantiation use the same syntax. (Just like parentheses are used for both bindings introduced by parameter lists and instantiations introduced by argument lists.) But writing them out with angle brackets is instructive because it focuses on what's new.

# Type Alias??? ListOrDict<T1, T2> = List<T2> | Dict<T1, T2>

Surely that has to be ListOrDict<T1, T2> = list[T2] | dict[T1, T2] The left-hand side is a binding of T1 and T2. The right-hand side is an instantiation of list and dict. On Mon, Mar 14, 2022 at 9:09 PM Eric Traut <eric@traut.com> wrote:

I'm glad to see interest in improving this aspect of the type system. The current mechanism for defining a TypeVar (and ParamSpec and TypeVarTuple) in Python is very confusing. It affects readability as well as understanding of type variable scoping.

Before we get too deep into a discussion about syntax options, I think we should establish some principles about what we are trying to achieve and the use cases it would cover. This aligns with Guido's suggestion that we enumerate the various ways that TypeVars can be used.

Let's first think about TypeVar scopes. PEP 484 provides some guidance about TypeVar scoping rules, but it is incomplete. As a precursor to this effort, I think we need to address some of the existing holes in the specification regarding TypeVar scoping.

PEP 484 talks about the "binding" of a TypeVar to a class or a function and describes rules in TypeVars of the same name can and cannot be used within nested contexts. It does not say anything about type variables that are bound in the context of a generic type alias. It also doesn't discuss ambiguous cases like the following:

```python def get_identity_func() -> Callable[[T], T]: ... ``` Is `T` bound to `get_identity_func`, or is it bound to the returned callable? If it's bound to the former, then there is no way to "solve for T" based on a call to `get_identity_func`. For that reason, mypy and pyright (and perhaps other type checkers — I'm not sure) treat `T` as though it's bound to the returned callable.

So I think we should consider the following use cases: 1. A TypeVarLike bound to a class 2. A TypeVarLike bound to a function/method 3. A TypeVarLike bound to a type alias And possibly: 4. A TypeVarLike bound to a callable

(Note: I'm using TypeVarLike above to refer to TypeVar, ParamSpec and TypeVarTuple.)

I don't see any way for a decorator approach to work for cases 3 and 4.

I'm not sure how to make the decorator approach work in practice, even for functions and classes. The problem is that decorators are evaluated after the statement they are decorating. For example, a decorator applied to a function is evaluated after the `def` statement is evaluated and a function object is constructed. If a `@typevar("T")` decorator is responsible for declaring a symbol `T` that is needed for the evaluation of the `def` statement, that won't work.

There's no precedent in the language for a symbol being defined within a scope through the use of a string literal. For this reason, `@typevar("T")` doesn't strike me as a good way to inject the symbol `T` into a decorated function or class scope.

For all of the above reasons, I don't think it's feasible to use decorators here. Unless someone has suggestions for overcoming all of the above problems, I think we can eliminate decorators from the list.

The square bracket and angle bracket proposals are similar. Let's consider what those might look like for the four use cases I listed above. I've used angle brackets here, but these could easily be swapped for square brackets.

```python # Class class Child<T1, T2>(Parent1[T1], Parent2[T2]): ...

# Function def foo<T1, T2>(x: T1, y: T2) -> List[T1 | T2]: ...

# Type Alias??? ListOrDict<T1, T2> = List<T2> | Dict<T1, T2>

# (with TypeAlias annotation) ListOrDict: TypeAlias<T1, T2> = List<T2> | Dict<T1, T2>

# (with function call) ListOrDict: TypeAlias = type_alias<T1, T2>(List<T2> | Dict<T1, T2>)

# (with a new keyword) alias ListOrDict<T1, T2> = List<T2> | Dict<T1, T2>

# Callable??? def get_identity_func() -> Callable<T>[[T], T]: ... ```

Another scope-related consideration... The signature of a function (the `def` statement) is evaluated outside of the scope of the function body. Therefore, by normal runtime rules, the symbol `T` must be available in the scope that contains the `def` statement. Can this be implemented such that a TypeVar `T` that is bound to one function doesn't "leak" into the scope that contains that function?

```python def func<T>(x: T) -> T: ...

# Is "T" accessible at this point in the code? ```

One solution to this scoping problem is to introduce another runtime scope for each function and class so there is a private symbol table that holds any type variables introduced into the scope. But this would represent a pretty significant runtime change (and potentially a perf and memory impact). I think we'd get quite a bit of pushback.

Maybe it's better to say that `T` can "leak" at runtime, but type checkers would generate an error if this leaked value was used beyond the context in which it was intended to be used.

Another option is to follow the precedent for the "except ... as" statement where the runtime explicitly deletes the symbol after the context in which it meant to be used. That way, any subsequent use of the symbol would generate a runtime exception.

```python def func<T>(x: T) -> T: ...

# translates to the following at runtime T = TypeVar("T") def func(x: T) -> T: ... del T ```

-Eric

-- Eric Traut Contributor to Pyright & Pylance Microsoft _______________________________________________ Typing-sig mailing list -- typing-sig@python.org To unsubscribe send an email to typing-sig-leave@python.org https://mail.python.org/mailman3/lists/typing-sig.python.org/ Member address: kmillikin@google.com

I think Jukka hit the nail on the head with his analysis of the problem with angle brackets -- it would be a shame if you couldn't split a list of typevars across a line. This wouldn't seem a problem if you just have a few typevars with short names, like class C<S, T>, but it could become a problem if we add syntax for upper bounds, which would also be needed. Currently: T = TypeVar("T", bound=str|int) def f(a: T) -> list[T]: ... That would have to become something like this (borrowing from TypeScript -- we could use some other syntax instead of an 'extends' (soft) keyword): def f<T extends int | str>(a: T) -> list[T]: ... If you have two typevars, each with their own upper bound, splitting them across multiple lines could be considered more readable -- compare: def f<T extends int | str, S extends list[int] | list[str]>(a: T, b: S) -> tuple[S, T]: ... to: def f< T extends int | str, S extends list[int] | list[str]

(a: T, b: S) -> tuple[S, T]: ...

But given the number of tools that currently know about Python indents and know they should not look for indents inside (parenthes), [brackets] and {braces}, adding <angle brackets> to the list but only in `def` and `class` statements would be a big hurdle to take Now, it makes me a bit sad that this is the case, because for readability I find angle brackets slightly easier on the eyes than square brackets, but I don't see a way around it, and it's probably not too bad. Here are the above three examples using square brackets: def f[T extends int | str](a: T) -> list[T]: ... def f[T extends int | str, S extends list[int] | list[str]](a: T, b: S) -> tuple[S, T]: ... def f[ T extends int | str, S extends list[int] | list[str] ](a: T, b: S) -> tuple[S, T]: ... All these will look bad to old-timers, so we'd really have to fight for this, but the square brackets have the best chance. (Sorry, Sebastian, I agree with Eric that the @typevar("T") approach just won't work.) An open issue is variance. There's a proposal floating around ( https://github.com/python/typing/issues/813) to use +T and -T to make T covariant or contravariant. We could easily support this with the new syntax: class C[+T]: ... (I forget if generic functions also have variance, and whether the variance is per typevar or per parameter. Jelle?) For type aliases, it does look like the only way to solve that would be to have an 'alias' keyword. I've got nothing to add to what Jukka said. (Honestly I didn't really have much to add to what he said about angle brackets either. :-) Finally, the conundrum of def get_identity_func() -> Callable[[T], T]: ... It certainly doesn't look like this corner case is worth making `Callable` a keyword (and with square brackets the syntax would be awkward). I can't quite follow Kevin Millikin's post, but if the solution is that this should be written as def get_identity_func[T]() -> Callable[[T], T]: ... but keep the semantics that mypy and pyright currently give it, that sounds fine to me. Oh. A final weird thought on syntax (I can't help it, syntax is what makes Python tick for me). Maybe the square brackets with type variable definitions should go *before* the class/function name? So immediately after the keyword: def [T] f(a: T) -> T: ... class [T] C: ... alias [T1, T2] ListOrDict = List[T2] | Dict[T1, T2] This might (or might not :-) look better than having the close ']' cramped against the open '(' of the arg list. We'd be breaking new syntactic ground though -- e.g. Scala uses class Name[T] { ... } and everyone else (C#, Java, JavaScript) seems to be using class Name<T> { ... }. I guess there's also C++ which AFAICT uses something like this: template <class T> class C { ... } I don't see much of an advantage in that, except that it's the most similar to Sebastian's `@typevar("T")` proposal. If we are going to introduce a new keyword we might be able to do something like that. It would still have to use square brackets though (the problem with angle brackets is still there). A nice thing would be that it can be uniformly used to prefix `class`, `def` and type alias definitions using assignments. Enough rambling. --Guido On Tue, Mar 15, 2022 at 4:29 AM Jukka Lehtosalo <jlehtosalo@gmail.com> wrote:

I also think that Eric's post is good.

However, I don't think that angle brackets are as viable as square brackets. First, it seems less consistent than square brackets -- I agree with Kevin here. There also seems to be some potential for ambiguity: Callable<T>[[T], T], for example, is already a valid expression, where < and > are treated as binary operators. Finally, square brackets would naturally support splitting long definitions into multiple lines (in this simple example it's not necessary to use multiple lines, but in more complex examples it can be helpful):

class C[ T, S, ](...): ...

Supporting this with angle brackets seems kind of hard (especially within expressions, where there is ambiguity with existing binary operators):

class C< T, S,

...

(...): ...

I guess we could somehow support this, but it could be hard to justify the extra complexity in a PEP.

The alternative of using backslashes looks pretty bad, I think:

class C< \ T, \ S, \

...

(...): ...

Of the Eric's proposed ways of expressing type aliases, only the idea with a new keyword seems practical to me if we use square brackets. This is the first alternative that Eric mentioned:

ListOrDict<T1, T2> = List<T2> | Dict<T1, T2>

If we'd use square brackets instead, this would be syntactically identical to an indexed assignment, which wouldn't define the type alias, T1 or T2, and thus wouldn't work at runtime:

ListOrDict[T1, T2] = List[T2] | Dict[T1, T2]

The second idea is similarly ambiguous and looks like a regular generic type annotation when using square brackets, unless TypeAlias is a keyword. Again, T1 and T2 won't be defined at runtime:

ListOrDict: TypeAlias[T1, T2] = List[T2] | Dict[T1, T2]

The third idea has the same issue if using square brackets -- it already has different meaning currently and doesn't define T1 or T2:

ListOrDict: TypeAlias = type_alias[T1, T2](List[T2] | Dict[T1, T2])

If we introduce a new keyword (only treated as a keyword in this context), there would be no ambiguity and everything would work as expected, I think -- even with square brackets:

alias ListOrDict[T1, T2] = List[T2] | Dict[T1, T2]

Jukka _______________________________________________ Typing-sig mailing list -- typing-sig@python.org To unsubscribe send an email to typing-sig-leave@python.org https://mail.python.org/mailman3/lists/typing-sig.python.org/ Member address: guido@python.org

-- --Guido van Rossum (python.org/~guido) *Pronouns: he/him **(why is my pronoun here?)* <http://feministing.com/2015/02/03/how-using-they-as-a-singular-pronoun-can-c...>

On Thu, Mar 17, 2022 at 2:03 AM Jelle Zijlstra <jelle.zijlstra@gmail.com> wrote:

I think that would give the wrong semantics for most cases. This case has been discussed on this list before and I believe different type checkers currently interpret it differently. The difference is about whether T is scoped to the function or to the Callable. It would be nice to have a way to explicitly scope the TypeVar to the Callable, but no elegant solution comes to mind.

Here's an ugly solution:

def get_identity_func() -> (T := TypeVar("T") and Callable[[T], T]): ...

def get_identity_func() -> (lambda T: Callable[[T], T]): ... :-)

On Mon, Mar 14, 2022 at 3:05 PM Sebastian Rittau <srittau@rittau.biz> wrote:

Am 14.03.22 um 15:51 schrieb Sergei Lebedev:

...

class C[T, S]: def foo[R=int](self) -> R: pass

Could you clarify what the "=int" part stands for?

A type var default (not currently supported by type vars).

I think it might be easier if the new syntax does not add any new features to type vars, at least initially.

...

Some ideas how we could express upper bounds, constraints, and variance: [...]

Wdyt about <= for defining an upper bound and +/- for variance annotations? IIRC both are used at least in Scala and OCaml has +/- for specifying variance and </> for polymorphic variants <https://ocaml.org/manual/polyvariant.html>.

This could work, but I'm not too fond of using too many punctuation characters. Python has always been the antithesis to "line-noise" Perl, and I think "key words" (not necessarily "keywords" from a parser perspective) are more readable and obvious.

Fair enough. C# and Kotlin <https://kotlinlang.org/docs/generics.html> have nice syntax for variance annotations via in/out keywords. Apart from being more "readable", I also found them to be more instructive when debugging variance-related type errors. Sergei

I'm -1 on the specific of the proposal because "let" has a different semantic meaning in most languages (it's a general way of declaring a variable), and "in" already has a different semantic meaning in Python (it is an operator that implies a containment check). But you may be onto something. We want TypeVars to have well-defined scopes that are clearly delineated within the code. What if we modeled TypeVars as context managers? This would be similar to your proposal except that it would use "with / as" instead of "let / in". I think that "with / as" will feel more natural to most Python users. Here's what this would look like: ```python from types import TracebackType from typing import Any, Generic, Protocol, TypeVar from typing_extensions import Self class _TypeVar: def __init__(self, __name: str, *, bound: Any = None): self._name = __name def __enter__(self): return TypeVar(self._name) def __exit__( self, __exc_type: type[BaseException] | None, __exc_val: BaseException | None, __exc_tb: TracebackType | None, ) -> None: pass class SupportsSum(Protocol): def __add__(self, __other: object) -> Self: ... with _TypeVar("T", bound=SupportsSum) as T: class Adder(Generic[T]): def add(self, a: T, b: T) -> T: return a + b with _TypeVar("T") as T: with _TypeVar("U") as U: def two_tvars(x: T, y: U) -> T | U: return x if x else y Adder[int]().add(1, 2) two_tvars(1, "hi") ``` This is not only syntactically legal (and backward compatible!), but it's also semantically correct at runtime, and it runs without any issues. Both mypy and pyright generate errors, but they could easily be taught that this form is valid. So this would have all the advantages of your proposal plus it could be introduced with no grammar changes, no runtime changes, and full backward compatibility to all supported versions of Python.

One disadvantage is that let...in opens a new scope without adding an indentation level, which is weird for python. Decorators are nice because their scoping is already well understood. martin On Tue, Apr 5, 2022 at 8:55 PM Jelle Zijlstra <jelle.zijlstra@gmail.com> wrote:

I have another alternative idea. It will look like this:

let T = TypeVar("T") in class list(Generic[T]): def append(self, obj: T, /) -> None: ...

let T = TypeVar("T") in let U = TypeVar("U") in def two_tvars(x: T, y: U) -> T | U: ...

SimpleGenerator: TypeAlias = let T = TypeVar("T") in Generator[T, None, None]

def make_identity_func() -> let T = TypeVar("T") in Callable[[T], T]: ...

`let` would be a new soft keyword. The syntax would be available in all expressions and in class and def statements.

At runtime, the first example would be equivalent to something like:

T = TypeVar("T") class list(Generic[T]): def append(self, obj: T, /) -> None: ... # allow runtime introspection of parameters list.__set_parameters__({"T": T}) del T

Advantages: * Less new syntax, so it is easier to get into the language and easier to extend with new flavors of type variables that we may introduce in the future. * The new syntax is potentially useful outside of typing. * Allows a way to explicitly scope type variables to part of a type or to a type alias.

Disadvantages: * Less elegant syntax for generic classes (the flipside of it being more general). * Does not remove the redundancy in T = TypeVar("T")

_______________________________________________ Typing-sig mailing list -- typing-sig@python.org To unsubscribe send an email to typing-sig-leave@python.org https://mail.python.org/mailman3/lists/typing-sig.python.org/ Member address: mdemello@google.com

I don’t think proposals that use TypeVar(“T”) offer enough improvement over the status quo to be worth the tooling changes. On Sun, Apr 17, 2022 at 03:54 David Foster <davidfstr@gmail.com> wrote:

Here's another notation idea:

``` def two_tvars[T := TypeVar("T"), U := TypeVar("U")](x: T, y: U) -> T | U: return x if x else y ```

Here (1) the walrus operator is used to bring T and U into scope, and (2) a def expression allows an arbitrary indexed expression directly after the function name, whose value is discarded at runtime.

Pros: * Minimum of new syntax introduced. * And this syntax *might* be useful for non-typing use cases (which seems to be a general consideration for approving syntax changes).

Cons: * It's verbose. * Still has noise related to having T and U mentioned twice it its TypeVar definition. * Leaves TypeVars (like T and U) in scope after the function definition.

-- David Foster | Seattle, WA, USA Contributor to TypedDict, mypy, and Python's typing system _______________________________________________ Typing-sig mailing list -- typing-sig@python.org To unsubscribe send an email to typing-sig-leave@python.org https://mail.python.org/mailman3/lists/typing-sig.python.org/ Member address: guido@python.org

-- --Guido (mobile)