Let me look at this from the other end. Suppose we have a variadic class Tensor and some tensor types: ``` Ts = TypeTupleVar("Ts") class Tensor(Generic[*Ts]): ... class Width: pass class Height: pass class Dim: pass A = Tensor[Width, Height] B = Tensor[Height, Width] C = Tensor[Dim, Width, Height] ``` This Tensor class is not a subclass of Sequence and has no homogeneous "fallback" type. A and B are incompatible, and the join of A and C is either their union or object. There is no concept of Tensor[T, ...] in this case.

If you wanted to use e.g. Tensor[Dim, ...] for some universal dimension, you could change the Tensor class to this: ``` class Tensor(Sequence[Union[*Ts]], Generic[*Ts]): ... def f(seq: Sequence[T]) -> T: ... t1: Tensor[Dim, Dim] i1 = f(t1) t2: Tensor[Width, Height] i2 = f(t2) ``` In the call to f(t1), t1 is converted to its fallback class Sequence[Union[Dim, Dim]], i.e. Sequence[Dim], and the type of i1 is Dim. In the call to f(t2), we end up with i2 having type Union[Width, Height], or perhaps object (depending on how the checker implements join()).

Note that in the first example, there is no way to even express the type of Tensor[T, ...], whereas in the second example, it is expressed as Sequence[T].

From all this I also realize that there is one thing that remains special about tuples: the notation Tuple[T, ...] is not supported by any other type. So the specialness that Eric wants to convey using arbitrary_len=True is exactly that. IOW we could write this: ``` t3: Tensor[Dim, ...] i3 = f(t3) ``` Here t3's type is converted from Tensor[Dim, ...] to Sequence[Dim] in the call, and the type of i3 is Dim.

Formulating the rule for the specification for arbitrary_len=True is relatively simple: if a class C inherits from Generic[*Ts] and Ts has arbitrary_len=True, then the class can be parameterized as C[T, ...], and if arbitrary_len=False that notation is not available. However, what is the fallback type? Why would Sequence be special?

With the Sequence[Union[*Ts]] notation it is a bit murkier to write the rule for when C[T, ...] is allowed (maybe it's just "whenever there's another base class that uses *Ts" ?), but the fallback type is precisely specified in the class definition., and we could have some other generic type (e.g. LinkedList[Union[*Ts]] or possibly Tuple[*Ts]). I like that aspect.

OT bikeshed: having typed it half a dozen times no, I can testify that arbitrary_len is a very awkward thing to type. And why not arbitrary_length? :-)

On Fri, Dec 11, 2020 at 11:45 AM Matthew Rahtz via Typing-sig < typing-sig@python.org> wrote:

Let me check whether I understand:

- tuple currently inherits from Sequence[T], where T is a single type.
- That's a problem, considering that tuple is variadic. To deal with
types like Tuple[int, str], type checkers have to special-case the conversion from the e.g. two types specified to the type of the Sequence (e.g. with a Union). 3. If we introduced variadic type variables, we could get rid of that special-casing - either because the conversion from multiple types to a single type is specified explicitly (as in Sequence[Union[*Ts]]), or because the conversion is an explicit part of the specification (as in Sequence[Ts]).

If this is right, my preference would be for making the conversion from a list of types to a single type explicit, as in Union[*Ts]. I think that preference is mostly based on the intuition that it's usually better to be explicit, but I'd be curious to hear arguments the other way.

Eric - to make sure we're on the same page about variadic type variables in general - I was a little confused about this:

We could extend variadic (list) type variables to accept an optional

parameter called `arbitrary_len` that indicates whether a variadic type variable supports homogenous, arbitrary-length forms. *By default, variadic type variables wouldn't allow this.* *We'd need to make it illegal to use type variables that support arbitrary lengths with an unpack operator or `Expand` because the result would be undefined.*

I was thinking that variadic type variables *would* be arbitrary-length (and heterogeneous) by default. Is the point that unless we enforce some restrictions here, we can't properly define how to handle the conversion to a single type?

Wouldn't the following work, even without those restrictions?

Ts = TypeVar('Ts', list=True) # Arbitrary-length, homogeneous or heterogeneous

class tuple(Generic[*Ts], Sequence[Union[*Ts]]): ...

x: tuple[int] # Ts is bound to [int]; x is a Sequence[int] y: tuple[int, str] # Ts is bound to [int, str]; y is a Sequence[Union[int, str]] z: tuple[int, ...] # Ts is bound to [int, ...]; z is a Sequence[int]

Hmm, maybe this is equivalent to your proposal - except here, the "work" is done by the Union.

Oh, I think I see what you're saying. So variadic type variables would be arbitrary-length in that they can be bound to a fixed-size type list of any size, but they're *not* arbitrary length in that they *can't* be bound to an arbitrary-sized type list [int, ...]. Is that right?

OK, so it seems like the crux of the matter is whether [int, ...] is a valid thing for a variadic type variable to be bound to. Because in order to define tuple using a variadic type variable, [int, ...] *would* have to be a valid thing for a variadic type variable to be bound to.

Alright, I'll pause here to check whether I've understood so far :)

On Fri, 11 Dec 2020 at 00:24, Eric Traut eric@traut.com wrote:

In your example above, you asked "What is the type of b?". I agree that it should be `Union[int, str]`. That's consistent with pyright's current behavior.

Here are a few thoughts about how we could generalize variadic type variables to work with tuple.

We could extend variadic (list) type variables to accept an optional parameter called `arbitrary_len` that indicates whether variadic type variable supports homogenous, arbitrary-length forms. By default, variadic type variables wouldn't allow this. We'd need to make it illegal to use type variables that support arbitrary lengths with an unpack operator or `Expand` because the result would be undefined.

For type variables that support arbitrary length lists, we could define some simple rules for how they "collapse" to traditional (non-list) types.

- For a variadic TypeVar that is bound to a homogenous, arbitrary-length
list, the type list [T, ...] collapses to a type of T. 2. For a variadic TypeVar that is bound to a heterogenous, fixed-length list, the type list collapses to a union of the individual subtypes with literals stripped. For example, [Literal['a'], int] collapses to Union[str, int]. For an explanation of why it's important to strip literals in this case, refer to [this discussion]( https://github.com/microsoft/pyright/issues/1249).

This would allow us to define the `tuple` class and its constructor as follows:

`_T = TypeVar("_T") _Ts = TypeVar("_Ts", list=True, arbitrary_len=True) class tuple(Sequence[_Ts]): @overload def __new__(cls: Type[_T], iterable: Tuple[_Ts] = ...) -> _T: ... @overload def __new__(cls: Type[_T], iterable: Iterable[_Ts] = ...) -> _T: ...`

Thoughts?

-- Eric Traut Contributor to pyright/pylance _______________________________________________ Typing-sig mailing list -- typing-sig@python.org To unsubscribe send an email to typing-sig-leave@python.org https://mail.python.org/mailman3/lists/typing-sig.python.org/ Member address: mrahtz@google.com

Typing-sig mailing list -- typing-sig@python.org To unsubscribe send an email to typing-sig-leave@python.org https://mail.python.org/mailman3/lists/typing-sig.python.org/ Member address: guido@python.org