Let me look at this from the other end. Suppose we have a variadic class Tensor and some tensor types:
```
Ts = TypeTupleVar("Ts")
class Tensor(Generic[*Ts]): ...
class Width: pass
class Height: pass
class Dim: pass
A = Tensor[Width, Height]
B = Tensor[Height, Width]
C = Tensor[Dim, Width, Height]
```
This Tensor class is not a subclass of Sequence and has no homogeneous "fallback" type. A and B are incompatible, and the join of A and C is either their union or object. There is no concept of Tensor[T, ...] in this case.

If you wanted to use e.g. Tensor[Dim, ...] for some universal dimension, you could change the Tensor class to this:
```
class Tensor(Sequence[Union[*Ts]], Generic[*Ts]): ...
def f(seq: Sequence[T]) -> T: ...
t1: Tensor[Dim, Dim]
i1 = f(t1)
t2: Tensor[Width, Height]
i2 = f(t2)
```
In the call to f(t1), t1 is converted to its fallback class Sequence[Union[Dim, Dim]], i.e. Sequence[Dim], and the type of i1 is Dim. In the call to f(t2), we end up with i2 having type Union[Width, Height], or perhaps object (depending on how the checker implements join()).

Note that in the first example, there is no way to even express the type of Tensor[T, ...], whereas in the second example, it is expressed as Sequence[T].

From all this I also realize that there is one thing that remains special about tuples: the notation Tuple[T, ...] is not supported by any other type. So the specialness that Eric wants to convey using arbitrary_len=True is exactly that. IOW we could write this:
```
t3: Tensor[Dim, ...]
i3 = f(t3)
```
Here t3's type is converted from Tensor[Dim, ...] to Sequence[Dim] in the call, and the type of i3 is Dim.

Formulating the rule for the specification for arbitrary_len=True is relatively simple: if a class C inherits from Generic[*Ts] and Ts has arbitrary_len=True, then the class can be parameterized as C[T, ...], and if arbitrary_len=False that notation is not available. However, what is the fallback type? Why would Sequence be special?

With the Sequence[Union[*Ts]] notation it is a bit murkier to write the rule for when C[T, ...] is allowed (maybe it's just "whenever there's another base class that uses *Ts" ?), but the fallback type is precisely specified in the class definition., and we could have some other generic type (e.g. LinkedList[Union[*Ts]] or possibly Tuple[*Ts]). I like that aspect.

OT bikeshed: having typed it half a dozen times no, I can testify that arbitrary_len is a very awkward thing to type. And why not arbitrary_length? :-)

On Fri, Dec 11, 2020 at 11:45 AM Matthew Rahtz via Typing-sig <typing-sig@python.org> wrote:
Let me check whether I understand:
1. tuple currently inherits from Sequence[T], where T is a single type.
2. That's a problem, considering that tuple is variadic. To deal with types like Tuple[int, str], type checkers have to special-case the conversion from the e.g. two types specified to the type of the Sequence (e.g. with a Union).
3. If we introduced variadic type variables, we could get rid of that special-casing - either because the conversion from multiple types to a single type is specified explicitly (as in Sequence[Union[*Ts]]), or because the conversion is an explicit part of the specification (as in Sequence[Ts]).
If this is right, my preference would be for making the conversion from a list of types to a single type explicit, as in Union[*Ts]. I think that preference is mostly based on the intuition that it's usually better to be explicit, but I'd be curious to hear arguments the other way.

---

We could extend variadic (list) type variables to accept an optional parameter called `arbitrary_len` that indicates whether a variadic type variable supports homogenous, arbitrary-length forms. By default, variadic type variables wouldn't allow this. We'd need to make it illegal to use type variables that support arbitrary lengths with an unpack operator or `Expand` because the result would be undefined.

I was thinking that variadic type variables would be arbitrary-length (and heterogeneous) by default. Is the point that unless we enforce some restrictions here, we can't properly define how to handle the conversion to a single type?

Wouldn't the following work, even without those restrictions?

Ts = TypeVar('Ts', list=True)  # Arbitrary-length, homogeneous or heterogeneous

class tuple(Generic[*Ts], Sequence[Union[*Ts]]): ...

x: tuple[int]       # Ts is bound to [int]; x is a Sequence[int]
y: tuple[int, str]  # Ts is bound to [int, str]; y is a Sequence[Union[int, str]]
z: tuple[int, ...]  # Ts is bound to [int, ...]; z is a Sequence[int]

Hmm, maybe this is equivalent to your proposal - except here, the "work" is done by the Union.

Oh, I think I see what you're saying. So variadic type variables would be arbitrary-length in that they can be bound to a fixed-size type list of any size, but they're not arbitrary length in that they can't be bound to an arbitrary-sized type list [int, ...]. Is that right?

OK, so it seems like the crux of the matter is whether [int, ...] is a valid thing for a variadic type variable to be bound to. Because in order to define tuple using a variadic type variable, [int, ...] would have to be a valid thing for a variadic type variable to be bound to.

Alright, I'll pause here to check whether I've understood so far :)

On Fri, 11 Dec 2020 at 00:24, Eric Traut <eric@traut.com> wrote:
In your example above, you asked "What is the type of b?". I agree that it should be `Union[int, str]`. That's consistent with pyright's current behavior.

Here are a few thoughts about how we could generalize variadic type variables to work with tuple.

We could extend variadic (list) type variables to accept an optional parameter called `arbitrary_len` that indicates whether variadic type variable supports homogenous, arbitrary-length forms. By default, variadic type variables wouldn't allow this. We'd need to make it illegal to use type variables that support arbitrary lengths with an unpack operator or `Expand` because the result would be undefined.

For type variables that support arbitrary length lists, we could define some simple rules for how they "collapse" to traditional (non-list) types.
1. For a variadic TypeVar that is bound to a homogenous, arbitrary-length list, the type list [T, ...] collapses to a type of T.
2. For a variadic TypeVar that is bound to a heterogenous, fixed-length list, the type list collapses to a union of the individual subtypes with literals stripped. For example, [Literal['a'], int] collapses to Union[str, int]. For an explanation of why it's important to strip literals in this case, refer to [this discussion](https://github.com/microsoft/pyright/issues/1249).

This would allow us to define the `tuple` class and its constructor as follows:
```python
_T = TypeVar("_T")
_Ts = TypeVar("_Ts", list=True, arbitrary_len=True)

class tuple(Sequence[_Ts]):
def __new__(cls: Type[_T], iterable: Tuple[_Ts] = ...) -> _T:
...

def __new__(cls: Type[_T], iterable: Iterable[_Ts] = ...) -> _T:
...
```

Thoughts?

--
Eric Traut
Contributor to pyright/pylance
_______________________________________________
Typing-sig mailing list -- typing-sig@python.org
To unsubscribe send an email to typing-sig-leave@python.org
https://mail.python.org/mailman3/lists/typing-sig.python.org/