Let me look at this from the other end. Suppose we have a variadic class Tensor and some tensor types:

```

Ts = TypeTupleVar("Ts")

class Tensor(Generic[*Ts]): ...

class Width: pass

class Height: pass

class Dim: pass

A = Tensor[Width, Height]

B = Tensor[Height, Width]

C = Tensor[Dim, Width, Height]

```

This Tensor class is not a subclass of Sequence and has no homogeneous "fallback" type. A and B are incompatible, and the join of A and C is either their union or object. There is no concept of Tensor[T, ...] in this case.

If you wanted to use e.g. Tensor[Dim, ...] for some universal dimension, you could change the Tensor class to this:

```

class Tensor(Sequence[Union[*Ts]], Generic[*Ts]): ...

def f(seq: Sequence[T]) -> T: ...

t1: Tensor[Dim, Dim]

i1 = f(t1)

t2: Tensor[Width, Height]

i2 = f(t2)

```

In the call to f(t1), t1 is converted to its fallback class Sequence[Union[Dim, Dim]], i.e. Sequence[Dim], and the type of i1 is Dim. In the call to f(t2), we end up with i2 having type Union[Width, Height], or perhaps object (depending on how the checker implements join()).

Note that in the first example, there is no way to even express the type of Tensor[T, ...], whereas in the second example, it is expressed as Sequence[T].

From all this I also realize that there is one thing that remains special about tuples: the notation Tuple[T, ...] is not supported by any other type. So the specialness that Eric wants to convey using arbitrary_len=True is exactly that. IOW we could write this:

```

t3: Tensor[Dim, ...]

i3 = f(t3)

```

Here t3's type is converted from Tensor[Dim, ...] to Sequence[Dim] in the call, and the type of i3 is Dim.

Formulating the rule for the specification for arbitrary_len=True is relatively simple: if a class C inherits from Generic[*Ts] and Ts has arbitrary_len=True, then the class can be parameterized as C[T, ...], and if arbitrary_len=False that notation is not available. However, what is the fallback type? Why would Sequence be special?

With the Sequence[Union[*Ts]] notation it is a bit murkier to write the rule for when C[T, ...] is allowed (maybe it's just "whenever there's another base class that uses *Ts" ?), but the fallback type is precisely specified in the class definition., and we could have some other generic type (e.g. LinkedList[Union[*Ts]] or possibly Tuple[*Ts]). I like that aspect.

OT bikeshed: having typed it half a dozen times no, I can testify that arbitrary_len is a very awkward thing to type. And why not arbitrary_length? :-)

On Fri, Dec 11, 2020 at 11:45 AM Matthew Rahtz via Typing-sig <typing-sig@python.org> wrote:

Let me check whether I understand:
tuple currently inherits from Sequence[T], where T is a single type.
That's a problem, considering that tuple is variadic. To deal with types like Tuple[int, str], type checkers have to special-case the conversion from the e.g. two types specified to the type of the Sequence (e.g. with a Union).
If we introduced variadic type variables, we could get rid of that special-casing - either because the conversion from multiple types to a single type is specified explicitly (as in Sequence[Union[*Ts]]), or because the conversion is an explicit part of the specification (as in Sequence[Ts]).
If this is right, my preference would be for making the conversion from a list of types to a single type explicit, as in Union[*Ts]. I think that preference is mostly based on the intuition that it's usually better to be explicit, but I'd be curious to hear arguments the other way.

---

Eric - to make sure we're on the same page about variadic type variables in general - I was a little confused about this:

We could extend variadic (list) type variables to accept an optional parameter called `arbitrary_len` that indicates whether a variadic type variable supports homogenous, arbitrary-length forms. By default, variadic type variables wouldn't allow this. We'd need to make it illegal to use type variables that support arbitrary lengths with an unpack operator or `Expand` because the result would be undefined.

I was thinking that variadic type variables would be arbitrary-length (and heterogeneous) by default. Is the point that unless we enforce some restrictions here, we can't properly define how to handle the conversion to a single type?

Wouldn't the following work, even without those restrictions?

Ts = TypeVar('Ts', list=True) # Arbitrary-length, homogeneous or heterogeneous

class tuple(Generic[*Ts], Sequence[Union[*Ts]]): ...

x: tuple[int] # Ts is bound to [int]; x is a Sequence[int]
y: tuple[int, str] # Ts is bound to [int, str]; y is a Sequence[Union[int, str]]
z: tuple[int, ...] # Ts is bound to [int, ...]; z is a Sequence[int]

Hmm, maybe this is equivalent to your proposal - except here, the "work" is done by the Union.

Oh, I think I see what you're saying. So variadic type variables would be arbitrary-length in that they can be bound to a fixed-size type list of any size, but they're not arbitrary length in that they can't be bound to an arbitrary-sized type list [int, ...]. Is that right?

OK, so it seems like the crux of the matter is whether [int, ...] is a valid thing for a variadic type variable to be bound to. Because in order to define tuple using a variadic type variable, [int, ...] would have to be a valid thing for a variadic type variable to be bound to.

Alright, I'll pause here to check whether I've understood so far :)

On Fri, 11 Dec 2020 at 00:24, Eric Traut <eric@traut.com> wrote:
In your example above, you asked "What is the type of b?". I agree that it should be `Union[int, str]`. That's consistent with pyright's current behavior.

Here are a few thoughts about how we could generalize variadic type variables to work with tuple.

We could extend variadic (list) type variables to accept an optional parameter called `arbitrary_len` that indicates whether variadic type variable supports homogenous, arbitrary-length forms. By default, variadic type variables wouldn't allow this. We'd need to make it illegal to use type variables that support arbitrary lengths with an unpack operator or `Expand` because the result would be undefined.

For type variables that support arbitrary length lists, we could define some simple rules for how they "collapse" to traditional (non-list) types.
1. For a variadic TypeVar that is bound to a homogenous, arbitrary-length list, the type list [T, ...] collapses to a type of T.
2. For a variadic TypeVar that is bound to a heterogenous, fixed-length list, the type list collapses to a union of the individual subtypes with literals stripped. For example, [Literal['a'], int] collapses to Union[str, int]. For an explanation of why it's important to strip literals in this case, refer to [this discussion](https://github.com/microsoft/pyright/issues/1249).

This would allow us to define the `tuple` class and its constructor as follows:
```python
_T = TypeVar("_T")
_Ts = TypeVar("_Ts", list=True, arbitrary_len=True)

class tuple(Sequence[_Ts]):
@overload
def __new__(cls: Type[_T], iterable: Tuple[_Ts] = ...) -> _T:
...

@overload
def __new__(cls: Type[_T], iterable: Iterable[_Ts] = ...) -> _T:
...
```

Thoughts?

--
Eric Traut
Contributor to pyright/pylance
_______________________________________________
Typing-sig mailing list -- typing-sig@python.org
To unsubscribe send an email to typing-sig-leave@python.org
https://mail.python.org/mailman3/lists/typing-sig.python.org/
Member address: mrahtz@google.com

_______________________________________________
Typing-sig mailing list -- typing-sig@python.org
To unsubscribe send an email to typing-sig-leave@python.org
https://mail.python.org/mailman3/lists/typing-sig.python.org/
Member address: guido@python.org