Mailman 3 PEP 646: Revisiting binding to arbitrary-length types - Typing-sig

newer
Next tensor typing meeting: Monday...

PEP 646: Revisiting binding to arbitrary-length types

older
Opened and changed typing issues...

Matthew Rahtz

16 Nov 2021 16 Nov '21

7:35 p.m.

(Not sure whether typing-sig or the Issues of the GitHub typing repository is the best place for this discussion; posting it here for the time being to make sure it reaches all the right people.) While thinking through the implications of making PEP 646's typing of *args more flexible in https://github.com/python/peps/pull/2125, we realised we might have made a mistake in restricting type variable tuples from being bound to arbitrary-length types. As a refresher, PEP 646 currently disallows (in the 'Type Variable Tuples Must Have Known Length' section) the following: Ts = TypeVarTuple('Ts') def foo(x: tuple[*Ts]): ... x: tuple[float, ...] foo(x) # Not allowed, as Ts would be bound to tuple[float, ...] def Tensor(Generic[*Ts]): ... Tensor[*tuple[Any, ...]] # Not allowed, as Ts would be be bound to tuple[Any, ...] From what I remember, we disallowed this mainly for the following two reasons: 1. The natural syntax for making use of it would be e.g. Tensor[Any, ...] - but I wanted to reserve the meaning of this syntax for a later PEP, where we would say that Tensor[Foo, ...] would mean "Foo followed by an arbitrary number of type parameters of arbitrary types", rather than "An arbitrary number of type parameters, all type Foo". 2. There were some complications around how Union would behave (e.g. see this post https://mail.python.org/archives/list/typing-sig@python.org/message/25INUZ5K... from Eric Traut). We actually now realise we can work around reason 1 by simply making the syntax Tensor[*Tuple[Any, ...]]. And 2 is no longer an issue because we removed support for Union[*Ts] from the PEP for simplicity. Can anyone think of any other reason we shouldn't allow type variable tuples being bound to arbitrary-length types? Eric, we'd be particularly interested to hear your thoughts here. In the post linked above you say "We'd need to make it illegal to use type variables that support arbitrary lengths with an unpack operator or `Expand` because the result would be undefined." - can you remember what you meant by that? Thanks! Matthew and Pradeep

Attachments:

attachment.htm (text/html — 2.7 KB)

Show replies by date

Eric Traut

19 Nov 19 Nov

5:03 a.m.

I'm struggling to remember precisely what I was trying to express in that post. It was over a year ago, after all. :) Thinking about it now, it seems like `Unpack` should work OK with arbitrary-length types. There are some interesting edge cases to consider though. With a TypeVarTuple of known length, it's possible to do define a type like this: `tuple[int, *Ts, int]`. If `Ts` is then bound to `tuple[str, str]`, the resulting type is tuple[int, str, str, int]`. But if `Ts` is bound to an arbitrary-length tuple like `tuple[str, ...]`, what is the resulting type? Is it `tuple[int, str, ..., int]`? That's not a legal type, at least not today. Perhaps we can fix this by specifying that `*Ts` is not allowed for arbitrary-length tuples except in the last (right-most) type argument position? Or are you thinking that the type would be `tuple[int, *tuple[str, ...], int]`? That's pretty ugly. It would allow for arbitrary nesting (e.g. `tuple[int, *tuple[int, *tuple[str, ...], int], int]`). Pyright already has about a dozen places where it needs to special-case tuple unpacking and handle both arbitrary-length and fixed-length tuples. This would add significant complexity if every one of those places would need to be able to deal with nesting of unpacked ranges, some of them of arbitrary length. I don't think it would be a good idea to allow such a type. The interactions between unpacked TypeVarTuple and `Callable` also become more complex if we allow arbitrary-length tuples, but this is probably no more complex than `*args` today. I didn't realize that you had removed `Union[*Ts]` from the draft PEP. I've already implemented support for this in pyright. It's easy enough to remove, but at the same time, it didn't strike me as one of the more difficult parts of supporting TypeVarTuple. I also don't think that `Union[*Ts]` presents a problem if `Ts` is bound to an arbitrary-length tuple. Have you thought about how arbitrary-length tuple bindings would work with some of the type arithmetic operators that were recently proposed for working with dimensional transforms? Would those operators be undefined and therefore illegal if used with an arbitrary-length tuple binding? I know that Matthew would like to reserve the syntax `Tensor[Any, ...]` to mean something different in the future (point 1 above). My preference is to simply adopt the convention already established by `tuple`. The syntax `Tensor[*Tuple[Any, ...]]` is cumbersome and introduces added complexity as I noted above. Plus, the meaning of `tuple[Any, ...]` is already well established and broadly understood, so it's natural to assume that `Tensor[Any, ...]` has the same semantics. If we find in the future that there is really a need to specify "an arbitrary number of type parameters with arbitrary types", we could solve it in some other manner — one that doesn't conflict with existing conventions. -- Eric Traut Contributor to Pyright & Pylance Microsoft

Guido van Rossum

5:41 a.m.

I may sound like a broken record, but I feel compelled to point out that even though `tuple[int, str, ..., int]` is not legal *syntax* today, and there is currently no way to cause such a type to exist, we could have the concept (inside type checkers) without having syntax to spell it. I'm guessing the reason that Matthew rather wants `Tensor[int, ..., int]` to mean what you would spell as `Tensor[int, Any, ..., int]`, is that `a[i, ..., j]` is used to take a slice from a numpy array `a` that drops the first and last dimension. Note that if we were to accept your syntax, an edge case of `Tensor[int, str, ..., int]` would be `Tensor[int, int]` (i.e. the `str, ...` collapses to zero dimensions), as it does for `tuple[int, ...]`. On Thu, Nov 18, 2021 at 9:04 PM Eric Traut wrote:

...

I'm struggling to remember precisely what I was trying to express in that post. It was over a year ago, after all. :)

Thinking about it now, it seems like `Unpack` should work OK with arbitrary-length types. There are some interesting edge cases to consider though.

With a TypeVarTuple of known length, it's possible to do define a type like this: `tuple[int, *Ts, int]`. If `Ts` is then bound to `tuple[str, str]`, the resulting type is tuple[int, str, str, int]`. But if `Ts` is bound to an arbitrary-length tuple like `tuple[str, ...]`, what is the resulting type? Is it `tuple[int, str, ..., int]`? That's not a legal type, at least not today. Perhaps we can fix this by specifying that `*Ts` is not allowed for arbitrary-length tuples except in the last (right-most) type argument position? Or are you thinking that the type would be `tuple[int, *tuple[str, ...], int]`? That's pretty ugly. It would allow for arbitrary nesting (e.g. `tuple[int, *tuple[int, *tuple[str, ...], int], int]`). Pyright already has about a dozen places where it needs to special-case tuple unpacking and handle both arbitrary-length and fixed-length tuples. This would add significant complexity if every one of those places would need to be able to deal with nesting of unpacked ranges, some of them of arbitrary length. I don't think it would be a good idea to allow such a type.

The interactions between unpacked TypeVarTuple and `Callable` also become more complex if we allow arbitrary-length tuples, but this is probably no more complex than `*args` today.

I didn't realize that you had removed `Union[*Ts]` from the draft PEP. I've already implemented support for this in pyright. It's easy enough to remove, but at the same time, it didn't strike me as one of the more difficult parts of supporting TypeVarTuple. I also don't think that `Union[*Ts]` presents a problem if `Ts` is bound to an arbitrary-length tuple.

Have you thought about how arbitrary-length tuple bindings would work with some of the type arithmetic operators that were recently proposed for working with dimensional transforms? Would those operators be undefined and therefore illegal if used with an arbitrary-length tuple binding?

I know that Matthew would like to reserve the syntax `Tensor[Any, ...]` to mean something different in the future (point 1 above). My preference is to simply adopt the convention already established by `tuple`. The syntax `Tensor[*Tuple[Any, ...]]` is cumbersome and introduces added complexity as I noted above. Plus, the meaning of `tuple[Any, ...]` is already well established and broadly understood, so it's natural to assume that `Tensor[Any, ...]` has the same semantics. If we find in the future that there is really a need to specify "an arbitrary number of type parameters with arbitrary types", we could solve it in some other manner — one that doesn't conflict with existing conventions.

--

Eric Traut Contributor to Pyright & Pylance Microsoft _______________________________________________ Typing-sig mailing list -- typing-sig@python.org To unsubscribe send an email to typing-sig-leave@python.org https://mail.python.org/mailman3/lists/typing-sig.python.org/ Member address: guido@python.org

-- --Guido van Rossum (python.org/~guido) *Pronouns: he/him **(why is my pronoun here?)* http://feministing.com/2015/02/03/how-using-they-as-a-singular-pronoun-can-c...

S Pradeep Kumar

10:34 a.m.

Eric: Firstly, Pyre has always handled unpacking unbounded tuples. So, if you're curious, you could look at the Python test cases [1]. 1.

...

(paraphrasing) Can we substitute `Tuple[int, ...]` for Ts anywhere?

...

Have you thought about how arbitrary-length tuple bindings would work with some of the type arithmetic operators that were recently proposed for working with dimensional transforms? Would those operators be undefined and

Yes. Fwiw, I did not run into much difficulty with this since the data type definition is recursive. A tuple is either concrete, unbounded, or contains an "unpacked item" with a prefix and suffix. The "unpacked item" is either a variadic Ts or a tuple itself. So, it wasn't like I had to manually handle different levels of nesting; handling the tuple data type would take care of any recursive uses of tuples. Imposing artificial limitations like "unbounded tuples can only be unpacked at the end" will lead to inconsistent, confusing results. The only consistent implementation is to allow any tuple to be unpacked wherever *Ts is allowed. Your example `tuple[int, *tuple[str, ...], int]` is a meaningful type. It means a tuple that is guaranteed to begin and end with an int and may have any number of `str`s in between. This is *exactly* how TypeScript handles unpacking of unbounded tuples (playground link [2]): ``` declare function firstAndLast(arr: readonly [T, ...string[], R]): [T, R]; const r1 = firstAndLast([1, "foo", "bar", 4]); // OK - 2 strings. const r2 = firstAndLast([1, 4]); // OK - 0 strings. const r3 = firstAndLast([1]); // Not OK - missing trailing value. // How unpacking an unbounded tuple looks: declare function addPrefixSuffix(arr: T): [number, ...T, number]; const r4 = addPrefixSuffix(["foo", "bar"]); // OK - type of r4 is [number, ...string[], number] const r4_error = addPrefixSuffix([1, 2]) // Not OK - expected strings. ``` For example, unbounded tuples are useful when passing `y: Tensor` to a function `def foo(x: Tensor[DType, A, B]) -> None:`. Since `Tensor === Tensor[Any, *tuple[Any, ...]]`, we solve `*tuple[Any, ...]` against `A, B` to get `A = Any, B = Any`. Likewise, if we pass `x: Tensor[float32, *tuple[int, ...], L[42]]` to something expecting `Tensor[float32, A, B, C]`, then we will solve `A = int; B = int; C = L[42]`. (In case you're curious, this generalizes well to future unpackable items like `*Broadcast[A, B]`, `*Map[A, B]`, etc.) 2. therefore illegal if used with an arbitrary-length tuple binding? Yes, Pyre supports unbounded tuples wherever `Ts` can be used. For example, `Broadcast[Tuple[int, ...], Tuple[L[5], L[6]]] === Tuple[int, ...]`. 3.

...

I didn't realize that you had removed `Union[*Ts]` from the draft PEP. I've already implemented support for this in pyright. It's easy enough to remove, but at the same time, it didn't strike me as one of the more difficult parts of supporting TypeVarTuple. I also don't think that `Union[*Ts]` presents a problem if `Ts` is bound to an arbitrary-length tuple.

Yeah, there are no fundamental limitations to adding `Union[*Ts]`. IIRC we'd removed it because it was making the PEP harder to implement with no real use cases to justify it. I'm thinking of introducing it in the follow-up PEP for Map, where we will probably have more use cases. The alternative is to add it right here, but we're already delaying PEP approval because of the unbounded tuples amendment and this will delay it further. By deferring it, you don't have to delete your implementation either. Does that sound reasonable? 4. Guido:

...

I'm guessing the reason that Matthew rather wants `Tensor[int, ..., int]` to mean what you would spell as `Tensor[int, Any, ..., int]`, is that `a[i, ..., j]` is used to take a slice from a numpy array `a` that drops the first and last dimension.

If we want to add user-friendly syntax like `Tensor[float32, ...]` as syntactic sugar for `Tensor[float32, *tuple[Any, ...]]`, I'm ok with doing that in the future (if there is user demand). Likewise for `Tensor[float32, ..., str]`, which would be equivalent to `Tensor[float32, *tuple[Any, ...], str]`. But, as Guido pointed out, this lack of syntax shouldn't constrain our semantics. [1]: https://github.com/facebook/pyre-check/blob/c185dc1cb44e86dc60c1feea777fdb95... [2]: https://www.typescriptlang.org/play?ts=4.5.0-beta#code/CYUwxgNghgTiAEAzArgOz... On Thu, Nov 18, 2021 at 9:42 PM Guido van Rossum wrote:

...

I may sound like a broken record, but I feel compelled to point out that even though `tuple[int, str, ..., int]` is not legal *syntax* today, and there is currently no way to cause such a type to exist, we could have the concept (inside type checkers) without having syntax to spell it.

I'm guessing the reason that Matthew rather wants `Tensor[int, ..., int]` to mean what you would spell as `Tensor[int, Any, ..., int]`, is that `a[i, ..., j]` is used to take a slice from a numpy array `a` that drops the first and last dimension.

Note that if we were to accept your syntax, an edge case of `Tensor[int, str, ..., int]` would be `Tensor[int, int]` (i.e. the `str, ...` collapses to zero dimensions), as it does for `tuple[int, ...]`.

On Thu, Nov 18, 2021 at 9:04 PM Eric Traut wrote:

...
I'm struggling to remember precisely what I was trying to express in that post. It was over a year ago, after all. :)

Thinking about it now, it seems like `Unpack` should work OK with arbitrary-length types. There are some interesting edge cases to consider though.

With a TypeVarTuple of known length, it's possible to do define a type like this: `tuple[int, *Ts, int]`. If `Ts` is then bound to `tuple[str, str]`, the resulting type is tuple[int, str, str, int]`. But if `Ts` is bound to an arbitrary-length tuple like `tuple[str, ...]`, what is the resulting type? Is it `tuple[int, str, ..., int]`? That's not a legal type, at least not today. Perhaps we can fix this by specifying that `*Ts` is not allowed for arbitrary-length tuples except in the last (right-most) type argument position? Or are you thinking that the type would be `tuple[int, *tuple[str, ...], int]`? That's pretty ugly. It would allow for arbitrary nesting (e.g. `tuple[int, *tuple[int, *tuple[str, ...], int], int]`). Pyright already has about a dozen places where it needs to special-case tuple unpacking and handle both arbitrary-length and fixed-length tuples. This would add significant complexity if every one of those places would need to be able to deal with nesting of unpacked ranges, some of them of arbitrary length. I don't think it would be a good idea to allow such a type.

The interactions between unpacked TypeVarTuple and `Callable` also become more complex if we allow arbitrary-length tuples, but this is probably no more complex than `*args` today.

I didn't realize that you had removed `Union[*Ts]` from the draft PEP. I've already implemented support for this in pyright. It's easy enough to remove, but at the same time, it didn't strike me as one of the more difficult parts of supporting TypeVarTuple. I also don't think that `Union[*Ts]` presents a problem if `Ts` is bound to an arbitrary-length tuple.

Have you thought about how arbitrary-length tuple bindings would work with some of the type arithmetic operators that were recently proposed for working with dimensional transforms? Would those operators be undefined and therefore illegal if used with an arbitrary-length tuple binding?

I know that Matthew would like to reserve the syntax `Tensor[Any, ...]` to mean something different in the future (point 1 above). My preference is to simply adopt the convention already established by `tuple`. The syntax `Tensor[*Tuple[Any, ...]]` is cumbersome and introduces added complexity as I noted above. Plus, the meaning of `tuple[Any, ...]` is already well established and broadly understood, so it's natural to assume that `Tensor[Any, ...]` has the same semantics. If we find in the future that there is really a need to specify "an arbitrary number of type parameters with arbitrary types", we could solve it in some other manner — one that doesn't conflict with existing conventions.

--

Eric Traut Contributor to Pyright & Pylance Microsoft _______________________________________________ Typing-sig mailing list -- typing-sig@python.org To unsubscribe send an email to typing-sig-leave@python.org https://mail.python.org/mailman3/lists/typing-sig.python.org/ Member address: guido@python.org

-- --Guido van Rossum (python.org/~guido) *Pronouns: he/him **(why is my pronoun here?)* http://feministing.com/2015/02/03/how-using-they-as-a-singular-pronoun-can-c... _______________________________________________ Typing-sig mailing list -- typing-sig@python.org To unsubscribe send an email to typing-sig-leave@python.org https://mail.python.org/mailman3/lists/typing-sig.python.org/ Member address: gohanpra@gmail.com

-- S Pradeep Kumar

Matthew Rahtz

11:25 a.m.

I agree with the points made by Guido and Pradeep. [Eric] Or are you thinking that the type would be `tuple[int, *tuple[str, ...],

...

int]`? That's pretty ugly.

Agreed that it's ugly; this was one of the main reasons I pushed back against it initially. But given my strong desire to be careful about what we commit ellipsis to mean, this is the best option we've been able to come up with so far. [Eric] Have you thought about how arbitrary-length tuple bindings would work with

...

some of the type arithmetic operators that were recently proposed for working with dimensional transforms? Would those operators be undefined and therefore illegal if used with an arbitrary-length tuple binding?

I'm mostly happy with Pradeep's answer of "The Pyre implementation shows this is fine" - but to be on the safe side, what's your intuition about why some things might not work? [Guido/Eric] I'm guessing the reason that Matthew rather wants `Tensor[int, ..., int]`

...

to mean what you would spell as `Tensor[int, Any, ..., int]`, is that `a[i, ..., j]` is used to take a slice from a numpy array `a` that drops the first and last dimension.

Right, this precedent is one reason. The more general reason is that I think of an ellipsis in natural language as meaning "There's a bunch of stuff missing here, but you don't need to worry about what it is". Having `Tensor[int, ..., int]` mean "An int followed by a bunch of types we don't care about followed by an int" seems more compatible with that. It's also that, while I think the meaning of `tuple[int, ...]` isn't too hard to guess for a newcomer, if we said that `tuple[int, str, ..., float]` meant "An int followed by an arbitrary number of str followed by a float" - I think that meaning would be harder to guess. The ellipsis isn't clearly attached to str (unlike in regexp where we'd say something like 'int str* float'), so I think it's not obvious that the ellipsis modifies the str rather than the float - or indeed, modifies either. On Fri, 19 Nov 2021 at 10:35, S Pradeep Kumar wrote:

...

Eric:

Firstly, Pyre has always handled unpacking unbounded tuples. So, if you're curious, you could look at the Python test cases [1].

1.

...
(paraphrasing) Can we substitute `Tuple[int, ...]` for Ts anywhere?

Yes. Fwiw, I did not run into much difficulty with this since the data type definition is recursive. A tuple is either concrete, unbounded, or contains an "unpacked item" with a prefix and suffix. The "unpacked item" is either a variadic Ts or a tuple itself. So, it wasn't like I had to manually handle different levels of nesting; handling the tuple data type would take care of any recursive uses of tuples.

Imposing artificial limitations like "unbounded tuples can only be unpacked at the end" will lead to inconsistent, confusing results. The only consistent implementation is to allow any tuple to be unpacked wherever *Ts is allowed.

Your example `tuple[int, *tuple[str, ...], int]` is a meaningful type. It means a tuple that is guaranteed to begin and end with an int and may have any number of `str`s in between.

This is *exactly* how TypeScript handles unpacking of unbounded tuples (playground link [2]):

``` declare function firstAndLast(arr: readonly [T, ...string[], R]): [T, R];

const r1 = firstAndLast([1, "foo", "bar", 4]); // OK - 2 strings. const r2 = firstAndLast([1, 4]); // OK - 0 strings. const r3 = firstAndLast([1]); // Not OK - missing trailing value.

// How unpacking an unbounded tuple looks:

declare function addPrefixSuffix(arr: T): [number, ...T, number];

const r4 = addPrefixSuffix(["foo", "bar"]); // OK - type of r4 is [number, ...string[], number] const r4_error = addPrefixSuffix([1, 2]) // Not OK - expected strings. ```

For example, unbounded tuples are useful when passing `y: Tensor` to a function `def foo(x: Tensor[DType, A, B]) -> None:`. Since `Tensor === Tensor[Any, *tuple[Any, ...]]`, we solve `*tuple[Any, ...]` against `A, B` to get `A = Any, B = Any`.

Likewise, if we pass `x: Tensor[float32, *tuple[int, ...], L[42]]` to something expecting `Tensor[float32, A, B, C]`, then we will solve `A = int; B = int; C = L[42]`.

(In case you're curious, this generalizes well to future unpackable items like `*Broadcast[A, B]`, `*Map[A, B]`, etc.)

2.

...
Have you thought about how arbitrary-length tuple bindings would work with some of the type arithmetic operators that were recently proposed for working with dimensional transforms? Would those operators be undefined and therefore illegal if used with an arbitrary-length tuple binding?

Yes, Pyre supports unbounded tuples wherever `Ts` can be used. For example, `Broadcast[Tuple[int, ...], Tuple[L[5], L[6]]] === Tuple[int, ...]`.

3.

...
I didn't realize that you had removed `Union[*Ts]` from the draft PEP. I've already implemented support for this in pyright. It's easy enough to remove, but at the same time, it didn't strike me as one of the more difficult parts of supporting TypeVarTuple. I also don't think that `Union[*Ts]` presents a problem if `Ts` is bound to an arbitrary-length tuple.

Yeah, there are no fundamental limitations to adding `Union[*Ts]`. IIRC we'd removed it because it was making the PEP harder to implement with no real use cases to justify it. I'm thinking of introducing it in the follow-up PEP for Map, where we will probably have more use cases. The alternative is to add it right here, but we're already delaying PEP approval because of the unbounded tuples amendment and this will delay it further. By deferring it, you don't have to delete your implementation either. Does that sound reasonable?

4. Guido:

...
I'm guessing the reason that Matthew rather wants `Tensor[int, ..., int]` to mean what you would spell as `Tensor[int, Any, ..., int]`, is that `a[i, ..., j]` is used to take a slice from a numpy array `a` that drops the first and last dimension.

If we want to add user-friendly syntax like `Tensor[float32, ...]` as syntactic sugar for `Tensor[float32, *tuple[Any, ...]]`, I'm ok with doing that in the future (if there is user demand). Likewise for `Tensor[float32, ..., str]`, which would be equivalent to `Tensor[float32, *tuple[Any, ...], str]`. But, as Guido pointed out, this lack of syntax shouldn't constrain our semantics.

[1]: https://github.com/facebook/pyre-check/blob/c185dc1cb44e86dc60c1feea777fdb95...

[2]: https://www.typescriptlang.org/play?ts=4.5.0-beta#code/CYUwxgNghgTiAEAzArgOz...

On Thu, Nov 18, 2021 at 9:42 PM Guido van Rossum wrote:

...
I may sound like a broken record, but I feel compelled to point out that even though `tuple[int, str, ..., int]` is not legal *syntax* today, and there is currently no way to cause such a type to exist, we could have the concept (inside type checkers) without having syntax to spell it.

I'm guessing the reason that Matthew rather wants `Tensor[int, ..., int]` to mean what you would spell as `Tensor[int, Any, ..., int]`, is that `a[i, ..., j]` is used to take a slice from a numpy array `a` that drops the first and last dimension.

Note that if we were to accept your syntax, an edge case of `Tensor[int, str, ..., int]` would be `Tensor[int, int]` (i.e. the `str, ...` collapses to zero dimensions), as it does for `tuple[int, ...]`.

On Thu, Nov 18, 2021 at 9:04 PM Eric Traut wrote:

...
I'm struggling to remember precisely what I was trying to express in that post. It was over a year ago, after all. :)

Thinking about it now, it seems like `Unpack` should work OK with arbitrary-length types. There are some interesting edge cases to consider though.

With a TypeVarTuple of known length, it's possible to do define a type like this: `tuple[int, *Ts, int]`. If `Ts` is then bound to `tuple[str, str]`, the resulting type is tuple[int, str, str, int]`. But if `Ts` is bound to an arbitrary-length tuple like `tuple[str, ...]`, what is the resulting type? Is it `tuple[int, str, ..., int]`? That's not a legal type, at least not today. Perhaps we can fix this by specifying that `*Ts` is not allowed for arbitrary-length tuples except in the last (right-most) type argument position? Or are you thinking that the type would be `tuple[int, *tuple[str, ...], int]`? That's pretty ugly. It would allow for arbitrary nesting (e.g. `tuple[int, *tuple[int, *tuple[str, ...], int], int]`). Pyright already has about a dozen places where it needs to special-case tuple unpacking and handle both arbitrary-length and fixed-length tuples. This would add significant complexity if every one of those places would need to be able to deal with nesting of unpacked ranges, some of them of arbitrary length. I don't think it would be a good idea to allow such a type.

The interactions between unpacked TypeVarTuple and `Callable` also become more complex if we allow arbitrary-length tuples, but this is probably no more complex than `*args` today.

I didn't realize that you had removed `Union[*Ts]` from the draft PEP. I've already implemented support for this in pyright. It's easy enough to remove, but at the same time, it didn't strike me as one of the more difficult parts of supporting TypeVarTuple. I also don't think that `Union[*Ts]` presents a problem if `Ts` is bound to an arbitrary-length tuple.

Have you thought about how arbitrary-length tuple bindings would work with some of the type arithmetic operators that were recently proposed for working with dimensional transforms? Would those operators be undefined and therefore illegal if used with an arbitrary-length tuple binding?

I know that Matthew would like to reserve the syntax `Tensor[Any, ...]` to mean something different in the future (point 1 above). My preference is to simply adopt the convention already established by `tuple`. The syntax `Tensor[*Tuple[Any, ...]]` is cumbersome and introduces added complexity as I noted above. Plus, the meaning of `tuple[Any, ...]` is already well established and broadly understood, so it's natural to assume that `Tensor[Any, ...]` has the same semantics. If we find in the future that there is really a need to specify "an arbitrary number of type parameters with arbitrary types", we could solve it in some other manner — one that doesn't conflict with existing conventions.

--

Eric Traut Contributor to Pyright & Pylance Microsoft _______________________________________________ Typing-sig mailing list -- typing-sig@python.org To unsubscribe send an email to typing-sig-leave@python.org https://mail.python.org/mailman3/lists/typing-sig.python.org/ Member address: guido@python.org

-- --Guido van Rossum (python.org/~guido) *Pronouns: he/him **(why is my pronoun here?)* http://feministing.com/2015/02/03/how-using-they-as-a-singular-pronoun-can-c... _______________________________________________ Typing-sig mailing list -- typing-sig@python.org To unsubscribe send an email to typing-sig-leave@python.org https://mail.python.org/mailman3/lists/typing-sig.python.org/ Member address: gohanpra@gmail.com

-- S Pradeep Kumar _______________________________________________ Typing-sig mailing list -- typing-sig@python.org To unsubscribe send an email to typing-sig-leave@python.org https://mail.python.org/mailman3/lists/typing-sig.python.org/ Member address: mrahtz@google.com

Guido van Rossum

10:32 p.m.

Can you guys just please decide already? On Fri, Nov 19, 2021 at 3:26 AM Matthew Rahtz wrote:

...

I agree with the points made by Guido and Pradeep.

[Eric]

Or are you thinking that the type would be `tuple[int, *tuple[str, ...],

...
int]`? That's pretty ugly.

Agreed that it's ugly; this was one of the main reasons I pushed back against it initially. But given my strong desire to be careful about what we commit ellipsis to mean, this is the best option we've been able to come up with so far.

[Eric]

Have you thought about how arbitrary-length tuple bindings would work with

...
some of the type arithmetic operators that were recently proposed for working with dimensional transforms? Would those operators be undefined and therefore illegal if used with an arbitrary-length tuple binding?

I'm mostly happy with Pradeep's answer of "The Pyre implementation shows this is fine" - but to be on the safe side, what's your intuition about why some things might not work?

[Guido/Eric]

I'm guessing the reason that Matthew rather wants `Tensor[int, ..., int]`

...
to mean what you would spell as `Tensor[int, Any, ..., int]`, is that `a[i, ..., j]` is used to take a slice from a numpy array `a` that drops the first and last dimension.

Right, this precedent is one reason.

The more general reason is that I think of an ellipsis in natural language as meaning "There's a bunch of stuff missing here, but you don't need to worry about what it is". Having `Tensor[int, ..., int]` mean "An int followed by a bunch of types we don't care about followed by an int" seems more compatible with that.

It's also that, while I think the meaning of `tuple[int, ...]` isn't too hard to guess for a newcomer, if we said that `tuple[int, str, ..., float]` meant "An int followed by an arbitrary number of str followed by a float" - I think that meaning would be harder to guess. The ellipsis isn't clearly attached to str (unlike in regexp where we'd say something like 'int str* float'), so I think it's not obvious that the ellipsis modifies the str rather than the float - or indeed, modifies either.

On Fri, 19 Nov 2021 at 10:35, S Pradeep Kumar wrote:

...
Eric:

Firstly, Pyre has always handled unpacking unbounded tuples. So, if you're curious, you could look at the Python test cases [1].

1.

...
(paraphrasing) Can we substitute `Tuple[int, ...]` for Ts anywhere?

Yes. Fwiw, I did not run into much difficulty with this since the data type definition is recursive. A tuple is either concrete, unbounded, or contains an "unpacked item" with a prefix and suffix. The "unpacked item" is either a variadic Ts or a tuple itself. So, it wasn't like I had to manually handle different levels of nesting; handling the tuple data type would take care of any recursive uses of tuples.

Imposing artificial limitations like "unbounded tuples can only be unpacked at the end" will lead to inconsistent, confusing results. The only consistent implementation is to allow any tuple to be unpacked wherever *Ts is allowed.

Your example `tuple[int, *tuple[str, ...], int]` is a meaningful type. It means a tuple that is guaranteed to begin and end with an int and may have any number of `str`s in between.

This is *exactly* how TypeScript handles unpacking of unbounded tuples (playground link [2]):

``` declare function firstAndLast(arr: readonly [T, ...string[], R]): [T, R];

const r1 = firstAndLast([1, "foo", "bar", 4]); // OK - 2 strings. const r2 = firstAndLast([1, 4]); // OK - 0 strings. const r3 = firstAndLast([1]); // Not OK - missing trailing value.

// How unpacking an unbounded tuple looks:

declare function addPrefixSuffix(arr: T): [number, ...T, number];

const r4 = addPrefixSuffix(["foo", "bar"]); // OK - type of r4 is [number, ...string[], number] const r4_error = addPrefixSuffix([1, 2]) // Not OK - expected strings. ```

For example, unbounded tuples are useful when passing `y: Tensor` to a function `def foo(x: Tensor[DType, A, B]) -> None:`. Since `Tensor === Tensor[Any, *tuple[Any, ...]]`, we solve `*tuple[Any, ...]` against `A, B` to get `A = Any, B = Any`.

Likewise, if we pass `x: Tensor[float32, *tuple[int, ...], L[42]]` to something expecting `Tensor[float32, A, B, C]`, then we will solve `A = int; B = int; C = L[42]`.

(In case you're curious, this generalizes well to future unpackable items like `*Broadcast[A, B]`, `*Map[A, B]`, etc.)

2.

...
Have you thought about how arbitrary-length tuple bindings would work with some of the type arithmetic operators that were recently proposed for working with dimensional transforms? Would those operators be undefined and therefore illegal if used with an arbitrary-length tuple binding?

Yes, Pyre supports unbounded tuples wherever `Ts` can be used. For example, `Broadcast[Tuple[int, ...], Tuple[L[5], L[6]]] === Tuple[int, ...]`.

3.

...
I didn't realize that you had removed `Union[*Ts]` from the draft PEP. I've already implemented support for this in pyright. It's easy enough to remove, but at the same time, it didn't strike me as one of the more difficult parts of supporting TypeVarTuple. I also don't think that `Union[*Ts]` presents a problem if `Ts` is bound to an arbitrary-length tuple.

Yeah, there are no fundamental limitations to adding `Union[*Ts]`. IIRC we'd removed it because it was making the PEP harder to implement with no real use cases to justify it. I'm thinking of introducing it in the follow-up PEP for Map, where we will probably have more use cases. The alternative is to add it right here, but we're already delaying PEP approval because of the unbounded tuples amendment and this will delay it further. By deferring it, you don't have to delete your implementation either. Does that sound reasonable?

4. Guido:

...
I'm guessing the reason that Matthew rather wants `Tensor[int, ..., int]` to mean what you would spell as `Tensor[int, Any, ..., int]`, is that `a[i, ..., j]` is used to take a slice from a numpy array `a` that drops the first and last dimension.

If we want to add user-friendly syntax like `Tensor[float32, ...]` as syntactic sugar for `Tensor[float32, *tuple[Any, ...]]`, I'm ok with doing that in the future (if there is user demand). Likewise for `Tensor[float32, ..., str]`, which would be equivalent to `Tensor[float32, *tuple[Any, ...], str]`. But, as Guido pointed out, this lack of syntax shouldn't constrain our semantics.

[1]: https://github.com/facebook/pyre-check/blob/c185dc1cb44e86dc60c1feea777fdb95...

[2]: https://www.typescriptlang.org/play?ts=4.5.0-beta#code/CYUwxgNghgTiAEAzArgOz...

On Thu, Nov 18, 2021 at 9:42 PM Guido van Rossum wrote:

...
I may sound like a broken record, but I feel compelled to point out that even though `tuple[int, str, ..., int]` is not legal *syntax* today, and there is currently no way to cause such a type to exist, we could have the concept (inside type checkers) without having syntax to spell it.

I'm guessing the reason that Matthew rather wants `Tensor[int, ..., int]` to mean what you would spell as `Tensor[int, Any, ..., int]`, is that `a[i, ..., j]` is used to take a slice from a numpy array `a` that drops the first and last dimension.

Note that if we were to accept your syntax, an edge case of `Tensor[int, str, ..., int]` would be `Tensor[int, int]` (i.e. the `str, ...` collapses to zero dimensions), as it does for `tuple[int, ...]`.

On Thu, Nov 18, 2021 at 9:04 PM Eric Traut wrote:

...
I'm struggling to remember precisely what I was trying to express in that post. It was over a year ago, after all. :)

Thinking about it now, it seems like `Unpack` should work OK with arbitrary-length types. There are some interesting edge cases to consider though.

With a TypeVarTuple of known length, it's possible to do define a type like this: `tuple[int, *Ts, int]`. If `Ts` is then bound to `tuple[str, str]`, the resulting type is tuple[int, str, str, int]`. But if `Ts` is bound to an arbitrary-length tuple like `tuple[str, ...]`, what is the resulting type? Is it `tuple[int, str, ..., int]`? That's not a legal type, at least not today. Perhaps we can fix this by specifying that `*Ts` is not allowed for arbitrary-length tuples except in the last (right-most) type argument position? Or are you thinking that the type would be `tuple[int, *tuple[str, ...], int]`? That's pretty ugly. It would allow for arbitrary nesting (e.g. `tuple[int, *tuple[int, *tuple[str, ...], int], int]`). Pyright already has about a dozen places where it needs to special-case tuple unpacking and handle both arbitrary-length and fixed-length tuples. This would add significant complexity if every one of those places would need to be able to deal with nesting of unpacked ranges, some of them of arbitrary length. I don't think it would be a good idea to allow such a type.

The interactions between unpacked TypeVarTuple and `Callable` also become more complex if we allow arbitrary-length tuples, but this is probably no more complex than `*args` today.

I didn't realize that you had removed `Union[*Ts]` from the draft PEP. I've already implemented support for this in pyright. It's easy enough to remove, but at the same time, it didn't strike me as one of the more difficult parts of supporting TypeVarTuple. I also don't think that `Union[*Ts]` presents a problem if `Ts` is bound to an arbitrary-length tuple.

Have you thought about how arbitrary-length tuple bindings would work with some of the type arithmetic operators that were recently proposed for working with dimensional transforms? Would those operators be undefined and therefore illegal if used with an arbitrary-length tuple binding?

I know that Matthew would like to reserve the syntax `Tensor[Any, ...]` to mean something different in the future (point 1 above). My preference is to simply adopt the convention already established by `tuple`. The syntax `Tensor[*Tuple[Any, ...]]` is cumbersome and introduces added complexity as I noted above. Plus, the meaning of `tuple[Any, ...]` is already well established and broadly understood, so it's natural to assume that `Tensor[Any, ...]` has the same semantics. If we find in the future that there is really a need to specify "an arbitrary number of type parameters with arbitrary types", we could solve it in some other manner — one that doesn't conflict with existing conventions.

--

Eric Traut Contributor to Pyright & Pylance Microsoft _______________________________________________ Typing-sig mailing list -- typing-sig@python.org To unsubscribe send an email to typing-sig-leave@python.org https://mail.python.org/mailman3/lists/typing-sig.python.org/ Member address: guido@python.org

-- --Guido van Rossum (python.org/~guido) *Pronouns: he/him **(why is my pronoun here?)* http://feministing.com/2015/02/03/how-using-they-as-a-singular-pronoun-can-c... _______________________________________________ Typing-sig mailing list -- typing-sig@python.org To unsubscribe send an email to typing-sig-leave@python.org https://mail.python.org/mailman3/lists/typing-sig.python.org/ Member address: gohanpra@gmail.com

-- S Pradeep Kumar _______________________________________________ Typing-sig mailing list -- typing-sig@python.org To unsubscribe send an email to typing-sig-leave@python.org https://mail.python.org/mailman3/lists/typing-sig.python.org/ Member address: mrahtz@google.com

-- --Guido van Rossum (python.org/~guido) *Pronouns: he/him **(why is my pronoun here?)* http://feministing.com/2015/02/03/how-using-they-as-a-singular-pronoun-can-c...

S Pradeep Kumar

11:11 p.m.

Decided (by myself): - Any tuple can be unpacked just like `Ts`, whether it is concrete or unbounded. - No syntax changes in this PEP such as `Tensor[float32, ...]`, etc. Reference implementation available in Pyre. Only sections changed in the PEP are: - Using a Type Variable Tuple Inside a Tuple: allow any tuple to be unpacked. - Type Variable Tuples Must Have Known Length: This restriction no longer applies. Matthew or I can go ahead with this change to the PR and trigger the PEP re-review (sigh). Does that sound good, Guido? On Fri, Nov 19, 2021 at 2:33 PM Guido van Rossum wrote:

...

Can you guys just please decide already?

On Fri, Nov 19, 2021 at 3:26 AM Matthew Rahtz wrote:

...
I agree with the points made by Guido and Pradeep.

[Eric]

Or are you thinking that the type would be `tuple[int, *tuple[str, ...],

...
int]`? That's pretty ugly.

Agreed that it's ugly; this was one of the main reasons I pushed back against it initially. But given my strong desire to be careful about what we commit ellipsis to mean, this is the best option we've been able to come up with so far.

[Eric]

Have you thought about how arbitrary-length tuple bindings would work

...
with some of the type arithmetic operators that were recently proposed for working with dimensional transforms? Would those operators be undefined and therefore illegal if used with an arbitrary-length tuple binding?

I'm mostly happy with Pradeep's answer of "The Pyre implementation shows this is fine" - but to be on the safe side, what's your intuition about why some things might not work?

[Guido/Eric]

I'm guessing the reason that Matthew rather wants `Tensor[int, ..., int]`

...
to mean what you would spell as `Tensor[int, Any, ..., int]`, is that `a[i, ..., j]` is used to take a slice from a numpy array `a` that drops the first and last dimension.

Right, this precedent is one reason.

The more general reason is that I think of an ellipsis in natural language as meaning "There's a bunch of stuff missing here, but you don't need to worry about what it is". Having `Tensor[int, ..., int]` mean "An int followed by a bunch of types we don't care about followed by an int" seems more compatible with that.

It's also that, while I think the meaning of `tuple[int, ...]` isn't too hard to guess for a newcomer, if we said that `tuple[int, str, ..., float]` meant "An int followed by an arbitrary number of str followed by a float" - I think that meaning would be harder to guess. The ellipsis isn't clearly attached to str (unlike in regexp where we'd say something like 'int str* float'), so I think it's not obvious that the ellipsis modifies the str rather than the float - or indeed, modifies either.

On Fri, 19 Nov 2021 at 10:35, S Pradeep Kumar wrote:

...
Eric:

Firstly, Pyre has always handled unpacking unbounded tuples. So, if you're curious, you could look at the Python test cases [1].

1.

...
(paraphrasing) Can we substitute `Tuple[int, ...]` for Ts anywhere?

Yes. Fwiw, I did not run into much difficulty with this since the data type definition is recursive. A tuple is either concrete, unbounded, or contains an "unpacked item" with a prefix and suffix. The "unpacked item" is either a variadic Ts or a tuple itself. So, it wasn't like I had to manually handle different levels of nesting; handling the tuple data type would take care of any recursive uses of tuples.

Imposing artificial limitations like "unbounded tuples can only be unpacked at the end" will lead to inconsistent, confusing results. The only consistent implementation is to allow any tuple to be unpacked wherever *Ts is allowed.

Your example `tuple[int, *tuple[str, ...], int]` is a meaningful type. It means a tuple that is guaranteed to begin and end with an int and may have any number of `str`s in between.

This is *exactly* how TypeScript handles unpacking of unbounded tuples (playground link [2]):

``` declare function firstAndLast(arr: readonly [T, ...string[], R]): [T, R];

const r1 = firstAndLast([1, "foo", "bar", 4]); // OK - 2 strings. const r2 = firstAndLast([1, 4]); // OK - 0 strings. const r3 = firstAndLast([1]); // Not OK - missing trailing value.

// How unpacking an unbounded tuple looks:

declare function addPrefixSuffix(arr: T): [number, ...T, number];

const r4 = addPrefixSuffix(["foo", "bar"]); // OK - type of r4 is [number, ...string[], number] const r4_error = addPrefixSuffix([1, 2]) // Not OK - expected strings. ```

For example, unbounded tuples are useful when passing `y: Tensor` to a function `def foo(x: Tensor[DType, A, B]) -> None:`. Since `Tensor === Tensor[Any, *tuple[Any, ...]]`, we solve `*tuple[Any, ...]` against `A, B` to get `A = Any, B = Any`.

Likewise, if we pass `x: Tensor[float32, *tuple[int, ...], L[42]]` to something expecting `Tensor[float32, A, B, C]`, then we will solve `A = int; B = int; C = L[42]`.

(In case you're curious, this generalizes well to future unpackable items like `*Broadcast[A, B]`, `*Map[A, B]`, etc.)

2.

...
Have you thought about how arbitrary-length tuple bindings would work with some of the type arithmetic operators that were recently proposed for working with dimensional transforms? Would those operators be undefined and therefore illegal if used with an arbitrary-length tuple binding?

Yes, Pyre supports unbounded tuples wherever `Ts` can be used. For example, `Broadcast[Tuple[int, ...], Tuple[L[5], L[6]]] === Tuple[int, ...]`.

3.

...
I didn't realize that you had removed `Union[*Ts]` from the draft PEP. I've already implemented support for this in pyright. It's easy enough to remove, but at the same time, it didn't strike me as one of the more difficult parts of supporting TypeVarTuple. I also don't think that `Union[*Ts]` presents a problem if `Ts` is bound to an arbitrary-length tuple.

Yeah, there are no fundamental limitations to adding `Union[*Ts]`. IIRC we'd removed it because it was making the PEP harder to implement with no real use cases to justify it. I'm thinking of introducing it in the follow-up PEP for Map, where we will probably have more use cases. The alternative is to add it right here, but we're already delaying PEP approval because of the unbounded tuples amendment and this will delay it further. By deferring it, you don't have to delete your implementation either. Does that sound reasonable?

4. Guido:

...
I'm guessing the reason that Matthew rather wants `Tensor[int, ..., int]` to mean what you would spell as `Tensor[int, Any, ..., int]`, is that `a[i, ..., j]` is used to take a slice from a numpy array `a` that drops the first and last dimension.

If we want to add user-friendly syntax like `Tensor[float32, ...]` as syntactic sugar for `Tensor[float32, *tuple[Any, ...]]`, I'm ok with doing that in the future (if there is user demand). Likewise for `Tensor[float32, ..., str]`, which would be equivalent to `Tensor[float32, *tuple[Any, ...], str]`. But, as Guido pointed out, this lack of syntax shouldn't constrain our semantics.

[1]: https://github.com/facebook/pyre-check/blob/c185dc1cb44e86dc60c1feea777fdb95...

[2]: https://www.typescriptlang.org/play?ts=4.5.0-beta#code/CYUwxgNghgTiAEAzArgOz...

On Thu, Nov 18, 2021 at 9:42 PM Guido van Rossum wrote:

...
I may sound like a broken record, but I feel compelled to point out that even though `tuple[int, str, ..., int]` is not legal *syntax* today, and there is currently no way to cause such a type to exist, we could have the concept (inside type checkers) without having syntax to spell it.

I'm guessing the reason that Matthew rather wants `Tensor[int, ..., int]` to mean what you would spell as `Tensor[int, Any, ..., int]`, is that `a[i, ..., j]` is used to take a slice from a numpy array `a` that drops the first and last dimension.

Note that if we were to accept your syntax, an edge case of `Tensor[int, str, ..., int]` would be `Tensor[int, int]` (i.e. the `str, ...` collapses to zero dimensions), as it does for `tuple[int, ...]`.

On Thu, Nov 18, 2021 at 9:04 PM Eric Traut wrote:

...
I'm struggling to remember precisely what I was trying to express in that post. It was over a year ago, after all. :)

Thinking about it now, it seems like `Unpack` should work OK with arbitrary-length types. There are some interesting edge cases to consider though.

With a TypeVarTuple of known length, it's possible to do define a type like this: `tuple[int, *Ts, int]`. If `Ts` is then bound to `tuple[str, str]`, the resulting type is tuple[int, str, str, int]`. But if `Ts` is bound to an arbitrary-length tuple like `tuple[str, ...]`, what is the resulting type? Is it `tuple[int, str, ..., int]`? That's not a legal type, at least not today. Perhaps we can fix this by specifying that `*Ts` is not allowed for arbitrary-length tuples except in the last (right-most) type argument position? Or are you thinking that the type would be `tuple[int, *tuple[str, ...], int]`? That's pretty ugly. It would allow for arbitrary nesting (e.g. `tuple[int, *tuple[int, *tuple[str, ...], int], int]`). Pyright already has about a dozen places where it needs to special-case tuple unpacking and handle both arbitrary-length and fixed-length tuples. This would add significant complexity if every one of those places would need to be able to deal with nesting of unpacked ranges, some of them of arbitrary length. I don't think it would be a good idea to allow such a type.

The interactions between unpacked TypeVarTuple and `Callable` also become more complex if we allow arbitrary-length tuples, but this is probably no more complex than `*args` today.

I didn't realize that you had removed `Union[*Ts]` from the draft PEP. I've already implemented support for this in pyright. It's easy enough to remove, but at the same time, it didn't strike me as one of the more difficult parts of supporting TypeVarTuple. I also don't think that `Union[*Ts]` presents a problem if `Ts` is bound to an arbitrary-length tuple.

Have you thought about how arbitrary-length tuple bindings would work with some of the type arithmetic operators that were recently proposed for working with dimensional transforms? Would those operators be undefined and therefore illegal if used with an arbitrary-length tuple binding?

I know that Matthew would like to reserve the syntax `Tensor[Any, ...]` to mean something different in the future (point 1 above). My preference is to simply adopt the convention already established by `tuple`. The syntax `Tensor[*Tuple[Any, ...]]` is cumbersome and introduces added complexity as I noted above. Plus, the meaning of `tuple[Any, ...]` is already well established and broadly understood, so it's natural to assume that `Tensor[Any, ...]` has the same semantics. If we find in the future that there is really a need to specify "an arbitrary number of type parameters with arbitrary types", we could solve it in some other manner — one that doesn't conflict with existing conventions.

--

Eric Traut Contributor to Pyright & Pylance Microsoft _______________________________________________ Typing-sig mailing list -- typing-sig@python.org To unsubscribe send an email to typing-sig-leave@python.org https://mail.python.org/mailman3/lists/typing-sig.python.org/ Member address: guido@python.org

-- --Guido van Rossum (python.org/~guido) *Pronouns: he/him **(why is my pronoun here?)* http://feministing.com/2015/02/03/how-using-they-as-a-singular-pronoun-can-c... _______________________________________________ Typing-sig mailing list -- typing-sig@python.org To unsubscribe send an email to typing-sig-leave@python.org https://mail.python.org/mailman3/lists/typing-sig.python.org/ Member address: gohanpra@gmail.com

-- S Pradeep Kumar _______________________________________________ Typing-sig mailing list -- typing-sig@python.org To unsubscribe send an email to typing-sig-leave@python.org https://mail.python.org/mailman3/lists/typing-sig.python.org/ Member address: mrahtz@google.com

-- --Guido van Rossum (python.org/~guido) *Pronouns: he/him **(why is my pronoun here?)* http://feministing.com/2015/02/03/how-using-they-as-a-singular-pronoun-can-c...

-- S Pradeep Kumar

Guido van Rossum

11:27 p.m.

Sounds good if Matthew is also okay with that. On Fri, Nov 19, 2021 at 3:12 PM S Pradeep Kumar wrote:

...

Decided (by myself):

- Any tuple can be unpacked just like `Ts`, whether it is concrete or unbounded. - No syntax changes in this PEP such as `Tensor[float32, ...]`, etc.

Reference implementation available in Pyre.

Only sections changed in the PEP are:

- Using a Type Variable Tuple Inside a Tuple: allow any tuple to be unpacked. - Type Variable Tuples Must Have Known Length: This restriction no longer applies.

Matthew or I can go ahead with this change to the PR and trigger the PEP re-review (sigh). Does that sound good, Guido?

On Fri, Nov 19, 2021 at 2:33 PM Guido van Rossum wrote:

...
Can you guys just please decide already?

On Fri, Nov 19, 2021 at 3:26 AM Matthew Rahtz wrote:

...
I agree with the points made by Guido and Pradeep.

[Eric]

Or are you thinking that the type would be `tuple[int, *tuple[str, ...],

...
int]`? That's pretty ugly.

Agreed that it's ugly; this was one of the main reasons I pushed back against it initially. But given my strong desire to be careful about what we commit ellipsis to mean, this is the best option we've been able to come up with so far.

[Eric]

Have you thought about how arbitrary-length tuple bindings would work

...
with some of the type arithmetic operators that were recently proposed for working with dimensional transforms? Would those operators be undefined and therefore illegal if used with an arbitrary-length tuple binding?

I'm mostly happy with Pradeep's answer of "The Pyre implementation shows this is fine" - but to be on the safe side, what's your intuition about why some things might not work?

[Guido/Eric]

I'm guessing the reason that Matthew rather wants `Tensor[int, ...,

...
int]` to mean what you would spell as `Tensor[int, Any, ..., int]`, is that `a[i, ..., j]` is used to take a slice from a numpy array `a` that drops the first and last dimension.

Right, this precedent is one reason.

The more general reason is that I think of an ellipsis in natural language as meaning "There's a bunch of stuff missing here, but you don't need to worry about what it is". Having `Tensor[int, ..., int]` mean "An int followed by a bunch of types we don't care about followed by an int" seems more compatible with that.

It's also that, while I think the meaning of `tuple[int, ...]` isn't too hard to guess for a newcomer, if we said that `tuple[int, str, ..., float]` meant "An int followed by an arbitrary number of str followed by a float" - I think that meaning would be harder to guess. The ellipsis isn't clearly attached to str (unlike in regexp where we'd say something like 'int str* float'), so I think it's not obvious that the ellipsis modifies the str rather than the float - or indeed, modifies either.

On Fri, 19 Nov 2021 at 10:35, S Pradeep Kumar wrote:

...
Eric:

Firstly, Pyre has always handled unpacking unbounded tuples. So, if you're curious, you could look at the Python test cases [1].

1.

...
(paraphrasing) Can we substitute `Tuple[int, ...]` for Ts anywhere?

Yes. Fwiw, I did not run into much difficulty with this since the data type definition is recursive. A tuple is either concrete, unbounded, or contains an "unpacked item" with a prefix and suffix. The "unpacked item" is either a variadic Ts or a tuple itself. So, it wasn't like I had to manually handle different levels of nesting; handling the tuple data type would take care of any recursive uses of tuples.

Imposing artificial limitations like "unbounded tuples can only be unpacked at the end" will lead to inconsistent, confusing results. The only consistent implementation is to allow any tuple to be unpacked wherever *Ts is allowed.

Your example `tuple[int, *tuple[str, ...], int]` is a meaningful type. It means a tuple that is guaranteed to begin and end with an int and may have any number of `str`s in between.

This is *exactly* how TypeScript handles unpacking of unbounded tuples (playground link [2]):

``` declare function firstAndLast(arr: readonly [T, ...string[], R]): [T, R];

const r1 = firstAndLast([1, "foo", "bar", 4]); // OK - 2 strings. const r2 = firstAndLast([1, 4]); // OK - 0 strings. const r3 = firstAndLast([1]); // Not OK - missing trailing value.

// How unpacking an unbounded tuple looks:

declare function addPrefixSuffix(arr: T): [number, ...T, number];

const r4 = addPrefixSuffix(["foo", "bar"]); // OK - type of r4 is [number, ...string[], number] const r4_error = addPrefixSuffix([1, 2]) // Not OK - expected strings. ```

For example, unbounded tuples are useful when passing `y: Tensor` to a function `def foo(x: Tensor[DType, A, B]) -> None:`. Since `Tensor === Tensor[Any, *tuple[Any, ...]]`, we solve `*tuple[Any, ...]` against `A, B` to get `A = Any, B = Any`.

Likewise, if we pass `x: Tensor[float32, *tuple[int, ...], L[42]]` to something expecting `Tensor[float32, A, B, C]`, then we will solve `A = int; B = int; C = L[42]`.

(In case you're curious, this generalizes well to future unpackable items like `*Broadcast[A, B]`, `*Map[A, B]`, etc.)

2.

...
Have you thought about how arbitrary-length tuple bindings would work with some of the type arithmetic operators that were recently proposed for working with dimensional transforms? Would those operators be undefined and therefore illegal if used with an arbitrary-length tuple binding?

Yes, Pyre supports unbounded tuples wherever `Ts` can be used. For example, `Broadcast[Tuple[int, ...], Tuple[L[5], L[6]]] === Tuple[int, ...]`.

3.

...
I didn't realize that you had removed `Union[*Ts]` from the draft PEP. I've already implemented support for this in pyright. It's easy enough to remove, but at the same time, it didn't strike me as one of the more difficult parts of supporting TypeVarTuple. I also don't think that `Union[*Ts]` presents a problem if `Ts` is bound to an arbitrary-length tuple.

Yeah, there are no fundamental limitations to adding `Union[*Ts]`. IIRC we'd removed it because it was making the PEP harder to implement with no real use cases to justify it. I'm thinking of introducing it in the follow-up PEP for Map, where we will probably have more use cases. The alternative is to add it right here, but we're already delaying PEP approval because of the unbounded tuples amendment and this will delay it further. By deferring it, you don't have to delete your implementation either. Does that sound reasonable?

4. Guido:

...
I'm guessing the reason that Matthew rather wants `Tensor[int, ..., int]` to mean what you would spell as `Tensor[int, Any, ..., int]`, is that `a[i, ..., j]` is used to take a slice from a numpy array `a` that drops the first and last dimension.

If we want to add user-friendly syntax like `Tensor[float32, ...]` as syntactic sugar for `Tensor[float32, *tuple[Any, ...]]`, I'm ok with doing that in the future (if there is user demand). Likewise for `Tensor[float32, ..., str]`, which would be equivalent to `Tensor[float32, *tuple[Any, ...], str]`. But, as Guido pointed out, this lack of syntax shouldn't constrain our semantics.

[1]: https://github.com/facebook/pyre-check/blob/c185dc1cb44e86dc60c1feea777fdb95...

[2]: https://www.typescriptlang.org/play?ts=4.5.0-beta#code/CYUwxgNghgTiAEAzArgOz...

On Thu, Nov 18, 2021 at 9:42 PM Guido van Rossum wrote:

...
I may sound like a broken record, but I feel compelled to point out that even though `tuple[int, str, ..., int]` is not legal *syntax* today, and there is currently no way to cause such a type to exist, we could have the concept (inside type checkers) without having syntax to spell it.

I'm guessing the reason that Matthew rather wants `Tensor[int, ..., int]` to mean what you would spell as `Tensor[int, Any, ..., int]`, is that `a[i, ..., j]` is used to take a slice from a numpy array `a` that drops the first and last dimension.

Note that if we were to accept your syntax, an edge case of `Tensor[int, str, ..., int]` would be `Tensor[int, int]` (i.e. the `str, ...` collapses to zero dimensions), as it does for `tuple[int, ...]`.

On Thu, Nov 18, 2021 at 9:04 PM Eric Traut wrote:

...
I'm struggling to remember precisely what I was trying to express in that post. It was over a year ago, after all. :)

Thinking about it now, it seems like `Unpack` should work OK with arbitrary-length types. There are some interesting edge cases to consider though.

With a TypeVarTuple of known length, it's possible to do define a type like this: `tuple[int, *Ts, int]`. If `Ts` is then bound to `tuple[str, str]`, the resulting type is tuple[int, str, str, int]`. But if `Ts` is bound to an arbitrary-length tuple like `tuple[str, ...]`, what is the resulting type? Is it `tuple[int, str, ..., int]`? That's not a legal type, at least not today. Perhaps we can fix this by specifying that `*Ts` is not allowed for arbitrary-length tuples except in the last (right-most) type argument position? Or are you thinking that the type would be `tuple[int, *tuple[str, ...], int]`? That's pretty ugly. It would allow for arbitrary nesting (e.g. `tuple[int, *tuple[int, *tuple[str, ...], int], int]`). Pyright already has about a dozen places where it needs to special-case tuple unpacking and handle both arbitrary-length and fixed-length tuples. This would add significant complexity if every one of those places would need to be able to deal with nesting of unpacked ranges, some of them of arbitrary length. I don't think it would be a good idea to allow such a type.

The interactions between unpacked TypeVarTuple and `Callable` also become more complex if we allow arbitrary-length tuples, but this is probably no more complex than `*args` today.

I didn't realize that you had removed `Union[*Ts]` from the draft PEP. I've already implemented support for this in pyright. It's easy enough to remove, but at the same time, it didn't strike me as one of the more difficult parts of supporting TypeVarTuple. I also don't think that `Union[*Ts]` presents a problem if `Ts` is bound to an arbitrary-length tuple.

Have you thought about how arbitrary-length tuple bindings would work with some of the type arithmetic operators that were recently proposed for working with dimensional transforms? Would those operators be undefined and therefore illegal if used with an arbitrary-length tuple binding?

I know that Matthew would like to reserve the syntax `Tensor[Any, ...]` to mean something different in the future (point 1 above). My preference is to simply adopt the convention already established by `tuple`. The syntax `Tensor[*Tuple[Any, ...]]` is cumbersome and introduces added complexity as I noted above. Plus, the meaning of `tuple[Any, ...]` is already well established and broadly understood, so it's natural to assume that `Tensor[Any, ...]` has the same semantics. If we find in the future that there is really a need to specify "an arbitrary number of type parameters with arbitrary types", we could solve it in some other manner — one that doesn't conflict with existing conventions.

--

Eric Traut Contributor to Pyright & Pylance Microsoft _______________________________________________ Typing-sig mailing list -- typing-sig@python.org To unsubscribe send an email to typing-sig-leave@python.org https://mail.python.org/mailman3/lists/typing-sig.python.org/ Member address: guido@python.org

-- --Guido van Rossum (python.org/~guido) *Pronouns: he/him **(why is my pronoun here?)* http://feministing.com/2015/02/03/how-using-they-as-a-singular-pronoun-can-c... _______________________________________________ Typing-sig mailing list -- typing-sig@python.org To unsubscribe send an email to typing-sig-leave@python.org https://mail.python.org/mailman3/lists/typing-sig.python.org/ Member address: gohanpra@gmail.com

-- S Pradeep Kumar _______________________________________________ Typing-sig mailing list -- typing-sig@python.org To unsubscribe send an email to typing-sig-leave@python.org https://mail.python.org/mailman3/lists/typing-sig.python.org/ Member address: mrahtz@google.com

-- --Guido van Rossum (python.org/~guido) *Pronouns: he/him **(why is my pronoun here?)* http://feministing.com/2015/02/03/how-using-they-as-a-singular-pronoun-can-c...

-- S Pradeep Kumar

-- --Guido van Rossum (python.org/~guido) *Pronouns: he/him **(why is my pronoun here?)* http://feministing.com/2015/02/03/how-using-they-as-a-singular-pronoun-can-c...

Matthew Rahtz

22 Nov 22 Nov

4:30 p.m.

Sounds good to me too. Happy for you to make the changes, Pradeep. Some other things we should probably add are: - Make it explicit that a variadic generic instantiated without any type parameters behaves like Tensor[*Tuple[Any, ...]] - Make it explicit how to have a function which has constraints only on some parts of a variadic parameter list: def foo(x: Tensor[Batch, *Tuple[Any, ...]]) Might be clearest to add a dedicated section explaining these, but up to you. On Fri, 19 Nov 2021 at 23:28, Guido van Rossum wrote:

...

Sounds good if Matthew is also okay with that.

On Fri, Nov 19, 2021 at 3:12 PM S Pradeep Kumar wrote:

...
Decided (by myself):

- Any tuple can be unpacked just like `Ts`, whether it is concrete or unbounded. - No syntax changes in this PEP such as `Tensor[float32, ...]`, etc.

Reference implementation available in Pyre.

Only sections changed in the PEP are:

- Using a Type Variable Tuple Inside a Tuple: allow any tuple to be unpacked. - Type Variable Tuples Must Have Known Length: This restriction no longer applies.

Matthew or I can go ahead with this change to the PR and trigger the PEP re-review (sigh). Does that sound good, Guido?

On Fri, Nov 19, 2021 at 2:33 PM Guido van Rossum wrote:

...
Can you guys just please decide already?

On Fri, Nov 19, 2021 at 3:26 AM Matthew Rahtz wrote:

...
I agree with the points made by Guido and Pradeep.

[Eric]

Or are you thinking that the type would be `tuple[int, *tuple[str,

...
...], int]`? That's pretty ugly.

Agreed that it's ugly; this was one of the main reasons I pushed back against it initially. But given my strong desire to be careful about what we commit ellipsis to mean, this is the best option we've been able to come up with so far.

[Eric]

Have you thought about how arbitrary-length tuple bindings would work

...
with some of the type arithmetic operators that were recently proposed for working with dimensional transforms? Would those operators be undefined and therefore illegal if used with an arbitrary-length tuple binding?

I'm mostly happy with Pradeep's answer of "The Pyre implementation shows this is fine" - but to be on the safe side, what's your intuition about why some things might not work?

[Guido/Eric]

I'm guessing the reason that Matthew rather wants `Tensor[int, ...,

...
int]` to mean what you would spell as `Tensor[int, Any, ..., int]`, is that `a[i, ..., j]` is used to take a slice from a numpy array `a` that drops the first and last dimension.

Right, this precedent is one reason.

The more general reason is that I think of an ellipsis in natural language as meaning "There's a bunch of stuff missing here, but you don't need to worry about what it is". Having `Tensor[int, ..., int]` mean "An int followed by a bunch of types we don't care about followed by an int" seems more compatible with that.

It's also that, while I think the meaning of `tuple[int, ...]` isn't too hard to guess for a newcomer, if we said that `tuple[int, str, ..., float]` meant "An int followed by an arbitrary number of str followed by a float" - I think that meaning would be harder to guess. The ellipsis isn't clearly attached to str (unlike in regexp where we'd say something like 'int str* float'), so I think it's not obvious that the ellipsis modifies the str rather than the float - or indeed, modifies either.

On Fri, 19 Nov 2021 at 10:35, S Pradeep Kumar wrote:

...
Eric:

Firstly, Pyre has always handled unpacking unbounded tuples. So, if you're curious, you could look at the Python test cases [1].

1.

...
(paraphrasing) Can we substitute `Tuple[int, ...]` for Ts anywhere?

Yes. Fwiw, I did not run into much difficulty with this since the data type definition is recursive. A tuple is either concrete, unbounded, or contains an "unpacked item" with a prefix and suffix. The "unpacked item" is either a variadic Ts or a tuple itself. So, it wasn't like I had to manually handle different levels of nesting; handling the tuple data type would take care of any recursive uses of tuples.

Imposing artificial limitations like "unbounded tuples can only be unpacked at the end" will lead to inconsistent, confusing results. The only consistent implementation is to allow any tuple to be unpacked wherever *Ts is allowed.

Your example `tuple[int, *tuple[str, ...], int]` is a meaningful type. It means a tuple that is guaranteed to begin and end with an int and may have any number of `str`s in between.

This is *exactly* how TypeScript handles unpacking of unbounded tuples (playground link [2]):

``` declare function firstAndLast(arr: readonly [T, ...string[], R]): [T, R];

const r1 = firstAndLast([1, "foo", "bar", 4]); // OK - 2 strings. const r2 = firstAndLast([1, 4]); // OK - 0 strings. const r3 = firstAndLast([1]); // Not OK - missing trailing value.

// How unpacking an unbounded tuple looks:

declare function addPrefixSuffix(arr: T): [number, ...T, number];

const r4 = addPrefixSuffix(["foo", "bar"]); // OK - type of r4 is [number, ...string[], number] const r4_error = addPrefixSuffix([1, 2]) // Not OK - expected strings. ```

For example, unbounded tuples are useful when passing `y: Tensor` to a function `def foo(x: Tensor[DType, A, B]) -> None:`. Since `Tensor === Tensor[Any, *tuple[Any, ...]]`, we solve `*tuple[Any, ...]` against `A, B` to get `A = Any, B = Any`.

Likewise, if we pass `x: Tensor[float32, *tuple[int, ...], L[42]]` to something expecting `Tensor[float32, A, B, C]`, then we will solve `A = int; B = int; C = L[42]`.

(In case you're curious, this generalizes well to future unpackable items like `*Broadcast[A, B]`, `*Map[A, B]`, etc.)

2.

...
Have you thought about how arbitrary-length tuple bindings would work with some of the type arithmetic operators that were recently proposed for working with dimensional transforms? Would those operators be undefined and therefore illegal if used with an arbitrary-length tuple binding?

Yes, Pyre supports unbounded tuples wherever `Ts` can be used. For example, `Broadcast[Tuple[int, ...], Tuple[L[5], L[6]]] === Tuple[int, ...]`.

3.

...
I didn't realize that you had removed `Union[*Ts]` from the draft PEP. I've already implemented support for this in pyright. It's easy enough to remove, but at the same time, it didn't strike me as one of the more difficult parts of supporting TypeVarTuple. I also don't think that `Union[*Ts]` presents a problem if `Ts` is bound to an arbitrary-length tuple.

Yeah, there are no fundamental limitations to adding `Union[*Ts]`. IIRC we'd removed it because it was making the PEP harder to implement with no real use cases to justify it. I'm thinking of introducing it in the follow-up PEP for Map, where we will probably have more use cases. The alternative is to add it right here, but we're already delaying PEP approval because of the unbounded tuples amendment and this will delay it further. By deferring it, you don't have to delete your implementation either. Does that sound reasonable?

4. Guido:

...
I'm guessing the reason that Matthew rather wants `Tensor[int, ..., int]` to mean what you would spell as `Tensor[int, Any, ..., int]`, is that `a[i, ..., j]` is used to take a slice from a numpy array `a` that drops the first and last dimension.

If we want to add user-friendly syntax like `Tensor[float32, ...]` as syntactic sugar for `Tensor[float32, *tuple[Any, ...]]`, I'm ok with doing that in the future (if there is user demand). Likewise for `Tensor[float32, ..., str]`, which would be equivalent to `Tensor[float32, *tuple[Any, ...], str]`. But, as Guido pointed out, this lack of syntax shouldn't constrain our semantics.

[1]: https://github.com/facebook/pyre-check/blob/c185dc1cb44e86dc60c1feea777fdb95...

[2]: https://www.typescriptlang.org/play?ts=4.5.0-beta#code/CYUwxgNghgTiAEAzArgOz...

On Thu, Nov 18, 2021 at 9:42 PM Guido van Rossum wrote:

...
I may sound like a broken record, but I feel compelled to point out that even though `tuple[int, str, ..., int]` is not legal *syntax* today, and there is currently no way to cause such a type to exist, we could have the concept (inside type checkers) without having syntax to spell it.

I'm guessing the reason that Matthew rather wants `Tensor[int, ..., int]` to mean what you would spell as `Tensor[int, Any, ..., int]`, is that `a[i, ..., j]` is used to take a slice from a numpy array `a` that drops the first and last dimension.

Note that if we were to accept your syntax, an edge case of `Tensor[int, str, ..., int]` would be `Tensor[int, int]` (i.e. the `str, ...` collapses to zero dimensions), as it does for `tuple[int, ...]`.

On Thu, Nov 18, 2021 at 9:04 PM Eric Traut wrote:

> I'm struggling to remember precisely what I was trying to express in > that post. It was over a year ago, after all. :) > > Thinking about it now, it seems like `Unpack` should work OK with > arbitrary-length types. There are some interesting edge cases to consider > though. > > With a TypeVarTuple of known length, it's possible to do define a > type like this: `tuple[int, *Ts, int]`. If `Ts` is then bound to > `tuple[str, str]`, the resulting type is tuple[int, str, str, int]`. But if > `Ts` is bound to an arbitrary-length tuple like `tuple[str, ...]`, what is > the resulting type? Is it `tuple[int, str, ..., int]`? That's not a legal > type, at least not today. Perhaps we can fix this by specifying that `*Ts` > is not allowed for arbitrary-length tuples except in the last (right-most) > type argument position? Or are you thinking that the type would be > `tuple[int, *tuple[str, ...], int]`? That's pretty ugly. It would allow for > arbitrary nesting (e.g. `tuple[int, *tuple[int, *tuple[str, ...], int], > int]`). Pyright already has about a dozen places where it needs to > special-case tuple unpacking and handle both arbitrary-length and > fixed-length tuples. This would add significant complexity if every one of > those places would need to be able to deal with nesting of unpacked ranges, > some of them of arbitrary length. I don't think it would be a good idea to > allow such a type. > > The interactions between unpacked TypeVarTuple and `Callable` also > become more complex if we allow arbitrary-length tuples, but this is > probably no more complex than `*args` today. > > I didn't realize that you had removed `Union[*Ts]` from the draft > PEP. I've already implemented support for this in pyright. It's easy enough > to remove, but at the same time, it didn't strike me as one of the more > difficult parts of supporting TypeVarTuple. I also don't think that > `Union[*Ts]` presents a problem if `Ts` is bound to an arbitrary-length > tuple. > > Have you thought about how arbitrary-length tuple bindings would > work with some of the type arithmetic operators that were recently proposed > for working with dimensional transforms? Would those operators be undefined > and therefore illegal if used with an arbitrary-length tuple binding? > > I know that Matthew would like to reserve the syntax `Tensor[Any, > ...]` to mean something different in the future (point 1 above). My > preference is to simply adopt the convention already established by > `tuple`. The syntax `Tensor[*Tuple[Any, ...]]` is cumbersome and introduces > added complexity as I noted above. Plus, the meaning of `tuple[Any, ...]` > is already well established and broadly understood, so it's natural to > assume that `Tensor[Any, ...]` has the same semantics. If we find in the > future that there is really a need to specify "an arbitrary number of type > parameters with arbitrary types", we could solve it in some other manner — > one that doesn't conflict with existing conventions. > > -- > > Eric Traut > Contributor to Pyright & Pylance > Microsoft > _______________________________________________ > Typing-sig mailing list -- typing-sig@python.org > To unsubscribe send an email to typing-sig-leave@python.org > https://mail.python.org/mailman3/lists/typing-sig.python.org/ > Member address: guido@python.org >

-- --Guido van Rossum (python.org/~guido) *Pronouns: he/him **(why is my pronoun here?)* http://feministing.com/2015/02/03/how-using-they-as-a-singular-pronoun-can-c... _______________________________________________ Typing-sig mailing list -- typing-sig@python.org To unsubscribe send an email to typing-sig-leave@python.org https://mail.python.org/mailman3/lists/typing-sig.python.org/ Member address: gohanpra@gmail.com

-- S Pradeep Kumar _______________________________________________ Typing-sig mailing list -- typing-sig@python.org To unsubscribe send an email to typing-sig-leave@python.org https://mail.python.org/mailman3/lists/typing-sig.python.org/ Member address: mrahtz@google.com

-- --Guido van Rossum (python.org/~guido) *Pronouns: he/him **(why is my pronoun here?)* http://feministing.com/2015/02/03/how-using-they-as-a-singular-pronoun-can-c...

-- S Pradeep Kumar

-- --Guido van Rossum (python.org/~guido) *Pronouns: he/him **(why is my pronoun here?)* http://feministing.com/2015/02/03/how-using-they-as-a-singular-pronoun-can-c...

S Pradeep Kumar

24 Nov 24 Nov

9:58 p.m.

Guido, I've put up https://github.com/python/peps/pull/2162 On Mon, Nov 22, 2021 at 8:30 AM Matthew Rahtz wrote:

...

Sounds good to me too.

Happy for you to make the changes, Pradeep. Some other things we should probably add are:

- Make it explicit that a variadic generic instantiated without any type parameters behaves like Tensor[*Tuple[Any, ...]] - Make it explicit how to have a function which has constraints only on some parts of a variadic parameter list: def foo(x: Tensor[Batch, *Tuple[Any, ...]])

Might be clearest to add a dedicated section explaining these, but up to you.

On Fri, 19 Nov 2021 at 23:28, Guido van Rossum wrote:

...
Sounds good if Matthew is also okay with that.

On Fri, Nov 19, 2021 at 3:12 PM S Pradeep Kumar wrote:

...
Decided (by myself):

- Any tuple can be unpacked just like `Ts`, whether it is concrete or unbounded. - No syntax changes in this PEP such as `Tensor[float32, ...]`, etc.

Reference implementation available in Pyre.

Only sections changed in the PEP are:

- Using a Type Variable Tuple Inside a Tuple: allow any tuple to be unpacked. - Type Variable Tuples Must Have Known Length: This restriction no longer applies.

Matthew or I can go ahead with this change to the PR and trigger the PEP re-review (sigh). Does that sound good, Guido?

On Fri, Nov 19, 2021 at 2:33 PM Guido van Rossum wrote:

...
Can you guys just please decide already?

On Fri, Nov 19, 2021 at 3:26 AM Matthew Rahtz wrote:

...
I agree with the points made by Guido and Pradeep.

[Eric]

Or are you thinking that the type would be `tuple[int, *tuple[str,

...
...], int]`? That's pretty ugly.

Agreed that it's ugly; this was one of the main reasons I pushed back against it initially. But given my strong desire to be careful about what we commit ellipsis to mean, this is the best option we've been able to come up with so far.

[Eric]

Have you thought about how arbitrary-length tuple bindings would work

...
with some of the type arithmetic operators that were recently proposed for working with dimensional transforms? Would those operators be undefined and therefore illegal if used with an arbitrary-length tuple binding?

I'm mostly happy with Pradeep's answer of "The Pyre implementation shows this is fine" - but to be on the safe side, what's your intuition about why some things might not work?

[Guido/Eric]

I'm guessing the reason that Matthew rather wants `Tensor[int, ...,

...
int]` to mean what you would spell as `Tensor[int, Any, ..., int]`, is that `a[i, ..., j]` is used to take a slice from a numpy array `a` that drops the first and last dimension.

Right, this precedent is one reason.

The more general reason is that I think of an ellipsis in natural language as meaning "There's a bunch of stuff missing here, but you don't need to worry about what it is". Having `Tensor[int, ..., int]` mean "An int followed by a bunch of types we don't care about followed by an int" seems more compatible with that.

It's also that, while I think the meaning of `tuple[int, ...]` isn't too hard to guess for a newcomer, if we said that `tuple[int, str, ..., float]` meant "An int followed by an arbitrary number of str followed by a float" - I think that meaning would be harder to guess. The ellipsis isn't clearly attached to str (unlike in regexp where we'd say something like 'int str* float'), so I think it's not obvious that the ellipsis modifies the str rather than the float - or indeed, modifies either.

On Fri, 19 Nov 2021 at 10:35, S Pradeep Kumar wrote:

...
Eric:

Firstly, Pyre has always handled unpacking unbounded tuples. So, if you're curious, you could look at the Python test cases [1].

1.

> (paraphrasing) Can we substitute `Tuple[int, ...]` for Ts anywhere?

Yes. Fwiw, I did not run into much difficulty with this since the data type definition is recursive. A tuple is either concrete, unbounded, or contains an "unpacked item" with a prefix and suffix. The "unpacked item" is either a variadic Ts or a tuple itself. So, it wasn't like I had to manually handle different levels of nesting; handling the tuple data type would take care of any recursive uses of tuples.

Imposing artificial limitations like "unbounded tuples can only be unpacked at the end" will lead to inconsistent, confusing results. The only consistent implementation is to allow any tuple to be unpacked wherever *Ts is allowed.

Your example `tuple[int, *tuple[str, ...], int]` is a meaningful type. It means a tuple that is guaranteed to begin and end with an int and may have any number of `str`s in between.

This is *exactly* how TypeScript handles unpacking of unbounded tuples (playground link [2]):

``` declare function firstAndLast(arr: readonly [T, ...string[], R]): [T, R];

const r1 = firstAndLast([1, "foo", "bar", 4]); // OK - 2 strings. const r2 = firstAndLast([1, 4]); // OK - 0 strings. const r3 = firstAndLast([1]); // Not OK - missing trailing value.

// How unpacking an unbounded tuple looks:

declare function addPrefixSuffix(arr: T): [number, ...T, number];

const r4 = addPrefixSuffix(["foo", "bar"]); // OK - type of r4 is [number, ...string[], number] const r4_error = addPrefixSuffix([1, 2]) // Not OK - expected strings. ```

For example, unbounded tuples are useful when passing `y: Tensor` to a function `def foo(x: Tensor[DType, A, B]) -> None:`. Since `Tensor === Tensor[Any, *tuple[Any, ...]]`, we solve `*tuple[Any, ...]` against `A, B` to get `A = Any, B = Any`.

Likewise, if we pass `x: Tensor[float32, *tuple[int, ...], L[42]]` to something expecting `Tensor[float32, A, B, C]`, then we will solve `A = int; B = int; C = L[42]`.

(In case you're curious, this generalizes well to future unpackable items like `*Broadcast[A, B]`, `*Map[A, B]`, etc.)

2.

> Have you thought about how arbitrary-length tuple bindings would work with some of the type arithmetic operators that were recently proposed for working with dimensional transforms? Would those operators be undefined and therefore illegal if used with an arbitrary-length tuple binding?

Yes, Pyre supports unbounded tuples wherever `Ts` can be used. For example, `Broadcast[Tuple[int, ...], Tuple[L[5], L[6]]] === Tuple[int, ...]`.

3.

> I didn't realize that you had removed `Union[*Ts]` from the draft PEP. I've already implemented support for this in pyright. It's easy enough to remove, but at the same time, it didn't strike me as one of the more difficult parts of supporting TypeVarTuple. I also don't think that `Union[*Ts]` presents a problem if `Ts` is bound to an arbitrary-length tuple.

Yeah, there are no fundamental limitations to adding `Union[*Ts]`. IIRC we'd removed it because it was making the PEP harder to implement with no real use cases to justify it. I'm thinking of introducing it in the follow-up PEP for Map, where we will probably have more use cases. The alternative is to add it right here, but we're already delaying PEP approval because of the unbounded tuples amendment and this will delay it further. By deferring it, you don't have to delete your implementation either. Does that sound reasonable?

4. Guido:

> I'm guessing the reason that Matthew rather wants `Tensor[int, ..., int]` to mean what you would spell as `Tensor[int, Any, ..., int]`, is that `a[i, ..., j]` is used to take a slice from a numpy array `a` that drops the first and last dimension.

If we want to add user-friendly syntax like `Tensor[float32, ...]` as syntactic sugar for `Tensor[float32, *tuple[Any, ...]]`, I'm ok with doing that in the future (if there is user demand). Likewise for `Tensor[float32, ..., str]`, which would be equivalent to `Tensor[float32, *tuple[Any, ...], str]`. But, as Guido pointed out, this lack of syntax shouldn't constrain our semantics.

[1]: https://github.com/facebook/pyre-check/blob/c185dc1cb44e86dc60c1feea777fdb95...

[2]: https://www.typescriptlang.org/play?ts=4.5.0-beta#code/CYUwxgNghgTiAEAzArgOz...

On Thu, Nov 18, 2021 at 9:42 PM Guido van Rossum wrote:

> I may sound like a broken record, but I feel compelled to point out > that even though `tuple[int, str, ..., int]` is not legal *syntax* today, > and there is currently no way to cause such a type to exist, we could have > the concept (inside type checkers) without having syntax to spell it. > > I'm guessing the reason that Matthew rather wants `Tensor[int, ..., > int]` to mean what you would spell as `Tensor[int, Any, ..., int]`, is that > `a[i, ..., j]` is used to take a slice from a numpy array `a` that drops > the first and last dimension. > > Note that if we were to accept your syntax, an edge case of > `Tensor[int, str, ..., int]` would be `Tensor[int, int]` (i.e. the `str, > ...` collapses to zero dimensions), as it does for `tuple[int, ...]`. > > > On Thu, Nov 18, 2021 at 9:04 PM Eric Traut wrote: > >> I'm struggling to remember precisely what I was trying to express >> in that post. It was over a year ago, after all. :) >> >> Thinking about it now, it seems like `Unpack` should work OK with >> arbitrary-length types. There are some interesting edge cases to consider >> though. >> >> With a TypeVarTuple of known length, it's possible to do define a >> type like this: `tuple[int, *Ts, int]`. If `Ts` is then bound to >> `tuple[str, str]`, the resulting type is tuple[int, str, str, int]`. But if >> `Ts` is bound to an arbitrary-length tuple like `tuple[str, ...]`, what is >> the resulting type? Is it `tuple[int, str, ..., int]`? That's not a legal >> type, at least not today. Perhaps we can fix this by specifying that `*Ts` >> is not allowed for arbitrary-length tuples except in the last (right-most) >> type argument position? Or are you thinking that the type would be >> `tuple[int, *tuple[str, ...], int]`? That's pretty ugly. It would allow for >> arbitrary nesting (e.g. `tuple[int, *tuple[int, *tuple[str, ...], int], >> int]`). Pyright already has about a dozen places where it needs to >> special-case tuple unpacking and handle both arbitrary-length and >> fixed-length tuples. This would add significant complexity if every one of >> those places would need to be able to deal with nesting of unpacked ranges, >> some of them of arbitrary length. I don't think it would be a good idea to >> allow such a type. >> >> The interactions between unpacked TypeVarTuple and `Callable` also >> become more complex if we allow arbitrary-length tuples, but this is >> probably no more complex than `*args` today. >> >> I didn't realize that you had removed `Union[*Ts]` from the draft >> PEP. I've already implemented support for this in pyright. It's easy enough >> to remove, but at the same time, it didn't strike me as one of the more >> difficult parts of supporting TypeVarTuple. I also don't think that >> `Union[*Ts]` presents a problem if `Ts` is bound to an arbitrary-length >> tuple. >> >> Have you thought about how arbitrary-length tuple bindings would >> work with some of the type arithmetic operators that were recently proposed >> for working with dimensional transforms? Would those operators be undefined >> and therefore illegal if used with an arbitrary-length tuple binding? >> >> I know that Matthew would like to reserve the syntax `Tensor[Any, >> ...]` to mean something different in the future (point 1 above). My >> preference is to simply adopt the convention already established by >> `tuple`. The syntax `Tensor[*Tuple[Any, ...]]` is cumbersome and introduces >> added complexity as I noted above. Plus, the meaning of `tuple[Any, ...]` >> is already well established and broadly understood, so it's natural to >> assume that `Tensor[Any, ...]` has the same semantics. If we find in the >> future that there is really a need to specify "an arbitrary number of type >> parameters with arbitrary types", we could solve it in some other manner — >> one that doesn't conflict with existing conventions. >> >> -- >> >> Eric Traut >> Contributor to Pyright & Pylance >> Microsoft >> _______________________________________________ >> Typing-sig mailing list -- typing-sig@python.org >> To unsubscribe send an email to typing-sig-leave@python.org >> https://mail.python.org/mailman3/lists/typing-sig.python.org/ >> Member address: guido@python.org >> > > > -- > --Guido van Rossum (python.org/~guido) > *Pronouns: he/him **(why is my pronoun here?)* > http://feministing.com/2015/02/03/how-using-they-as-a-singular-pronoun-can-c... > _______________________________________________ > Typing-sig mailing list -- typing-sig@python.org > To unsubscribe send an email to typing-sig-leave@python.org > https://mail.python.org/mailman3/lists/typing-sig.python.org/ > Member address: gohanpra@gmail.com >

-- S Pradeep Kumar _______________________________________________ Typing-sig mailing list -- typing-sig@python.org To unsubscribe send an email to typing-sig-leave@python.org https://mail.python.org/mailman3/lists/typing-sig.python.org/ Member address: mrahtz@google.com

-- --Guido van Rossum (python.org/~guido) *Pronouns: he/him **(why is my pronoun here?)* http://feministing.com/2015/02/03/how-using-they-as-a-singular-pronoun-can-c...

-- S Pradeep Kumar

-- --Guido van Rossum (python.org/~guido) *Pronouns: he/him **(why is my pronoun here?)* http://feministing.com/2015/02/03/how-using-they-as-a-singular-pronoun-can-c...

-- S Pradeep Kumar

877

Age (days ago)

885

Last active (days ago)

List overview

Download

9 comments

4 participants

participants (4)

Eric Traut
Guido van Rossum
Matthew Rahtz
S Pradeep Kumar

PEP 646: Revisiting binding to arbitrary-length types

tags

participants (4)