PEP 646 interesting case
I came across a case where I think PEP 646 would provide relief, but I don't know how to spell it. Suppose we have a class Property[T]: ``` class Property(Generic[T]): ... ``` Now we have a function tuples that takes several Property arguments with different type parameters and returns a Property whose type parameter is a Tuple: ``` def tuples(*possibilities: Possibility[?]) -> Possibility[?]: ``` The idea is that you call ``` p1: Possibility[int] p2: Possibility[str] p = tuples(p1, p2) reveal_type(p) # Possibility[Tuple[int, str]] ``` Somehow I can't figure out how we're supposed to write that with PEP 646. The best I can come up with is ``` def tuples(*possibilities: *Possibility[*Ts]) -> Possibility[Tuple[*Ts]]:` ``` but the first `*Ts` feels wrong -- it looks as if Possibility takes multiple type parameters, but that's not the case. Somehow my brain wants to use a loop construct, e.g. ``` def tuples(*possibilities: *Possibility[T] for T in Ts) -> Possibility[Tuple[*Ts]]: ``` Does anyone recall how this is supposed to be handled? Or was it put off till a later PEP? (That would be a shame, since this is the first variadic signature I've encountered in the wild in a long time.) PS. If you really want to know: https://github.com/DRMacIver/minithesis/pull/9 -- --Guido van Rossum (python.org/~guido) *Pronouns: he/him **(why is my pronoun here?)* <http://feministing.com/2015/02/03/how-using-they-as-a-singular-pronoun-can-change-the-world/>
Somehow my brain wants to use a loop construct
Or was it put off till a later PEP? (That would be a shame, since this is
This would use the Map operator: ``` def tuples(*possibilities: *Map[Possibility, Ts]) -> Possibility[Tuple[*Ts]]: ... p1: Possibility[int] p2: Possibility[str] p = tuples(p1, p2) ``` + Ts will be bound to Tuple[int, str] + That way, Map[Possibility, Ts] === Tuple[Possibility[int], Possibility[str]] + Return type becomes Possibility[Tuple[int, str]] the first variadic signature I've encountered in the wild in a long time.) Yeah, this was part of earlier drafts but we deferred this to a later PEP. There were objections since this would require checking that the type argument to Map was a higher-order type like `Possibility` or a generic alias. We would need this for functions like zip and asyncio.gather as well (but not really for tensor typing). Let me know if it would be better to incorporate this within PEP 646 itself instead of a follow-up PEP. On Fri, May 21, 2021 at 11:24 AM Guido van Rossum <guido@python.org> wrote:
I came across a case where I think PEP 646 would provide relief, but I don't know how to spell it.
Suppose we have a class Property[T]: ``` class Property(Generic[T]): ... ``` Now we have a function tuples that takes several Property arguments with different type parameters and returns a Property whose type parameter is a Tuple: ``` def tuples(*possibilities: Possibility[?]) -> Possibility[?]: ``` The idea is that you call ``` p1: Possibility[int] p2: Possibility[str] p = tuples(p1, p2) reveal_type(p) # Possibility[Tuple[int, str]] ``` Somehow I can't figure out how we're supposed to write that with PEP 646. The best I can come up with is ``` def tuples(*possibilities: *Possibility[*Ts]) -> Possibility[Tuple[*Ts]]:` ``` but the first `*Ts` feels wrong -- it looks as if Possibility takes multiple type parameters, but that's not the case. Somehow my brain wants to use a loop construct, e.g. ``` def tuples(*possibilities: *Possibility[T] for T in Ts) -> Possibility[Tuple[*Ts]]: ``` Does anyone recall how this is supposed to be handled? Or was it put off till a later PEP? (That would be a shame, since this is the first variadic signature I've encountered in the wild in a long time.)
PS. If you really want to know: https://github.com/DRMacIver/minithesis/pull/9
-- --Guido van Rossum (python.org/~guido) *Pronouns: he/him **(why is my pronoun here?)* <http://feministing.com/2015/02/03/how-using-they-as-a-singular-pronoun-can-change-the-world/> _______________________________________________ Typing-sig mailing list -- typing-sig@python.org To unsubscribe send an email to typing-sig-leave@python.org https://mail.python.org/mailman3/lists/typing-sig.python.org/ Member address: gohanpra@gmail.com
-- S Pradeep Kumar
Given that much of my *own* excitement for variadics comes from the prospect of typing zip and gather, and that we have some time before this goes into 3.11, I'd be in favor of adding Map[] to PEP 646. On Fri, May 21, 2021 at 11:46 AM S Pradeep Kumar <gohanpra@gmail.com> wrote:
Somehow my brain wants to use a loop construct
This would use the Map operator:
``` def tuples(*possibilities: *Map[Possibility, Ts]) -> Possibility[Tuple[*Ts]]: ...
p1: Possibility[int] p2: Possibility[str] p = tuples(p1, p2) ```
+ Ts will be bound to Tuple[int, str] + That way, Map[Possibility, Ts] === Tuple[Possibility[int], Possibility[str]] + Return type becomes Possibility[Tuple[int, str]]
Or was it put off till a later PEP? (That would be a shame, since this is the first variadic signature I've encountered in the wild in a long time.)
Yeah, this was part of earlier drafts but we deferred this to a later PEP. There were objections since this would require checking that the type argument to Map was a higher-order type like `Possibility` or a generic alias.
We would need this for functions like zip and asyncio.gather as well (but not really for tensor typing). Let me know if it would be better to incorporate this within PEP 646 itself instead of a follow-up PEP.
On Fri, May 21, 2021 at 11:24 AM Guido van Rossum <guido@python.org> wrote:
I came across a case where I think PEP 646 would provide relief, but I don't know how to spell it.
Suppose we have a class Property[T]: ``` class Property(Generic[T]): ... ``` Now we have a function tuples that takes several Property arguments with different type parameters and returns a Property whose type parameter is a Tuple: ``` def tuples(*possibilities: Possibility[?]) -> Possibility[?]: ``` The idea is that you call ``` p1: Possibility[int] p2: Possibility[str] p = tuples(p1, p2) reveal_type(p) # Possibility[Tuple[int, str]] ``` Somehow I can't figure out how we're supposed to write that with PEP 646. The best I can come up with is ``` def tuples(*possibilities: *Possibility[*Ts]) -> Possibility[Tuple[*Ts]]:` ``` but the first `*Ts` feels wrong -- it looks as if Possibility takes multiple type parameters, but that's not the case. Somehow my brain wants to use a loop construct, e.g. ``` def tuples(*possibilities: *Possibility[T] for T in Ts) -> Possibility[Tuple[*Ts]]: ``` Does anyone recall how this is supposed to be handled? Or was it put off till a later PEP? (That would be a shame, since this is the first variadic signature I've encountered in the wild in a long time.)
PS. If you really want to know: https://github.com/DRMacIver/minithesis/pull/9
-- --Guido van Rossum (python.org/~guido) *Pronouns: he/him **(why is my pronoun here?)* <http://feministing.com/2015/02/03/how-using-they-as-a-singular-pronoun-can-change-the-world/> _______________________________________________ Typing-sig mailing list -- typing-sig@python.org To unsubscribe send an email to typing-sig-leave@python.org https://mail.python.org/mailman3/lists/typing-sig.python.org/ Member address: gohanpra@gmail.com
-- S Pradeep Kumar
-- --Guido van Rossum (python.org/~guido) *Pronouns: he/him **(why is my pronoun here?)* <http://feministing.com/2015/02/03/how-using-they-as-a-singular-pronoun-can-change-the-world/>
What exactly are the proposed semantics of Map? I was recently trying to write the type for a decorator which is the moral equivalent of the following: def preserve_none(fn: Callable[..., T]) -> Callable[..., Optional[T]]: def wrapper(*args: Any) -> Optional[T]: if any(arg is None for arg in args): return None return fn(*args) return wrapper Would I be able to write this as: def preserve_none(fn: Callable[*Ts, T]) -> Callable[[Map[Optional, Ts]], Optional[T]]: def wrapper(*args: *Map[Optional, Ts]) -> Optional[T]: ... What is the type of Map[Dict, Ts]? Is it an error? Is there any way to write Map[Dict, Zip[Ks, Vs]]? — Caleb On Fri, May 21, 2021 at 12:08 PM Guido van Rossum <guido@python.org> wrote:
Given that much of my *own* excitement for variadics comes from the prospect of typing zip and gather, and that we have some time before this goes into 3.11, I'd be in favor of adding Map[] to PEP 646.
On Fri, May 21, 2021 at 11:46 AM S Pradeep Kumar <gohanpra@gmail.com> wrote:
Somehow my brain wants to use a loop construct
This would use the Map operator:
``` def tuples(*possibilities: *Map[Possibility, Ts]) -> Possibility[Tuple[*Ts]]: ...
p1: Possibility[int] p2: Possibility[str] p = tuples(p1, p2) ```
+ Ts will be bound to Tuple[int, str] + That way, Map[Possibility, Ts] === Tuple[Possibility[int], Possibility[str]] + Return type becomes Possibility[Tuple[int, str]]
Or was it put off till a later PEP? (That would be a shame, since this is the first variadic signature I've encountered in the wild in a long time.)
Yeah, this was part of earlier drafts but we deferred this to a later PEP. There were objections since this would require checking that the type argument to Map was a higher-order type like `Possibility` or a generic alias.
We would need this for functions like zip and asyncio.gather as well (but not really for tensor typing). Let me know if it would be better to incorporate this within PEP 646 itself instead of a follow-up PEP.
On Fri, May 21, 2021 at 11:24 AM Guido van Rossum <guido@python.org> wrote:
I came across a case where I think PEP 646 would provide relief, but I don't know how to spell it.
Suppose we have a class Property[T]: ``` class Property(Generic[T]): ... ``` Now we have a function tuples that takes several Property arguments with different type parameters and returns a Property whose type parameter is a Tuple: ``` def tuples(*possibilities: Possibility[?]) -> Possibility[?]: ``` The idea is that you call ``` p1: Possibility[int] p2: Possibility[str] p = tuples(p1, p2) reveal_type(p) # Possibility[Tuple[int, str]] ``` Somehow I can't figure out how we're supposed to write that with PEP 646. The best I can come up with is ``` def tuples(*possibilities: *Possibility[*Ts]) -> Possibility[Tuple[*Ts]]:` ``` but the first `*Ts` feels wrong -- it looks as if Possibility takes multiple type parameters, but that's not the case. Somehow my brain wants to use a loop construct, e.g. ``` def tuples(*possibilities: *Possibility[T] for T in Ts) -> Possibility[Tuple[*Ts]]: ``` Does anyone recall how this is supposed to be handled? Or was it put off till a later PEP? (That would be a shame, since this is the first variadic signature I've encountered in the wild in a long time.)
PS. If you really want to know: https://github.com/DRMacIver/minithesis/pull/9
-- --Guido van Rossum (python.org/~guido) *Pronouns: he/him **(why is my pronoun here?)* <http://feministing.com/2015/02/03/how-using-they-as-a-singular-pronoun-can-change-the-world/> _______________________________________________ Typing-sig mailing list -- typing-sig@python.org To unsubscribe send an email to typing-sig-leave@python.org https://mail.python.org/mailman3/lists/typing-sig.python.org/ Member address: gohanpra@gmail.com
-- S Pradeep Kumar
-- --Guido van Rossum (python.org/~guido) *Pronouns: he/him **(why is my pronoun here?)* <http://feministing.com/2015/02/03/how-using-they-as-a-singular-pronoun-can-change-the-world/> _______________________________________________ Typing-sig mailing list -- typing-sig@python.org To unsubscribe send an email to typing-sig-leave@python.org https://mail.python.org/mailman3/lists/typing-sig.python.org/ Member address: donovick@cs.stanford.edu
IIRC one of the reasons we punted on Map[] was questions like these. Map[Dict, Ts] would be an error. Do you have a real use case for Zip[]? I think we should at least explore using comprehension syntax instead of Map[] -- so you could write [Optional[T] for T in Ts] instead of Map[Optional, Ts], but many other things too, e.g. [Dict[str, T] for T in Ts] or [Tuple[T, List[T]] for T in Ts]. Note that this defines T as a local variable (there should not be a TypeVar('T') in scope, or at least it would be ignored). A problem with comprehensions would be what they would evaluate to at runtime. Though I think that shouldn't stop us. I suspect the best we can do is either produce [] or [Tuple[T, List[T]] as the value to be put in __annotations__. On Fri, May 21, 2021 at 4:24 PM Caleb Donovick <donovick@cs.stanford.edu> wrote:
What exactly are the proposed semantics of Map?
I was recently trying to write the type for a decorator which is the moral equivalent of the following:
def preserve_none(fn: Callable[..., T]) -> Callable[..., Optional[T]]: def wrapper(*args: Any) -> Optional[T]: if any(arg is None for arg in args): return None return fn(*args) return wrapper
Would I be able to write this as:
def preserve_none(fn: Callable[*Ts, T]) -> Callable[[Map[Optional, Ts]], Optional[T]]: def wrapper(*args: *Map[Optional, Ts]) -> Optional[T]: ...
What is the type of Map[Dict, Ts]? Is it an error?
Is there any way to write Map[Dict, Zip[Ks, Vs]]?
— Caleb
On Fri, May 21, 2021 at 12:08 PM Guido van Rossum <guido@python.org> wrote:
Given that much of my *own* excitement for variadics comes from the prospect of typing zip and gather, and that we have some time before this goes into 3.11, I'd be in favor of adding Map[] to PEP 646.
On Fri, May 21, 2021 at 11:46 AM S Pradeep Kumar <gohanpra@gmail.com> wrote:
Somehow my brain wants to use a loop construct
This would use the Map operator:
``` def tuples(*possibilities: *Map[Possibility, Ts]) -> Possibility[Tuple[*Ts]]: ...
p1: Possibility[int] p2: Possibility[str] p = tuples(p1, p2) ```
+ Ts will be bound to Tuple[int, str] + That way, Map[Possibility, Ts] === Tuple[Possibility[int], Possibility[str]] + Return type becomes Possibility[Tuple[int, str]]
Or was it put off till a later PEP? (That would be a shame, since this is the first variadic signature I've encountered in the wild in a long time.)
Yeah, this was part of earlier drafts but we deferred this to a later PEP. There were objections since this would require checking that the type argument to Map was a higher-order type like `Possibility` or a generic alias.
We would need this for functions like zip and asyncio.gather as well (but not really for tensor typing). Let me know if it would be better to incorporate this within PEP 646 itself instead of a follow-up PEP.
On Fri, May 21, 2021 at 11:24 AM Guido van Rossum <guido@python.org> wrote:
I came across a case where I think PEP 646 would provide relief, but I don't know how to spell it.
Suppose we have a class Property[T]: ``` class Property(Generic[T]): ... ``` Now we have a function tuples that takes several Property arguments with different type parameters and returns a Property whose type parameter is a Tuple: ``` def tuples(*possibilities: Possibility[?]) -> Possibility[?]: ``` The idea is that you call ``` p1: Possibility[int] p2: Possibility[str] p = tuples(p1, p2) reveal_type(p) # Possibility[Tuple[int, str]] ``` Somehow I can't figure out how we're supposed to write that with PEP 646. The best I can come up with is ``` def tuples(*possibilities: *Possibility[*Ts]) -> Possibility[Tuple[*Ts]]:` ``` but the first `*Ts` feels wrong -- it looks as if Possibility takes multiple type parameters, but that's not the case. Somehow my brain wants to use a loop construct, e.g. ``` def tuples(*possibilities: *Possibility[T] for T in Ts) -> Possibility[Tuple[*Ts]]: ``` Does anyone recall how this is supposed to be handled? Or was it put off till a later PEP? (That would be a shame, since this is the first variadic signature I've encountered in the wild in a long time.)
PS. If you really want to know: https://github.com/DRMacIver/minithesis/pull/9
-- --Guido van Rossum (python.org/~guido) *Pronouns: he/him **(why is my pronoun here?)* <http://feministing.com/2015/02/03/how-using-they-as-a-singular-pronoun-can-change-the-world/> _______________________________________________ Typing-sig mailing list -- typing-sig@python.org To unsubscribe send an email to typing-sig-leave@python.org https://mail.python.org/mailman3/lists/typing-sig.python.org/ Member address: gohanpra@gmail.com
-- S Pradeep Kumar
-- --Guido van Rossum (python.org/~guido) *Pronouns: he/him **(why is my pronoun here?)* <http://feministing.com/2015/02/03/how-using-they-as-a-singular-pronoun-can-change-the-world/> _______________________________________________ Typing-sig mailing list -- typing-sig@python.org To unsubscribe send an email to typing-sig-leave@python.org https://mail.python.org/mailman3/lists/typing-sig.python.org/ Member address: donovick@cs.stanford.edu
-- --Guido van Rossum (python.org/~guido) *Pronouns: he/him **(why is my pronoun here?)* <http://feministing.com/2015/02/03/how-using-they-as-a-singular-pronoun-can-change-the-world/>
In Hypothesis, we currently just erase the element types: ```python def tuples(*args: SearchStrategy) -> SearchStrategy[tuple]: ``` but for our 'union operator' `one_of()` we write out overloads for up to five arguments [1]. Maybe we should add similar overloads for tuples! In both cases PEP-646 would be a much nicer way to express this for both authors and consumers of the annotations. I also like the comprehension-syntax, but I'm a little more cautious about the runtime representation - which is consumed by Hypothesis' `from_type()` strategy directly and via argument inference in `builds()`. If the value of `__annotations__` can't fully represent what was written, I'd appreciate a way to tell that it's a simplification so that we can 'refuse the temptation to guess' and ask the user for more information. [1] https://github.com/HypothesisWorks/hypothesis/blob/37532c8fe168a6b4a4f287000... On Sat, 22 May 2021 at 11:29, Guido van Rossum <guido@python.org> wrote:
IIRC one of the reasons we punted on Map[] was questions like these.
Map[Dict, Ts] would be an error.
Do you have a real use case for Zip[]?
I think we should at least explore using comprehension syntax instead of Map[] -- so you could write [Optional[T] for T in Ts] instead of Map[Optional, Ts], but many other things too, e.g. [Dict[str, T] for T in Ts] or [Tuple[T, List[T]] for T in Ts].
Note that this defines T as a local variable (there should not be a TypeVar('T') in scope, or at least it would be ignored).
A problem with comprehensions would be what they would evaluate to at runtime. Though I think that shouldn't stop us. I suspect the best we can do is either produce [] or [Tuple[T, List[T]] as the value to be put in __annotations__.
On Fri, May 21, 2021 at 4:24 PM Caleb Donovick <donovick@cs.stanford.edu> wrote:
What exactly are the proposed semantics of Map?
I was recently trying to write the type for a decorator which is the moral equivalent of the following:
def preserve_none(fn: Callable[..., T]) -> Callable[..., Optional[T]]: def wrapper(*args: Any) -> Optional[T]: if any(arg is None for arg in args): return None return fn(*args) return wrapper
Would I be able to write this as:
def preserve_none(fn: Callable[*Ts, T]) -> Callable[[Map[Optional, Ts]], Optional[T]]: def wrapper(*args: *Map[Optional, Ts]) -> Optional[T]: ...
What is the type of Map[Dict, Ts]? Is it an error?
Is there any way to write Map[Dict, Zip[Ks, Vs]]?
— Caleb
On Fri, May 21, 2021 at 12:08 PM Guido van Rossum <guido@python.org> wrote:
Given that much of my *own* excitement for variadics comes from the prospect of typing zip and gather, and that we have some time before this goes into 3.11, I'd be in favor of adding Map[] to PEP 646.
On Fri, May 21, 2021 at 11:46 AM S Pradeep Kumar <gohanpra@gmail.com> wrote:
Somehow my brain wants to use a loop construct
This would use the Map operator:
``` def tuples(*possibilities: *Map[Possibility, Ts]) -> Possibility[Tuple[*Ts]]: ...
p1: Possibility[int] p2: Possibility[str] p = tuples(p1, p2) ```
+ Ts will be bound to Tuple[int, str] + That way, Map[Possibility, Ts] === Tuple[Possibility[int], Possibility[str]] + Return type becomes Possibility[Tuple[int, str]]
Or was it put off till a later PEP? (That would be a shame, since this is the first variadic signature I've encountered in the wild in a long time.)
Yeah, this was part of earlier drafts but we deferred this to a later PEP. There were objections since this would require checking that the type argument to Map was a higher-order type like `Possibility` or a generic alias.
We would need this for functions like zip and asyncio.gather as well (but not really for tensor typing). Let me know if it would be better to incorporate this within PEP 646 itself instead of a follow-up PEP.
On Fri, May 21, 2021 at 11:24 AM Guido van Rossum <guido@python.org> wrote:
I came across a case where I think PEP 646 would provide relief, but I don't know how to spell it.
Suppose we have a class Property[T]: ``` class Property(Generic[T]): ... ``` Now we have a function tuples that takes several Property arguments with different type parameters and returns a Property whose type parameter is a Tuple: ``` def tuples(*possibilities: Possibility[?]) -> Possibility[?]: ``` The idea is that you call ``` p1: Possibility[int] p2: Possibility[str] p = tuples(p1, p2) reveal_type(p) # Possibility[Tuple[int, str]] ``` Somehow I can't figure out how we're supposed to write that with PEP 646. The best I can come up with is ``` def tuples(*possibilities: *Possibility[*Ts]) -> Possibility[Tuple[*Ts]]:` ``` but the first `*Ts` feels wrong -- it looks as if Possibility takes multiple type parameters, but that's not the case. Somehow my brain wants to use a loop construct, e.g. ``` def tuples(*possibilities: *Possibility[T] for T in Ts) -> Possibility[Tuple[*Ts]]: ``` Does anyone recall how this is supposed to be handled? Or was it put off till a later PEP? (That would be a shame, since this is the first variadic signature I've encountered in the wild in a long time.)
PS. If you really want to know: https://github.com/DRMacIver/minithesis/pull/9
-- --Guido van Rossum (python.org/~guido) *Pronouns: he/him **(why is my pronoun here?)* <http://feministing.com/2015/02/03/how-using-they-as-a-singular-pronoun-can-change-the-world/> _______________________________________________ Typing-sig mailing list -- typing-sig@python.org To unsubscribe send an email to typing-sig-leave@python.org https://mail.python.org/mailman3/lists/typing-sig.python.org/ Member address: gohanpra@gmail.com
-- S Pradeep Kumar
-- --Guido van Rossum (python.org/~guido) *Pronouns: he/him **(why is my pronoun here?)* <http://feministing.com/2015/02/03/how-using-they-as-a-singular-pronoun-can-change-the-world/> _______________________________________________ Typing-sig mailing list -- typing-sig@python.org To unsubscribe send an email to typing-sig-leave@python.org https://mail.python.org/mailman3/lists/typing-sig.python.org/ Member address: donovick@cs.stanford.edu
-- --Guido van Rossum (python.org/~guido) *Pronouns: he/him **(why is my pronoun here?)* <http://feministing.com/2015/02/03/how-using-they-as-a-singular-pronoun-can-change-the-world/> _______________________________________________ Typing-sig mailing list -- typing-sig@python.org To unsubscribe send an email to typing-sig-leave@python.org https://mail.python.org/mailman3/lists/typing-sig.python.org/ Member address: zac.hatfield.dodds@gmail.com
so you could write [Optional[T] for T in Ts] instead of Map[Optional, Ts], but many other things too, e.g. [Dict[str, T] for T in Ts] or [Tuple[T, List[T]] for T in Ts].
@Guido: We would use a generic alias for these: ``` DictStr = Dict[str, T] MyTuple = Tuple[T, List[T]] Map[DictStr, Ts] Map[MyTuple, Ts] ``` There are other ideas we could also consider, such as inline generic aliases (Map[Tuple[T, List[T]], Ts]) and the comprehension syntax you mentioned. Given these questions and possible syntax changes, I strongly favor keeping this as a separate, follow-up PEP. There seems to be broad consensus in typing-sig for PEP 646 as such, so I'd prefer iterating on the Map proposals separately instead of changing PEP 646 itself. We should be able to get both PEPs in for 3.11. On Fri, May 21, 2021 at 8:18 PM Zac Hatfield Dodds < zac.hatfield.dodds@gmail.com> wrote:
In Hypothesis, we currently just erase the element types: ```python def tuples(*args: SearchStrategy) -> SearchStrategy[tuple]: ``` but for our 'union operator' `one_of()` we write out overloads for up to five arguments [1]. Maybe we should add similar overloads for tuples! In both cases PEP-646 would be a much nicer way to express this for both authors and consumers of the annotations.
I also like the comprehension-syntax, but I'm a little more cautious about the runtime representation - which is consumed by Hypothesis' `from_type()` strategy directly and via argument inference in `builds()`. If the value of `__annotations__` can't fully represent what was written, I'd appreciate a way to tell that it's a simplification so that we can 'refuse the temptation to guess' and ask the user for more information.
[1] https://github.com/HypothesisWorks/hypothesis/blob/37532c8fe168a6b4a4f287000...
On Sat, 22 May 2021 at 11:29, Guido van Rossum <guido@python.org> wrote:
IIRC one of the reasons we punted on Map[] was questions like these.
Map[Dict, Ts] would be an error.
Do you have a real use case for Zip[]?
I think we should at least explore using comprehension syntax instead of Map[] -- so you could write [Optional[T] for T in Ts] instead of Map[Optional, Ts], but many other things too, e.g. [Dict[str, T] for T in Ts] or [Tuple[T, List[T]] for T in Ts].
Note that this defines T as a local variable (there should not be a TypeVar('T') in scope, or at least it would be ignored).
A problem with comprehensions would be what they would evaluate to at runtime. Though I think that shouldn't stop us. I suspect the best we can do is either produce [] or [Tuple[T, List[T]] as the value to be put in __annotations__.
On Fri, May 21, 2021 at 4:24 PM Caleb Donovick <donovick@cs.stanford.edu> wrote:
What exactly are the proposed semantics of Map?
I was recently trying to write the type for a decorator which is the moral equivalent of the following:
def preserve_none(fn: Callable[..., T]) -> Callable[..., Optional[T]]: def wrapper(*args: Any) -> Optional[T]: if any(arg is None for arg in args): return None return fn(*args) return wrapper
Would I be able to write this as:
def preserve_none(fn: Callable[*Ts, T]) -> Callable[[Map[Optional, Ts]], Optional[T]]: def wrapper(*args: *Map[Optional, Ts]) -> Optional[T]: ...
What is the type of Map[Dict, Ts]? Is it an error?
Is there any way to write Map[Dict, Zip[Ks, Vs]]?
— Caleb
On Fri, May 21, 2021 at 12:08 PM Guido van Rossum <guido@python.org> wrote:
Given that much of my *own* excitement for variadics comes from the prospect of typing zip and gather, and that we have some time before this goes into 3.11, I'd be in favor of adding Map[] to PEP 646.
On Fri, May 21, 2021 at 11:46 AM S Pradeep Kumar <gohanpra@gmail.com> wrote:
Somehow my brain wants to use a loop construct
This would use the Map operator:
``` def tuples(*possibilities: *Map[Possibility, Ts]) -> Possibility[Tuple[*Ts]]: ...
p1: Possibility[int] p2: Possibility[str] p = tuples(p1, p2) ```
+ Ts will be bound to Tuple[int, str] + That way, Map[Possibility, Ts] === Tuple[Possibility[int], Possibility[str]] + Return type becomes Possibility[Tuple[int, str]]
Or was it put off till a later PEP? (That would be a shame, since this is the first variadic signature I've encountered in the wild in a long time.)
Yeah, this was part of earlier drafts but we deferred this to a later PEP. There were objections since this would require checking that the type argument to Map was a higher-order type like `Possibility` or a generic alias.
We would need this for functions like zip and asyncio.gather as well (but not really for tensor typing). Let me know if it would be better to incorporate this within PEP 646 itself instead of a follow-up PEP.
On Fri, May 21, 2021 at 11:24 AM Guido van Rossum <guido@python.org> wrote:
I came across a case where I think PEP 646 would provide relief, but I don't know how to spell it.
Suppose we have a class Property[T]: ``` class Property(Generic[T]): ... ``` Now we have a function tuples that takes several Property arguments with different type parameters and returns a Property whose type parameter is a Tuple: ``` def tuples(*possibilities: Possibility[?]) -> Possibility[?]: ``` The idea is that you call ``` p1: Possibility[int] p2: Possibility[str] p = tuples(p1, p2) reveal_type(p) # Possibility[Tuple[int, str]] ``` Somehow I can't figure out how we're supposed to write that with PEP 646. The best I can come up with is ``` def tuples(*possibilities: *Possibility[*Ts]) -> Possibility[Tuple[*Ts]]:` ``` but the first `*Ts` feels wrong -- it looks as if Possibility takes multiple type parameters, but that's not the case. Somehow my brain wants to use a loop construct, e.g. ``` def tuples(*possibilities: *Possibility[T] for T in Ts) -> Possibility[Tuple[*Ts]]: ``` Does anyone recall how this is supposed to be handled? Or was it put off till a later PEP? (That would be a shame, since this is the first variadic signature I've encountered in the wild in a long time.)
PS. If you really want to know: https://github.com/DRMacIver/minithesis/pull/9
-- --Guido van Rossum (python.org/~guido) *Pronouns: he/him **(why is my pronoun here?)* <http://feministing.com/2015/02/03/how-using-they-as-a-singular-pronoun-can-change-the-world/> _______________________________________________ Typing-sig mailing list -- typing-sig@python.org To unsubscribe send an email to typing-sig-leave@python.org https://mail.python.org/mailman3/lists/typing-sig.python.org/ Member address: gohanpra@gmail.com
-- S Pradeep Kumar
-- --Guido van Rossum (python.org/~guido) *Pronouns: he/him **(why is my pronoun here?)* <http://feministing.com/2015/02/03/how-using-they-as-a-singular-pronoun-can-change-the-world/> _______________________________________________ Typing-sig mailing list -- typing-sig@python.org To unsubscribe send an email to typing-sig-leave@python.org https://mail.python.org/mailman3/lists/typing-sig.python.org/ Member address: donovick@cs.stanford.edu
-- --Guido van Rossum (python.org/~guido) *Pronouns: he/him **(why is my pronoun here?)* <http://feministing.com/2015/02/03/how-using-they-as-a-singular-pronoun-can-change-the-world/> _______________________________________________ Typing-sig mailing list -- typing-sig@python.org To unsubscribe send an email to typing-sig-leave@python.org https://mail.python.org/mailman3/lists/typing-sig.python.org/ Member address: zac.hatfield.dodds@gmail.com
-- S Pradeep Kumar
Sorry to be slow on weighing in on this - had a paper deadline last week. I agree with Pradeep - let's keep Map for a follow-up PEP. Even though we have a while until Python 3.11, I want to get PEP 646 wrapped up first so as to remove the feeling of doom that comes with a large project being unfinished. As Pradeep alludes, I can imagine there could be quite a lot more discussion about whether there's a better way to do Map (particularly in light of discussions at the Typing Summit about more intuitive syntax for typing in general), and I don't think we should bottleneck PEP 646 on that discussion. Since it's one of the things you're most excited about with variadics though, Guido, and you've helped so much on this journey so far, I'll tell you what - I'll commit to writing the PEP for Map + gathering feedback on it as the first thing I'll do after we're done with the implementation of PEP 646 + interating on it with the SC :) On Sat, 22 May 2021 at 04:42, S Pradeep Kumar <gohanpra@gmail.com> wrote:
so you could write [Optional[T] for T in Ts] instead of Map[Optional, Ts], but many other things too, e.g. [Dict[str, T] for T in Ts] or [Tuple[T, List[T]] for T in Ts].
@Guido: We would use a generic alias for these:
``` DictStr = Dict[str, T] MyTuple = Tuple[T, List[T]]
Map[DictStr, Ts] Map[MyTuple, Ts] ```
There are other ideas we could also consider, such as inline generic aliases (Map[Tuple[T, List[T]], Ts]) and the comprehension syntax you mentioned.
Given these questions and possible syntax changes, I strongly favor keeping this as a separate, follow-up PEP. There seems to be broad consensus in typing-sig for PEP 646 as such, so I'd prefer iterating on the Map proposals separately instead of changing PEP 646 itself. We should be able to get both PEPs in for 3.11.
On Fri, May 21, 2021 at 8:18 PM Zac Hatfield Dodds < zac.hatfield.dodds@gmail.com> wrote:
In Hypothesis, we currently just erase the element types: ```python def tuples(*args: SearchStrategy) -> SearchStrategy[tuple]: ``` but for our 'union operator' `one_of()` we write out overloads for up to five arguments [1]. Maybe we should add similar overloads for tuples! In both cases PEP-646 would be a much nicer way to express this for both authors and consumers of the annotations.
I also like the comprehension-syntax, but I'm a little more cautious about the runtime representation - which is consumed by Hypothesis' `from_type()` strategy directly and via argument inference in `builds()`. If the value of `__annotations__` can't fully represent what was written, I'd appreciate a way to tell that it's a simplification so that we can 'refuse the temptation to guess' and ask the user for more information.
[1] https://github.com/HypothesisWorks/hypothesis/blob/37532c8fe168a6b4a4f287000...
On Sat, 22 May 2021 at 11:29, Guido van Rossum <guido@python.org> wrote:
IIRC one of the reasons we punted on Map[] was questions like these.
Map[Dict, Ts] would be an error.
Do you have a real use case for Zip[]?
I think we should at least explore using comprehension syntax instead of Map[] -- so you could write [Optional[T] for T in Ts] instead of Map[Optional, Ts], but many other things too, e.g. [Dict[str, T] for T in Ts] or [Tuple[T, List[T]] for T in Ts].
Note that this defines T as a local variable (there should not be a TypeVar('T') in scope, or at least it would be ignored).
A problem with comprehensions would be what they would evaluate to at runtime. Though I think that shouldn't stop us. I suspect the best we can do is either produce [] or [Tuple[T, List[T]] as the value to be put in __annotations__.
On Fri, May 21, 2021 at 4:24 PM Caleb Donovick <donovick@cs.stanford.edu> wrote:
What exactly are the proposed semantics of Map?
I was recently trying to write the type for a decorator which is the moral equivalent of the following:
def preserve_none(fn: Callable[..., T]) -> Callable[..., Optional[T]]: def wrapper(*args: Any) -> Optional[T]: if any(arg is None for arg in args): return None return fn(*args) return wrapper
Would I be able to write this as:
def preserve_none(fn: Callable[*Ts, T]) -> Callable[[Map[Optional, Ts]], Optional[T]]: def wrapper(*args: *Map[Optional, Ts]) -> Optional[T]: ...
What is the type of Map[Dict, Ts]? Is it an error?
Is there any way to write Map[Dict, Zip[Ks, Vs]]?
— Caleb
On Fri, May 21, 2021 at 12:08 PM Guido van Rossum <guido@python.org> wrote:
Given that much of my *own* excitement for variadics comes from the prospect of typing zip and gather, and that we have some time before this goes into 3.11, I'd be in favor of adding Map[] to PEP 646.
On Fri, May 21, 2021 at 11:46 AM S Pradeep Kumar <gohanpra@gmail.com> wrote:
> Somehow my brain wants to use a loop construct
This would use the Map operator:
``` def tuples(*possibilities: *Map[Possibility, Ts]) -> Possibility[Tuple[*Ts]]: ...
p1: Possibility[int] p2: Possibility[str] p = tuples(p1, p2) ```
+ Ts will be bound to Tuple[int, str] + That way, Map[Possibility, Ts] === Tuple[Possibility[int], Possibility[str]] + Return type becomes Possibility[Tuple[int, str]]
> Or was it put off till a later PEP? (That would be a shame, since this is the first variadic signature I've encountered in the wild in a long time.)
Yeah, this was part of earlier drafts but we deferred this to a later PEP. There were objections since this would require checking that the type argument to Map was a higher-order type like `Possibility` or a generic alias.
We would need this for functions like zip and asyncio.gather as well (but not really for tensor typing). Let me know if it would be better to incorporate this within PEP 646 itself instead of a follow-up PEP.
On Fri, May 21, 2021 at 11:24 AM Guido van Rossum <guido@python.org> wrote:
> I came across a case where I think PEP 646 would provide relief, but > I don't know how to spell it. > > Suppose we have a class Property[T]: > ``` > class Property(Generic[T]): > ... > ``` > Now we have a function tuples that takes several Property arguments > with different type parameters and returns a Property whose type parameter > is a Tuple: > ``` > def tuples(*possibilities: Possibility[?]) -> Possibility[?]: > ``` > The idea is that you call > ``` > p1: Possibility[int] > p2: Possibility[str] > p = tuples(p1, p2) > reveal_type(p) # Possibility[Tuple[int, str]] > ``` > Somehow I can't figure out how we're supposed to write that with PEP > 646. The best I can come up with is > ``` > def tuples(*possibilities: *Possibility[*Ts]) -> > Possibility[Tuple[*Ts]]:` > ``` > but the first `*Ts` feels wrong -- it looks as if Possibility takes > multiple type parameters, but that's not the case. Somehow my brain wants > to use a loop construct, e.g. > ``` > def tuples(*possibilities: *Possibility[T] for T in Ts) -> > Possibility[Tuple[*Ts]]: > ``` > Does anyone recall how this is supposed to be handled? Or was it put > off till a later PEP? (That would be a shame, since this is the first > variadic signature I've encountered in the wild in a long time.) > > PS. If you really want to know: > https://github.com/DRMacIver/minithesis/pull/9 > > -- > --Guido van Rossum (python.org/~guido) > *Pronouns: he/him **(why is my pronoun here?)* > <http://feministing.com/2015/02/03/how-using-they-as-a-singular-pronoun-can-change-the-world/> > _______________________________________________ > Typing-sig mailing list -- typing-sig@python.org > To unsubscribe send an email to typing-sig-leave@python.org > https://mail.python.org/mailman3/lists/typing-sig.python.org/ > Member address: gohanpra@gmail.com >
-- S Pradeep Kumar
-- --Guido van Rossum (python.org/~guido) *Pronouns: he/him **(why is my pronoun here?)* <http://feministing.com/2015/02/03/how-using-they-as-a-singular-pronoun-can-change-the-world/> _______________________________________________ Typing-sig mailing list -- typing-sig@python.org To unsubscribe send an email to typing-sig-leave@python.org https://mail.python.org/mailman3/lists/typing-sig.python.org/ Member address: donovick@cs.stanford.edu
-- --Guido van Rossum (python.org/~guido) *Pronouns: he/him **(why is my pronoun here?)* <http://feministing.com/2015/02/03/how-using-they-as-a-singular-pronoun-can-change-the-world/> _______________________________________________ Typing-sig mailing list -- typing-sig@python.org To unsubscribe send an email to typing-sig-leave@python.org https://mail.python.org/mailman3/lists/typing-sig.python.org/ Member address: zac.hatfield.dodds@gmail.com
-- S Pradeep Kumar
That's totally cool. We should just first focus on the homework that the SC gave us, which is mostly describing the proposed new syntax (adding '*' in a few places) more precisely. I hope you'll be able to enroll someone else's help in that -- let me know if you need some pointers. And while I appreciate the gesture, I don't ever want anyone to feel obligated to write a PEP. On Sun, May 30, 2021 at 10:31 AM Matthew Rahtz <mrahtz@google.com> wrote:
Sorry to be slow on weighing in on this - had a paper deadline last week.
I agree with Pradeep - let's keep Map for a follow-up PEP. Even though we have a while until Python 3.11, I want to get PEP 646 wrapped up first so as to remove the feeling of doom that comes with a large project being unfinished. As Pradeep alludes, I can imagine there could be quite a lot more discussion about whether there's a better way to do Map (particularly in light of discussions at the Typing Summit about more intuitive syntax for typing in general), and I don't think we should bottleneck PEP 646 on that discussion.
Since it's one of the things you're most excited about with variadics though, Guido, and you've helped so much on this journey so far, I'll tell you what - I'll commit to writing the PEP for Map + gathering feedback on it as the first thing I'll do after we're done with the implementation of PEP 646 + interating on it with the SC :)
On Sat, 22 May 2021 at 04:42, S Pradeep Kumar <gohanpra@gmail.com> wrote:
so you could write [Optional[T] for T in Ts] instead of Map[Optional, Ts], but many other things too, e.g. [Dict[str, T] for T in Ts] or [Tuple[T, List[T]] for T in Ts].
@Guido: We would use a generic alias for these:
``` DictStr = Dict[str, T] MyTuple = Tuple[T, List[T]]
Map[DictStr, Ts] Map[MyTuple, Ts] ```
There are other ideas we could also consider, such as inline generic aliases (Map[Tuple[T, List[T]], Ts]) and the comprehension syntax you mentioned.
Given these questions and possible syntax changes, I strongly favor keeping this as a separate, follow-up PEP. There seems to be broad consensus in typing-sig for PEP 646 as such, so I'd prefer iterating on the Map proposals separately instead of changing PEP 646 itself. We should be able to get both PEPs in for 3.11.
On Fri, May 21, 2021 at 8:18 PM Zac Hatfield Dodds < zac.hatfield.dodds@gmail.com> wrote:
In Hypothesis, we currently just erase the element types: ```python def tuples(*args: SearchStrategy) -> SearchStrategy[tuple]: ``` but for our 'union operator' `one_of()` we write out overloads for up to five arguments [1]. Maybe we should add similar overloads for tuples! In both cases PEP-646 would be a much nicer way to express this for both authors and consumers of the annotations.
I also like the comprehension-syntax, but I'm a little more cautious about the runtime representation - which is consumed by Hypothesis' `from_type()` strategy directly and via argument inference in `builds()`. If the value of `__annotations__` can't fully represent what was written, I'd appreciate a way to tell that it's a simplification so that we can 'refuse the temptation to guess' and ask the user for more information.
[1] https://github.com/HypothesisWorks/hypothesis/blob/37532c8fe168a6b4a4f287000...
On Sat, 22 May 2021 at 11:29, Guido van Rossum <guido@python.org> wrote:
IIRC one of the reasons we punted on Map[] was questions like these.
Map[Dict, Ts] would be an error.
Do you have a real use case for Zip[]?
I think we should at least explore using comprehension syntax instead of Map[] -- so you could write [Optional[T] for T in Ts] instead of Map[Optional, Ts], but many other things too, e.g. [Dict[str, T] for T in Ts] or [Tuple[T, List[T]] for T in Ts].
Note that this defines T as a local variable (there should not be a TypeVar('T') in scope, or at least it would be ignored).
A problem with comprehensions would be what they would evaluate to at runtime. Though I think that shouldn't stop us. I suspect the best we can do is either produce [] or [Tuple[T, List[T]] as the value to be put in __annotations__.
On Fri, May 21, 2021 at 4:24 PM Caleb Donovick < donovick@cs.stanford.edu> wrote:
What exactly are the proposed semantics of Map?
I was recently trying to write the type for a decorator which is the moral equivalent of the following:
def preserve_none(fn: Callable[..., T]) -> Callable[..., Optional[T]]: def wrapper(*args: Any) -> Optional[T]: if any(arg is None for arg in args): return None return fn(*args) return wrapper
Would I be able to write this as:
def preserve_none(fn: Callable[*Ts, T]) -> Callable[[Map[Optional, Ts]], Optional[T]]: def wrapper(*args: *Map[Optional, Ts]) -> Optional[T]: ...
What is the type of Map[Dict, Ts]? Is it an error?
Is there any way to write Map[Dict, Zip[Ks, Vs]]?
— Caleb
On Fri, May 21, 2021 at 12:08 PM Guido van Rossum <guido@python.org> wrote:
Given that much of my *own* excitement for variadics comes from the prospect of typing zip and gather, and that we have some time before this goes into 3.11, I'd be in favor of adding Map[] to PEP 646.
On Fri, May 21, 2021 at 11:46 AM S Pradeep Kumar <gohanpra@gmail.com> wrote:
> > Somehow my brain wants to use a loop construct > > This would use the Map operator: > > ``` > def tuples(*possibilities: *Map[Possibility, Ts]) -> > Possibility[Tuple[*Ts]]: ... > > p1: Possibility[int] > p2: Possibility[str] > p = tuples(p1, p2) > ``` > > + Ts will be bound to Tuple[int, str] > + That way, Map[Possibility, Ts] === Tuple[Possibility[int], > Possibility[str]] > + Return type becomes Possibility[Tuple[int, str]] > > > Or was it put off till a later PEP? (That would be a shame, since > this is the first variadic signature I've encountered in the wild in a long > time.) > > Yeah, this was part of earlier drafts but we deferred this to a > later PEP. There were objections since this would require checking that the > type argument to Map was a higher-order type like `Possibility` or a > generic alias. > > We would need this for functions like zip and asyncio.gather as well > (but not really for tensor typing). Let me know if it would be better to > incorporate this within PEP 646 itself instead of a follow-up PEP. > > On Fri, May 21, 2021 at 11:24 AM Guido van Rossum <guido@python.org> > wrote: > >> I came across a case where I think PEP 646 would provide relief, >> but I don't know how to spell it. >> >> Suppose we have a class Property[T]: >> ``` >> class Property(Generic[T]): >> ... >> ``` >> Now we have a function tuples that takes several Property arguments >> with different type parameters and returns a Property whose type parameter >> is a Tuple: >> ``` >> def tuples(*possibilities: Possibility[?]) -> Possibility[?]: >> ``` >> The idea is that you call >> ``` >> p1: Possibility[int] >> p2: Possibility[str] >> p = tuples(p1, p2) >> reveal_type(p) # Possibility[Tuple[int, str]] >> ``` >> Somehow I can't figure out how we're supposed to write that with >> PEP 646. The best I can come up with is >> ``` >> def tuples(*possibilities: *Possibility[*Ts]) -> >> Possibility[Tuple[*Ts]]:` >> ``` >> but the first `*Ts` feels wrong -- it looks as if Possibility takes >> multiple type parameters, but that's not the case. Somehow my brain wants >> to use a loop construct, e.g. >> ``` >> def tuples(*possibilities: *Possibility[T] for T in Ts) -> >> Possibility[Tuple[*Ts]]: >> ``` >> Does anyone recall how this is supposed to be handled? Or was it >> put off till a later PEP? (That would be a shame, since this is the first >> variadic signature I've encountered in the wild in a long time.) >> >> PS. If you really want to know: >> https://github.com/DRMacIver/minithesis/pull/9 >> >> -- >> --Guido van Rossum (python.org/~guido) >> *Pronouns: he/him **(why is my pronoun here?)* >> <http://feministing.com/2015/02/03/how-using-they-as-a-singular-pronoun-can-change-the-world/> >> _______________________________________________ >> Typing-sig mailing list -- typing-sig@python.org >> To unsubscribe send an email to typing-sig-leave@python.org >> https://mail.python.org/mailman3/lists/typing-sig.python.org/ >> Member address: gohanpra@gmail.com >> > > > -- > S Pradeep Kumar >
-- --Guido van Rossum (python.org/~guido) *Pronouns: he/him **(why is my pronoun here?)* <http://feministing.com/2015/02/03/how-using-they-as-a-singular-pronoun-can-change-the-world/> _______________________________________________ Typing-sig mailing list -- typing-sig@python.org To unsubscribe send an email to typing-sig-leave@python.org https://mail.python.org/mailman3/lists/typing-sig.python.org/ Member address: donovick@cs.stanford.edu
-- --Guido van Rossum (python.org/~guido) *Pronouns: he/him **(why is my pronoun here?)* <http://feministing.com/2015/02/03/how-using-they-as-a-singular-pronoun-can-change-the-world/> _______________________________________________ Typing-sig mailing list -- typing-sig@python.org To unsubscribe send an email to typing-sig-leave@python.org https://mail.python.org/mailman3/lists/typing-sig.python.org/ Member address: zac.hatfield.dodds@gmail.com
-- S Pradeep Kumar
-- --Guido van Rossum (python.org/~guido) *Pronouns: he/him **(why is my pronoun here?)* <http://feministing.com/2015/02/03/how-using-they-as-a-singular-pronoun-can-change-the-world/>
participants (5)
-
Caleb Donovick -
Guido van Rossum -
Matthew Rahtz -
S Pradeep Kumar -
Zac Hatfield Dodds