Generic NamedTuples
At the moment, this will throw a runtime error: from typing import Generic, NamedTuple, TypeVar T = TypeVar("T") class NT(NamedTuple, Generic[T]): foo: int bar: T There is an open CPython PR to add runtime support for this pattern: https://github.com/python/cpython/pull/92027. I'm inclined to accept this change, because it seems like a natural extension and would work well with the rest of the type system. Mypy doesn't currently support this for internal reasons, but I'm told Pyre already supports it. Let me know if there's some problem I missed with this idea. We just discussed this at the typing summit at PyCon, and one concern was brought up about variance. I think variance should work the same as with non-NamedTuple generic classes and no special treatment is required. Relatedly, there is another open PR that allows generalized multiple inheritance on NamedTuple (https://github.com/python/cpython/pull/31781). This would allow code like this: class A: @property def prop(self) -> int: ... class B(NamedTuple, A): foo: int assert_type(B(1).prop, int) I think this is too much to ask static type checkers to support: regular classes are too different from namedtuples. But I'd be happy to hear other opinions, especially from type checker maintainers.
There is also a PR for generic TypedDict: https://github.com/python/cpython/pull/27663 And an associated thread on typing-sig: https://mail.python.org/archives/list/typing-sig@python.org/thread/I7P3ER2NH...
Pyright already supports generic NamedTuples, but it currently emits an error when the class declaration combines NamedTuple and Generic to warn the user that it will generate a runtime exception. It would be trivial to remove that check if the runtime no longer imposes that restriction. Supporting arbitrary multiple inheritance for NamedTuple opens up a can of worms. We'd need to work out all of the typing and runtime rules and make sure that they're all consistent. It would require quite a bit of discussion and probably a formal PEP. I agree that it's not worth it. It's better to limit it to just Generic IMO. -Eric -- Eric Traut Contributor to Pyright & Pylance Microsoft
El vie, 29 abr 2022 a las 3:58, Thomas Kehrenberg (
There is also a PR for generic TypedDict: https://github.com/python/cpython/pull/27663
And an associated thread on typing-sig: https://mail.python.org/archives/list/typing-sig@python.org/thread/I7P3ER2NH...
Thanks for bringing this up. I'd be open to merging this PR and allowing generic TypedDicts at runtime in 3.11 if we're confident that the semantics are clear enough that we don't need to go through the full standardization process. Are there any concerns about allowing generic TypedDicts at runtime?
_______________________________________________ Typing-sig mailing list -- typing-sig@python.org To unsubscribe send an email to typing-sig-leave@python.org https://mail.python.org/mailman3/lists/typing-sig.python.org/ Member address: jelle.zijlstra@gmail.com
I think it's fine and we should do this. Thanks for asking around first!
On Fri, Apr 29, 2022 at 10:37 AM Jelle Zijlstra
El vie, 29 abr 2022 a las 3:58, Thomas Kehrenberg (
) escribió: There is also a PR for generic TypedDict: https://github.com/python/cpython/pull/27663
And an associated thread on typing-sig: https://mail.python.org/archives/list/typing-sig@python.org/thread/I7P3ER2NH...
Thanks for bringing this up. I'd be open to merging this PR and allowing generic TypedDicts at runtime in 3.11 if we're confident that the semantics are clear enough that we don't need to go through the full standardization process.
Are there any concerns about allowing generic TypedDicts at runtime?
_______________________________________________ Typing-sig mailing list -- typing-sig@python.org To unsubscribe send an email to typing-sig-leave@python.org https://mail.python.org/mailman3/lists/typing-sig.python.org/ Member address: jelle.zijlstra@gmail.com
_______________________________________________ Typing-sig mailing list -- typing-sig@python.org To unsubscribe send an email to typing-sig-leave@python.org https://mail.python.org/mailman3/lists/typing-sig.python.org/ Member address: guido@python.org
-- --Guido van Rossum (python.org/~guido) *Pronouns: he/him **(why is my pronoun here?)* http://feministing.com/2015/02/03/how-using-they-as-a-singular-pronoun-can-c...
We just discussed this at the typing summit at PyCon, and one concern was brought up about variance. I think variance should work the same as with non-NamedTuple generic classes and no special treatment is required.
I agree that no special treatment is required at runtime. But I am a bit unsure about if type checker should allow NamedTuples to be generic on contravariant typevars, due to subtyping compatibility with plain tuples. For example, if this is allowed: ``` T = TypeVar("T", contravariant=True) class Foo(NamedTuple): x: T ``` Then consider the following snippet: ``` def takes_tuple(x: tuple[str]) -> None: return x[0].startswith('derp') def test(x: int | str) -> None: takes_tuple(Foo(x=x)) test(42) ``` The code would crash at runtime. But the type checker would have to accept it, because: 1) Foo[int | str] is a subtype of Foo[int], thanks to contra-variance. 2) Foo[int] is a subtype of tuple[str], due to compatibility between NamedTuple and tuple. 3) Subtyping relation ought to be transitive. As a result, you can pass an object of type `Foo[int | str]` to a function that expects tuple[str]. And this is unsound.
So let this be a warning to type checkers -- those shouldn't allow contravariant type variables for NamedTuple. Are there similar issues with other generic classes? (Maybe TypedDict?) On Fri, Apr 29, 2022 at 11:14 AM Jia Chen via Typing-sig < typing-sig@python.org> wrote:
We just discussed this at the typing summit at PyCon, and one concern was brought up about variance. I think variance should work the same as with non-NamedTuple generic classes and no special treatment is required.
I agree that no special treatment is required at runtime. But I am a bit unsure about if type checker should allow NamedTuples to be generic on contravariant typevars, due to subtyping compatibility with plain tuples. For example, if this is allowed:
``` T = TypeVar("T", contravariant=True) class Foo(NamedTuple): x: T ```
Then consider the following snippet:
``` def takes_tuple(x: tuple[str]) -> None: return x[0].startswith('derp')
def test(x: int | str) -> None: takes_tuple(Foo(x=x))
test(42) ```
The code would crash at runtime. But the type checker would have to accept it, because: 1) Foo[int | str] is a subtype of Foo[int], thanks to contra-variance. 2) Foo[int] is a subtype of tuple[str], due to compatibility between NamedTuple and tuple. 3) Subtyping relation ought to be transitive.
As a result, you can pass an object of type `Foo[int | str]` to a function that expects tuple[str]. And this is unsound. _______________________________________________ Typing-sig mailing list -- typing-sig@python.org To unsubscribe send an email to typing-sig-leave@python.org https://mail.python.org/mailman3/lists/typing-sig.python.org/ Member address: guido@python.org
-- --Guido van Rossum (python.org/~guido) *Pronouns: he/him **(why is my pronoun here?)* http://feministing.com/2015/02/03/how-using-they-as-a-singular-pronoun-can-c...
So let this be a warning to type checkers -- those shouldn't allow contravariant type variables for NamedTuple.
Agreed.
Are there similar issues with other generic classes? (Maybe TypedDict?)
I think you are right. TypedDict should behave neither covariantly nor contravariantly due to mutability, if they were to be made generic. - Jia PS: I just found out I have some typos in the reasoning earlier which makes them confusing. This is why I should never send an email while listening to a talk at the same time :( The corrected lines of reasoning are: 1) Foo[int | str] is a subtype of Foo[str], thanks to contra-variance. 2) Foo[str] is compatible with tuple[str], due to compatibility between NamedTuple and tuple. 3) Compatibility relation ought to be transitive.
Isn't that just generally a problem for any attributes (and method return types) that have covariant generic types? That is, it's not specific to tuples and NamedTuple. On Fri, Apr 29, 2022 at 11:51 AM Jia Chen via Typing-sig < typing-sig@python.org> wrote:
So let this be a warning to type checkers -- those shouldn't allow contravariant type variables for NamedTuple.
Agreed.
Are there similar issues with other generic classes? (Maybe TypedDict?)
I think you are right. TypedDict should behave neither covariantly nor contravariantly due to mutability, if they were to be made generic.
- Jia
PS: I just found out I have some typos in the reasoning earlier which makes them confusing. This is why I should never send an email while listening to a talk at the same time :( The corrected lines of reasoning are:
1) Foo[int | str] is a subtype of Foo[str], thanks to contra-variance. 2) Foo[str] is compatible with tuple[str], due to compatibility between NamedTuple and tuple. 3) Compatibility relation ought to be transitive. _______________________________________________ Typing-sig mailing list -- typing-sig@python.org To unsubscribe send an email to typing-sig-leave@python.org https://mail.python.org/mailman3/lists/typing-sig.python.org/ Member address: kmillikin@google.com
Isn't that just generally a problem for any attributes (and method return types) that have covariant generic types? That is, it's not specific to tuples and NamedTuple.
I think it is. What's special about `NamedTuple`/`TypedDict` here is the fact that their covariance/invariance is not explicitly defined anywhere in code. That knowledge is instead hard-baked into the type checkers.
Update: Both generic NamedTuples and generic TypedDicts will be allowed at
runtime in Python 3.11.
I opened https://github.com/python/typing/issues/1174 to track backporting
this behavior for older Python versions.
El jue, 28 abr 2022 a las 14:55, Jelle Zijlstra (
At the moment, this will throw a runtime error:
from typing import Generic, NamedTuple, TypeVar T = TypeVar("T")
class NT(NamedTuple, Generic[T]): foo: int bar: T
There is an open CPython PR to add runtime support for this pattern: https://github.com/python/cpython/pull/92027. I'm inclined to accept this change, because it seems like a natural extension and would work well with the rest of the type system. Mypy doesn't currently support this for internal reasons, but I'm told Pyre already supports it.
Let me know if there's some problem I missed with this idea. We just discussed this at the typing summit at PyCon, and one concern was brought up about variance. I think variance should work the same as with non-NamedTuple generic classes and no special treatment is required.
Relatedly, there is another open PR that allows generalized multiple inheritance on NamedTuple (https://github.com/python/cpython/pull/31781). This would allow code like this:
class A: @property def prop(self) -> int: ...
class B(NamedTuple, A): foo: int
assert_type(B(1).prop, int)
I think this is too much to ask static type checkers to support: regular classes are too different from namedtuples. But I'd be happy to hear other opinions, especially from type checker maintainers.
Pyright 1.1.245 now contains full support for generic TypedDicts if you want to play around with the functionality. -- Eric Traut Contributor to Pyright & Pylance Microsoft
participants (6)
-
Eric Traut
-
Guido van Rossum
-
Jelle Zijlstra
-
Jia Chen
-
Kevin Millikin
-
Thomas Kehrenberg