Enabling support for generic TypedDicts
Hi all, I wanted to bring into your attention about enabling support for generic TypedDicts. I am coming in from the conversation at: [WIP] Adding support for generic TypedDicts (https://github.com/microsoft/pyright/pull/1390) The TypedDict PEP 589 does not necessarily talk about generics. And the runtime TypedDict implementation does not support it either. Traceback (most recent call last): File "testing.py", line 4, in <module> class TD(Generic[T], TypedDict): File "/usr/lib/python3.9/typing.py", line 1894, in __new__ raise TypeError('cannot inherit from both a TypedDict type ' TypeError: cannot inherit from both a TypedDict type and a non-TypedDict base class Here I would like to discuss here if the implementation is done this way for a good reason (may be due to structural subtyping complications) and if there might still be a way to move forward and add generic support for TypedDicts. As you can see below there is a natural syntax for it (in both ways) and have backward compatibility. from typing import TypedDict, TypeVar, Generic, ListT = TypeVar("T") class TD(Generic[T], TypedDict): f1: List[T] R = TypeVar("R") FileCachedData = TypedDict('FileCachedData', {'mtime': float, 'size': int, 'value': R}) def test_td(aa: TD[T], bb: TD[T]): return aa["f1"], bb["f1"] td1 = test_td({"f1": ["foo"]}, {"f1": ["bar"]})reveal_type(td1) # info: Type of "td1" is "tuple[List[str], List[str]]" test_td({"f1": ["foo"]}, {"f1": [1]}) # error class TD2(TD[T]): f2: T td21: TD2[str] = { 'f1':["far"], 'f2':"boo" }reveal_type(td21["f1"]) # info: Type of "td21["f1"]" is "List[str]" td22 = TD2( f1=["far"], f2="boo" )reveal_type(td22) # info: Type of "td22" is "TD2[str]" td23: TD2[str] = { 'f1':[0], 'f2':"boo" } # error Thank you, Samodya
On 1/19/21 8:40 PM, Samodya Abey wrote:
Hi all,
I wanted to bring into your attention about enabling support for generic TypedDicts.
I am coming in from the conversation at: [WIP] Adding support for generic TypedDicts (https://github.com/microsoft/pyright/pull/1390)
The TypedDict PEP 589 does not necessarily talk about generics. And the runtime TypedDict implementation does not support it either.
|Traceback (most recent call last): | |File "testing.py", line 4, in <module> class TD(Generic[T], TypedDict): File "/usr/lib/python3.9/typing.py", line 1894, in __new__ raise TypeError('cannot inherit from both a TypedDict type ' TypeError: cannot inherit from both a TypedDict type and a non-TypedDict base class|
Here I would like to discuss here if the implementation is done this way for a good reason (may be due to structural subtyping complications) and if there might still be a way to move forward and add generic support for TypedDicts.
The original implementation of TypedDict didn't support Generic type parameters for simplicity of implementation and design. Your example code below makes sense and I don't see any particular reason why we couldn't add support for it. It just requires various specification & implementation work. Since you seem keen, and might be interested in doing that work, here's a sketch of what I think you'd need to do: * Draft a PEP that explains the changes you're making at the syntax and semantic levels. And of course describe the motivation for making the extension in the first place. :) * Post the PEP for review here (to typing-sig@). Shepherd to soft-approval, to unblock prototype implementation work. * Implement a full prototype that works end-to-end in at least one type checker: o Extend the TypedDict type in the typing_extensions module to enable the runtime support that you note is lacking. o Implement support in at least one type checker. Looks like you've already done most of the work on this point with pyright. * Advocate for the PEP plus prototype to be approved. -- David Foster | Seattle, WA, USA Contributor to TypedDict support for mypy
As you can see below there is a natural syntax for it (in both ways) and have backward compatibility. from typing import TypedDict,TypeVar,Generic,List T = TypeVar("T")
class TD(Generic[T],TypedDict): f1:List[T]
|R = TypeVar("R") FileCachedData = TypedDict('FileCachedData', {'mtime': float, 'size': int, 'value': R})|
def test_td(aa:TD[T],bb:TD[T]): return aa["f1"],bb["f1"]
td1 = test_td({"f1": ["foo"]}, {"f1": ["bar"]}) reveal_type(td1)# info: Type of "td1" is "tuple[List[str], List[str]]"
test_td({"f1": ["foo"]}, {"f1": [1]})# error
class TD2(TD[T]): f2:T
td21:TD2[str]= { 'f1':["far"], 'f2':"boo" } reveal_type(td21["f1"])# info: Type of "td21["f1"]" is "List[str]"
td22 = TD2( f1=["far"], f2="boo" ) reveal_type(td22)# info: Type of "td22" is "TD2[str]"
td23:TD2[str]= { 'f1':[0], 'f2':"boo" }# error Thank you, Samodya
Sounds good. Thank you for the pointers. This is new territory for me but it is exciting. I must confess that I first didn't think (somewhat naively) that this change warrants a PEP. But now I can understand the need for careful consensus when adding a new capability with a huge ecosystem like Python. Just to write down a few notes before I go off to work on a draft, - I'm expecting this will be a mishmash of ideas from different concepts that we already have - Class based syntax already has Generic support, but the assignment syntax may need to be more explored. It will be similar to how we declare Generic aliases (Guido had mentioned in a linked Github issue). - Type consistency especially regarding structural subtyping will need to be carefully explored. I assume this will be similar to how we do Protocols with Generics. - After some investigation I noticed run time support for TypedDict before PEP 560 will need some metaclass hacks due to metaclass conflicts. But after version 3.7 with PEP 560 it seems simpler. (If any body is interested I will try to track my work here: https://github.com/sransara/python-generic-typeddict) Thank you, Samodya On Sun, Jan 24, 2021 at 3:41 PM David Foster <davidfstr@gmail.com> wrote:
On 1/19/21 8:40 PM, Samodya Abey wrote:
Hi all,
I wanted to bring into your attention about enabling support for generic TypedDicts.
I am coming in from the conversation at: [WIP] Adding support for generic TypedDicts (https://github.com/microsoft/pyright/pull/1390)
The TypedDict PEP 589 does not necessarily talk about generics. And the runtime TypedDict implementation does not support it either.
Traceback (most recent call last):
File "testing.py", line 4, in <module> class TD(Generic[T], TypedDict): File "/usr/lib/python3.9/typing.py", line 1894, in __new__ raise TypeError('cannot inherit from both a TypedDict type ' TypeError: cannot inherit from both a TypedDict type and a non-TypedDict base class
Here I would like to discuss here if the implementation is done this way for a good reason (may be due to structural subtyping complications) and if there might still be a way to move forward and add generic support for TypedDicts.
The original implementation of TypedDict didn't support Generic type parameters for simplicity of implementation and design.
Your example code below makes sense and I don't see any particular reason why we couldn't add support for it. It just requires various specification & implementation work.
Since you seem keen, and might be interested in doing that work, here's a sketch of what I think you'd need to do:
Draft a PEP that explains the changes you're making at the syntax and semantic levels. And of course describe the motivation for making the extension in the first place. :) Post the PEP for review here (to typing-sig@). Shepherd to soft-approval, to unblock prototype implementation work. Implement a full prototype that works end-to-end in at least one type checker:
Extend the TypedDict type in the typing_extensions module to enable the runtime support that you note is lacking. Implement support in at least one type checker. Looks like you've already done most of the work on this point with pyright.
Advocate for the PEP plus prototype to be approved.
-- David Foster | Seattle, WA, USA Contributor to TypedDict support for mypy
As you can see below there is a natural syntax for it (in both ways) and have backward compatibility.
from typing import TypedDict, TypeVar, Generic, List T = TypeVar("T")
class TD(Generic[T], TypedDict): f1: List[T]
R = TypeVar("R") FileCachedData = TypedDict('FileCachedData', {'mtime': float, 'size': int, 'value': R})
def test_td(aa: TD[T], bb: TD[T]): return aa["f1"], bb["f1"]
td1 = test_td({"f1": ["foo"]}, {"f1": ["bar"]}) reveal_type(td1) # info: Type of "td1" is "tuple[List[str], List[str]]"
test_td({"f1": ["foo"]}, {"f1": [1]}) # error
class TD2(TD[T]): f2: T
td21: TD2[str] = { 'f1':["far"], 'f2':"boo" } reveal_type(td21["f1"]) # info: Type of "td21["f1"]" is "List[str]"
td22 = TD2( f1=["far"], f2="boo" ) reveal_type(td22) # info: Type of "td22" is "TD2[str]"
td23: TD2[str] = { 'f1':[0], 'f2':"boo" } # error
Thank you,
Samodya
participants (2)
-
David Foster
-
Samodya Abey