Readonly TypedDicts
I would like to propose adding a boolean `readonly` parameter to TypedDict, which would remove all mutate operations from the resulting type. # Specification ```python class Request(TypedDict, total=False, readonly=True): some_field: string | None request: Request = { "some_field": "foo" } request["some_field"] = None # error del request["some_field"] # error ``` I suggest making it illegal to have a subclass that does not have the same readonly value as its superclass(es), as on the one hand, it would be natural to expect the following to be equivalent: ```python # Declare some_field explicitly class MutableRequest(TypedDict, total=False): some_field: string | None # Inherit some_field from Request class MutableRequest(Request, readonly=False): pass ```python But on the other hand, by analogy with total we would expect only the fields explicitly declared on the class to be affected by the readonly flag. I'm not aware of a usecase for the latter, and the ambiguity between the two possible interpretations would probably be a source of issues. # Use case We are making increasing use of TypedDict to type hint code that uses dictionaries to hold requests to and responses from services (both client and server side). However, we run into problems whenever any constraint is loosened, e.g. passing data from a response where a field is guaranteed to a request where it is optional: ```python class Response(TypedDict): some_field: string class Request(TypedDict, total=False): some_field: string | None class Server1: def fetch_data() -> Response: ... class Server2: def accept_data(request: Request): ... def my_code(server1: Server1, server2: Server2): data: Response = server1.fetch_data() server2.accept_data(data) # error: Response and Request are incompatible ```python This last line is not type-safe, as there are mutation operations on Request that aren't valid on Response. If we could remove those mutation operations from the Request type, then it would be a valid subtype of Response. This problem seems quite common, e.g. https://github.com/python/mypy/pull/12142. I am thinking of making a PEP (Pablo Galindo has agreed to sponsor). What does everyone think? Is the proposal sound and implementable? --- Alice Bloomberg
I really like this idea and it's something I've thought about in the past for a multitude of reasons. I think however, approaching it as just a TypedDict kwarg would be a mistake. When I was thinking about this I thought it would be better if it extended the type system using a TypedMapping class which would become a superclass of TypedDict and have all of the current functionality of TypedDict without the writeable nature of it. Supporting this would allow custom mapping subclasses to be type-able. TypedMapping would be effectively syntax sugar transforming something like ```py from typing import TypedMapping from multidict import MultiDict class MyMapping(MultiDict, TypedMapping): foo: int bar: NotRequired[str] ``` into ```py class MyMapping(MultiDict): def __init__(self, foo: int, bar: str = ...) -> None: ... @overload def __getitem__(self, key: Literal["foo"]) -> int: ... @overload def __getitem__(self, key: Literal["bar"]) -> str | Never: ... @overload def get(self, key: Literal["foo"], default: Never = ...) -> int: ... @overload def get(self, key: Literal["bar"], default: T = ...) -> str | T: ... ``` (at type checking time) Adding this would (at runtime) make TypedDict as simple as ```py class TypedDict(dict[str, Any], TypedMapping): pass ``` TypedMapping would support all of the current kwargs TypedDict does and the special forms like NotRequired and Required Do you have any thoughts on this idea?
As the problem of mutable data structures as function arguments is quite common (for example, List vs Sequence), maybe the problem should be solved at a more general level? Maybe there could be a way to indicate that a function argument should not be mutated. Something like ``` def f(x: Final[dict[str, int]]) -> None: x["foo"] = "bar" # type error! ``` Though really, function arguments should be non-mutable by default; mutating them is a somewhat weird thing to do. Maybe to mark it as mutable, you could do this? ``` def f(x: Mutable[dict[str, int]]) -> None: x["foo"] = "bar" # OK ```
TypedMapping class...Supporting this would allow custom mapping subclasses to be type-able
Unfortunately while this makes a lot of sense in general, our use-case specifically needs to constrain to `dict` as part of the typing (it hits C code which only accepts dicts). Not sure how to thread that needle. If there a way to intersect two type declarations (in this case, dict and TypedMapping) instead of unioning them, this would be workable for us, but that's a larger discussion.
Maybe there could be a way to indicate that a function argument should not be mutated
We thought about something like a `Readonly` generic, but the non-mutability needs to be deeply applied to nested TypedDicts, and that just blew the whole thing out of scope tbh. We decided to keep this proposal narrower, and hopefully it will still provide groundwork for something like this in future. OTOH if a (non-deep) `Readonly` generic is preferable to a kwarg, I think it would work for us; we could "deeply" apply it explicitly.
I've thought more about James' suggestion and I agree with it with one significant change, namely about the behaviour when a class subclasses both TypedMapping and TypedDict. Here is my revised proposal. # Specification ```python class Request(TypedMapping, total=False): some_field: string | None request: Request = { "some_field": "foo" } request2: Request = {} # fine because total=False request["some_field"] = None # error del request["some_field"] # error ``` TypedMapping is a structural type that mirrors TypedDict except for: 1) instances need not be subclasses of dict; 2) no mutate methods will be generated 3) subclasses can narrow field types (consequence of 2) Rules for 3 mirror Protocols. ## Multiple inheritance and TypedDict A type that inherits from a TypedMapping subclass and from TypedDict (either directly or indirectly): 4) is the structural intersection of its parents, or invalid if no such intersection exists 5) instances must be a dict subclass 6) adds mutate methods only for fields it explicitly (re)declares ```python class A(TypedMapping): field1: int field2: str class B(A, TypedDict): field1: int b: B = { "field1": 5, "field2": "value" } b["field1"] = 6 # Fine, mutate methods added in definition of B b["field2"] = "value2" # Error, field2 mutator not declared ``` The alternative here would be: 6b) adds mutate methods for all fields declared in superclasses (unless redeclared in the body) I think this unnecessarily restricts usage though, and doesn't match how inheritance works in other cases (e.g. total=False and total=True do not affect fields not specified in the class body). --- Alice Bloomberg
Since this update seems uncontroversial, I will cross-post to python-dev at the suggestion of Pablo to widen the audience before drafting a PEP. Thanks James and Thomas for your feedback!
participants (4)
-
Alice Purcell -
Alice Purcell
-
James H-B -
Thomas Kehrenberg