Partial type (similar to typescript)

Hi folks! I just wanted to get feelings on having something like Partial in TypeScript. The way it works is that it makes every key inside a TS type/interface optional. Here's an example of how it works: https://www.typescriptlang.org/play#code/JYOwLgpgTgZghgYwgAgCoHsAm7kG8BQyyYw... For a library I work on we might have an use case that could use a way to make a partial class from another class, for example: ``` @dataclass class Todo: title: str description: str def update_todo(todo: Todo, fields_to_update: Partial[Todo]) -> Todo: ... todo1 = Todo(title="organize desk", description="clear clutter"); update_todo(todo1, Partial[Todo](description="Something")) ``` I purposely didn't do the example using typed dicts, mostly because I was curious to see if we can find a nice API that can work for classes too. And my example is a test 馃槉 What do you think? Do you see us having a Partial type that *also* works with classes?

A few thoughts... First, TypeScript's `Partial` leverages the well-defined semantics for what it means for a field to be "optional". This doesn't mean the same as `Optional` in Python's type system (which is a union with `None`). TypeScript's "optional" is akin to `NotRequired` in the Python type system. The only place where `NotRequired` is supported today in Python is with `TypedDict`. I can see how a `Partial` could be applied to a `TypedDict` class. It would effectively make all `Required` fields `NotRequired`. I don't see how `Partial` could be applied to any other Python type, including a `dataclass` like in your example above. A `dataclass` doesn't have the notion of a `NotRequired` field. All fields defined in a dataclass are always required, so I don't know what it would mean to have a "partial" dataclass type. The same is true for other classes. I suppose it would be possible to create a new variant of `dataclass` that allows for `NotRequired` fields, but that would require pretty significant behavioral changes to dataclass. Second, TypeScript types are erased at runtime. By contrast, Python types exist at runtime, and it's possible for runtime type checkers to use introspection to understand their internals. `Partial[Todo]` wouldn't work like you have shown it in your example above. Instead, `Partial` would need to be a function that creates a new runtime type. Something like: class Todo(TypedDict): title: str description: str PartialTodo = Partial("PartialTodo", Todo) So I think `Partial` could work in Python, but its use would be limited to TypedDict, and it would effectively be a transform that creates a new TypedDict class from an existing TypedDict class. That's probably much more limited than what you had in mind. -Eric -- Eric Traut Contributor to Pyright & Pylance Microsoft

This is an interesting idea, it's been requested a fair bit on pydantic, see this issue <https://github.com/pydantic/pydantic/issues/1673> for example, it even seems that someone has created a pydantic-partial package to accomplish this, though I haven't looked at it. My main reasons for refusing this until now are: - It would have been hard to do with the pydantic V1 codebase; however it would be very easy with pydantic-core/pydantic V2 - There's no way of expressing this with types, thus any implementation would not be type safe - clearly if a Partial type was added to typing, this issue would be solved Eric brings up two issues, one I think has an obviously solution, the other is more new nuanced: *1. What does `Partial[Todo]` do at runtime?* I think this is reasonably easy to solve, `Partial` could call a new dunder classmethod on the class, e.g. `__partial__`, then the class is responsible for the logic of creating a new partial variant of itself. Obviously the standard library would need to take care of this for typeddict, dataclass, namedtuple, but it would leave other libraries (like pydantic) to implement it themselves. *1. What does `Partial[Todo]` mean?* There are two parts to this: - what does `Partial[Todo]` mean when instantiating `PartialTodo`? - that seems pretty simple, all arguments become optional - what does `Partial[Todo]` mean when accessing `partial_todo.title`? - here there are two obvious options: - all attributes which were omitted when instantiating an instance get a default value - `None` would be the most obvious option, but I guess we could allow another default - all attributes which were omitted when instantiating an instance are considered to not exist and raise an AttributeError if you try to access them - this seems more radical, but I think it's what most people want they talk about partial In both these questions, there's a question about what to do with arguments/attributes which already have a default value or default factory: they could either get their default value from the original definition of Todo, or get the general default / be missing. In the end, provided the documentation & reference implementations are clear on the meaning, I think both could work. Overall, I'm +1 for Partial. Samuel -- Samuel Colvin On Thu, 27 Oct 2022 at 14:49, Eric Traut <eric@traut.com> wrote:

Thanks all for a great discussion! Why would we give special status to dataclasses? The whole point of those is that they're just ordinary classes. If we can have Partial on a class (which does sound attractive) it should be supported for any "ordinary" class (protocols may be a different situation). If this means we have to support NotRequired[] on attribute declaration is classes, that sounds like a decent idea -- in practice, at runtime, many attributes are already "not required" (basically anything that doesn't have a default set in the class may be omitted, though in general we don't talk about that). On the issue of what the default should be, I feel that for the key use case, `Partial[C]` should give _all_ attributes of C the NotRequired status, even the ones that have a default in C. Because I imagine that the use case would be something like this (extending the OP's example): def update_todo(todo: Todo, fields_to_update: Partial[Todo]) -> Todo: for k, v in fields_to_update.__dict__.items(): setattr(todo, k, v) I wouldn't want this to replace fields that were set in `todo` but not in `fields_to_update` with their default value. And it should work the same way if I somehow wrote it out using hasattr() -- maybe for some fields I had additional processing I'd like to do: if hasattr(fields_to_update, "title"): todo.title = fields_to_update.title if hasattr(...etc...) FWIW such a Partial type would also make for a nice type related to PEP 692: def update_by_kw(todo: Todo, **kwargs: **Partial[Todo]) -> Todo: ... This could neatly describe the signature of methods like <code_object>.replace(). Finally. This would definitely require a PEP (obviously) and it would be a pretty "heavy" PEP -- the design space is pretty large (e.g. the question of all classes vs. dataclasses vs. TypedDict only). We already have several PEPs in flight, especially PEP 695. Maybe we should focus on getting that (and PEP 692) accepted first? This community doesn't have _that_ much bandwidth (I know I don't, alas). --Guido -- --Guido van Rossum (python.org/~guido) *Pronouns: he/him **(why is my pronoun here?)* <http://feministing.com/2015/02/03/how-using-they-as-a-singular-pronoun-can-c...>

Hi all, I'm glad there's interest in this! On Thu, 27 Oct 2022 at 09:51, Guido van Rossum <guido@python.org> wrote:
I agree on this, I was using dataclasses mostly because they'd potentially require some additional behaviour to support this (the constructor would need to change I guess). On the issue of what the default should be, I feel that for the key use
Yes, exactly this. The use case is mostly, I have something I want to update, but I want to be able to do partial updates, I guess it is the same for Pydantic.
This would work well for typedicts, but not for classes, right? This could neatly describe the signature of methods like
+1 on this, there's also some work on a PEP for Intersection that's going on already, and it might also help with some decision for a potential PEP for Partial 馃槉
This community doesn't have _that_ much bandwidth (I know I don't, alas).
I'm personally fine to leave this to a future time (maybe 3.13?), I'd love to work on it, but also don't have a lot of time (and experience with writing PEPs yet) Also what @Jelle Zijlstra <jelle.zijlstra@gmail.com> mentioned is quite interesting, being able to express Partial like it's done in TS would be pretty nice and it will probably allow us for other features (like Pick that allows to get a "subset" of a type), but again, it's the same story as this PEP, it's "easy" for typed dicts, difficult for classes (or, at least, it needs more decisions) -- Patrick Arminio

On Thu, 27 Oct 2022 at 11:36, Eric V. Smith <eric@trueblade.com> wrote:
I'm thinking that if I do something like this: ``` @dataclass class ABC: name: str age: int UpdateABC = Partial[ABC] ``` you should be able to call UpdateAbc with only one argument, no? I guess this could be implemented by dataclasess (or other libraries/usercode), like this: ``` def partial(cls: T) -> Partial[T]: return _make_partial(cls) ``` but Partial[T] would still need to make __init__ fields non required -- Patrick Arminio

An interesting aspect of TypeScript's Partial is that it is not actually a language feature, but a library feature: type Partial<T> = { [P in keyof T]?: T[P]; }; TypeScript's type system essentially provides a way to build a type that maps over the attributes of another type. Introducing a mechanism like that in Python would be very ambitious, but it could unlock other use cases. El mi茅, 26 oct 2022 a las 13:33, Patrick Arminio (<patrick.arminio@gmail.com>) escribi贸:

I don't think this solves the original request completely but I think it would be nice to generally support `NotRequired` on classes. Like Guido said at runtime it's quite common to have "maybe missing" attributes, it would be nice if that was integrated into the type system. Apologies if I'm missing something but I think this looks pretty compelling: ``` from typing import TypedDict, NotRequired, reveal_type class Movie(TypedDict): title: str year: NotRequired[int] def get_movie_year(movie: Movie) -> int: if "year" in movie: reveal_type(movie["year"]) # int return movie["year"] movie["year"] # typechecker error raise LookupError("Movie has no year!") class MovieClass: title: str year: NotRequired[int] def movie_class_to_string(movie: MovieClass) -> int: if hasattr(movie, "year"): # is Any (mypy) or NotRequired[int] (pyright) # should be an int reveal_type(movie.year) return movie.year # not a typechecker error movie.year # should be a typechecker error raise LookupError("Movie has no year!") ``` On Thu, Oct 27, 2022 at 11:59 AM Jelle Zijlstra <jelle.zijlstra@gmail.com> wrote:

What would be the __init__ semantics of a field in a dataclass annotated with NotRequired? 1. You implicitly default it to None? 2. You implicitly default it to some sentinel value a la PEP 661? 3. You only expect it as a **kwarg and ignore its absence? 4. ??? On Thu, 2022-10-27 at 13:24 -0500, Adrian Garcia Badaracco wrote:

A few thoughts... First, TypeScript's `Partial` leverages the well-defined semantics for what it means for a field to be "optional". This doesn't mean the same as `Optional` in Python's type system (which is a union with `None`). TypeScript's "optional" is akin to `NotRequired` in the Python type system. The only place where `NotRequired` is supported today in Python is with `TypedDict`. I can see how a `Partial` could be applied to a `TypedDict` class. It would effectively make all `Required` fields `NotRequired`. I don't see how `Partial` could be applied to any other Python type, including a `dataclass` like in your example above. A `dataclass` doesn't have the notion of a `NotRequired` field. All fields defined in a dataclass are always required, so I don't know what it would mean to have a "partial" dataclass type. The same is true for other classes. I suppose it would be possible to create a new variant of `dataclass` that allows for `NotRequired` fields, but that would require pretty significant behavioral changes to dataclass. Second, TypeScript types are erased at runtime. By contrast, Python types exist at runtime, and it's possible for runtime type checkers to use introspection to understand their internals. `Partial[Todo]` wouldn't work like you have shown it in your example above. Instead, `Partial` would need to be a function that creates a new runtime type. Something like: class Todo(TypedDict): title: str description: str PartialTodo = Partial("PartialTodo", Todo) So I think `Partial` could work in Python, but its use would be limited to TypedDict, and it would effectively be a transform that creates a new TypedDict class from an existing TypedDict class. That's probably much more limited than what you had in mind. -Eric -- Eric Traut Contributor to Pyright & Pylance Microsoft

This is an interesting idea, it's been requested a fair bit on pydantic, see this issue <https://github.com/pydantic/pydantic/issues/1673> for example, it even seems that someone has created a pydantic-partial package to accomplish this, though I haven't looked at it. My main reasons for refusing this until now are: - It would have been hard to do with the pydantic V1 codebase; however it would be very easy with pydantic-core/pydantic V2 - There's no way of expressing this with types, thus any implementation would not be type safe - clearly if a Partial type was added to typing, this issue would be solved Eric brings up two issues, one I think has an obviously solution, the other is more new nuanced: *1. What does `Partial[Todo]` do at runtime?* I think this is reasonably easy to solve, `Partial` could call a new dunder classmethod on the class, e.g. `__partial__`, then the class is responsible for the logic of creating a new partial variant of itself. Obviously the standard library would need to take care of this for typeddict, dataclass, namedtuple, but it would leave other libraries (like pydantic) to implement it themselves. *1. What does `Partial[Todo]` mean?* There are two parts to this: - what does `Partial[Todo]` mean when instantiating `PartialTodo`? - that seems pretty simple, all arguments become optional - what does `Partial[Todo]` mean when accessing `partial_todo.title`? - here there are two obvious options: - all attributes which were omitted when instantiating an instance get a default value - `None` would be the most obvious option, but I guess we could allow another default - all attributes which were omitted when instantiating an instance are considered to not exist and raise an AttributeError if you try to access them - this seems more radical, but I think it's what most people want they talk about partial In both these questions, there's a question about what to do with arguments/attributes which already have a default value or default factory: they could either get their default value from the original definition of Todo, or get the general default / be missing. In the end, provided the documentation & reference implementations are clear on the meaning, I think both could work. Overall, I'm +1 for Partial. Samuel -- Samuel Colvin On Thu, 27 Oct 2022 at 14:49, Eric Traut <eric@traut.com> wrote:

Thanks all for a great discussion! Why would we give special status to dataclasses? The whole point of those is that they're just ordinary classes. If we can have Partial on a class (which does sound attractive) it should be supported for any "ordinary" class (protocols may be a different situation). If this means we have to support NotRequired[] on attribute declaration is classes, that sounds like a decent idea -- in practice, at runtime, many attributes are already "not required" (basically anything that doesn't have a default set in the class may be omitted, though in general we don't talk about that). On the issue of what the default should be, I feel that for the key use case, `Partial[C]` should give _all_ attributes of C the NotRequired status, even the ones that have a default in C. Because I imagine that the use case would be something like this (extending the OP's example): def update_todo(todo: Todo, fields_to_update: Partial[Todo]) -> Todo: for k, v in fields_to_update.__dict__.items(): setattr(todo, k, v) I wouldn't want this to replace fields that were set in `todo` but not in `fields_to_update` with their default value. And it should work the same way if I somehow wrote it out using hasattr() -- maybe for some fields I had additional processing I'd like to do: if hasattr(fields_to_update, "title"): todo.title = fields_to_update.title if hasattr(...etc...) FWIW such a Partial type would also make for a nice type related to PEP 692: def update_by_kw(todo: Todo, **kwargs: **Partial[Todo]) -> Todo: ... This could neatly describe the signature of methods like <code_object>.replace(). Finally. This would definitely require a PEP (obviously) and it would be a pretty "heavy" PEP -- the design space is pretty large (e.g. the question of all classes vs. dataclasses vs. TypedDict only). We already have several PEPs in flight, especially PEP 695. Maybe we should focus on getting that (and PEP 692) accepted first? This community doesn't have _that_ much bandwidth (I know I don't, alas). --Guido -- --Guido van Rossum (python.org/~guido) *Pronouns: he/him **(why is my pronoun here?)* <http://feministing.com/2015/02/03/how-using-they-as-a-singular-pronoun-can-c...>

Hi all, I'm glad there's interest in this! On Thu, 27 Oct 2022 at 09:51, Guido van Rossum <guido@python.org> wrote:
I agree on this, I was using dataclasses mostly because they'd potentially require some additional behaviour to support this (the constructor would need to change I guess). On the issue of what the default should be, I feel that for the key use
Yes, exactly this. The use case is mostly, I have something I want to update, but I want to be able to do partial updates, I guess it is the same for Pydantic.
This would work well for typedicts, but not for classes, right? This could neatly describe the signature of methods like
+1 on this, there's also some work on a PEP for Intersection that's going on already, and it might also help with some decision for a potential PEP for Partial 馃槉
This community doesn't have _that_ much bandwidth (I know I don't, alas).
I'm personally fine to leave this to a future time (maybe 3.13?), I'd love to work on it, but also don't have a lot of time (and experience with writing PEPs yet) Also what @Jelle Zijlstra <jelle.zijlstra@gmail.com> mentioned is quite interesting, being able to express Partial like it's done in TS would be pretty nice and it will probably allow us for other features (like Pick that allows to get a "subset" of a type), but again, it's the same story as this PEP, it's "easy" for typed dicts, difficult for classes (or, at least, it needs more decisions) -- Patrick Arminio

On Thu, 27 Oct 2022 at 11:36, Eric V. Smith <eric@trueblade.com> wrote:
I'm thinking that if I do something like this: ``` @dataclass class ABC: name: str age: int UpdateABC = Partial[ABC] ``` you should be able to call UpdateAbc with only one argument, no? I guess this could be implemented by dataclasess (or other libraries/usercode), like this: ``` def partial(cls: T) -> Partial[T]: return _make_partial(cls) ``` but Partial[T] would still need to make __init__ fields non required -- Patrick Arminio

An interesting aspect of TypeScript's Partial is that it is not actually a language feature, but a library feature: type Partial<T> = { [P in keyof T]?: T[P]; }; TypeScript's type system essentially provides a way to build a type that maps over the attributes of another type. Introducing a mechanism like that in Python would be very ambitious, but it could unlock other use cases. El mi茅, 26 oct 2022 a las 13:33, Patrick Arminio (<patrick.arminio@gmail.com>) escribi贸:

I don't think this solves the original request completely but I think it would be nice to generally support `NotRequired` on classes. Like Guido said at runtime it's quite common to have "maybe missing" attributes, it would be nice if that was integrated into the type system. Apologies if I'm missing something but I think this looks pretty compelling: ``` from typing import TypedDict, NotRequired, reveal_type class Movie(TypedDict): title: str year: NotRequired[int] def get_movie_year(movie: Movie) -> int: if "year" in movie: reveal_type(movie["year"]) # int return movie["year"] movie["year"] # typechecker error raise LookupError("Movie has no year!") class MovieClass: title: str year: NotRequired[int] def movie_class_to_string(movie: MovieClass) -> int: if hasattr(movie, "year"): # is Any (mypy) or NotRequired[int] (pyright) # should be an int reveal_type(movie.year) return movie.year # not a typechecker error movie.year # should be a typechecker error raise LookupError("Movie has no year!") ``` On Thu, Oct 27, 2022 at 11:59 AM Jelle Zijlstra <jelle.zijlstra@gmail.com> wrote:

What would be the __init__ semantics of a field in a dataclass annotated with NotRequired? 1. You implicitly default it to None? 2. You implicitly default it to some sentinel value a la PEP 661? 3. You only expect it as a **kwarg and ignore its absence? 4. ??? On Thu, 2022-10-27 at 13:24 -0500, Adrian Garcia Badaracco wrote:
participants (8)
-
Adrian Garcia Badaracco
-
Eric Traut
-
Eric V. Smith
-
Guido van Rossum
-
Jelle Zijlstra
-
Patrick Arminio
-
Paul Bryan
-
Samuel Colvin