Inline syntax for TypedDict
Hi everyone! ## Problem Dicts are fundamental to Python programming. And today I found myself writing this piece of code (again): ```python def process_user(user_dict: User) -> ModifiedUser: ... class User(TypedDict): ... class ModifiedUser(User): ... ``` While in reality it was as simple as: `{'user_id': int}` -> `{'user_id': int, 'external_id': int}` What really bothers me is that I have to either: 1. Create a new whole class for it 2. Create a new type variable in a form of `User = TypedDict('User', {...})` I think that it hurts the readability quite a lot, because the definition of this type might not be near with a function that uses it. Another problem is that due to the nature of `TypedDict`s there are quite a lot of them in a regular web application, because of json. The biggest problem (for me personally) is that you cannot simply nest TypedDict. So, if you have a nested dict that you want to use, like `{'user': {'email': str}}`? You have to define *two* classes. And the final touch: it looks like many TypedDict are just used in one place. They are just like throw-away definitions. And don't forget that classes cannot have all keys in their body: `return`, `@1`, etc are all not allowed. ## Proposal To solve all the problems above, I propose adding "inline" syntax for typed dicts: ```python def get_user_data(arg: {'user': {'email': str}}) ->{'email': str}: return user['user'] ``` It has great potential: 0. Grammar already allows that with no changes from our side 1. `Required` and `NotRequired` can be used to enforce the presence of keys and values 2. Dicts can be nested with ease, just like in my example. Even multiline nesting is supported: ```python
def multiline() -> { ... 'user': { ... 'email': str, ... }, ... }: ... ... ...
3. The intent is pretty clear: any user familiar with dicts would be able
to read and understand this code
4. Other languages already have this feature, TS example:
https://www.typescriptlang.org/play?#code/GYVwdgxgLglg9mABAcwKZQPogM6oE4YAmAhlMQBQ74BciA3gORV4O2OoC2xMANq4tih4YYZAF8xASjYNO3PrUHDRY+gFgAUIm2I86EHiTMA2k1wsAuprFA
5. Existing tooling continues to work, see this example:
```python
>>> import typing
>>> def get_user_data(arg: {'user': {'email': str}}) ->{'email': str}: ...
...
>>> get_user_data.__annotations__
{'arg': {'user': {'email': <class 'str'>}}, 'return': {'email': <class
'str'>}}
>>> typing.get_type_hints(get_user_data)
{'arg': {'user': {'email': <class 'str'>}}, 'return': {'email': <class
'str'>}}
And: ```python
import typing class Some: ... a: {'x': int} = {'x': 1} ... def method(self, arg: {'x': int}) -> {'x': int, 'y': int}: ... return {'x': 1, 'y': 2} ... Some.__annotations__ {'a': {'x': <class 'int'>}} Some.method.__annotations__ {'arg': {'x': <class 'int'>}, 'return': {'x': <class 'int'>, 'y': <class 'int'>}} typing.get_type_hints(Some) {'a': {'x': <class 'int'>}} typing.get_type_hints(Some.method) {'arg': {'x': <class 'int'>}, 'return': {'x': <class 'int'>, 'y': <class 'int'>}}
6. It is just a regular dict! No new things to create :)
## Extra
We can also discuss the variable syntax change from:
```python
X = TypedDict('X', {'a': int})
To: ```python X: TypeAlias = {'a': int} ``` Because again, it is simpler: no need to duplicate the name and values can be nested. ## Problems The most important problem I see is that it is just a syntax to solve a problem with an existing solution. But, I think it is a good addition overall. ## Typecheckers From what I know, it would not be too hard to support this in mypy. We would need to add new logic to generate names from context (instead of an explicit name). Nesting needs to be tested, we might have some troubles here. I cannot speak for other typecheckers. ## Feedback This is a very rough idea for now. But, what do you think? Is that a problem you face in your projects? I would be glad to hear the feedback from the Python's typing community :) Best, Nikita Sobolev
On Sat, Mar 4, 2023, at 10:17, Никита Соболев wrote:
This is a very rough idea for now. But, what do you think? Is that a problem you face in your projects?
I would be glad to hear the feedback from the Python's typing community :)
This is much needed. Me and my teammates often complain about not having inline typed dicts. Semantically, with recent enhancements Python typing has very solid structural typing support, the only reason we don't use it more is purely syntactic. The main unergonomic situation is one-offs. As you say, if you want to use a typed dict, you need to: - Pick a place in the file to put the class. - Come up with a name. both of which are not conceptually needed. And when reading the code, you need to jump to the TypedDict definition in order to see the fields, which breaks the flow. So we often end up using `dict[str, Any]` or such instead... ---- I think that the proposal can (for the sake of discussion) be broken down into 3 steps, each of which introduces new implications/discussion points: 1. Allow existing `TypedDict('DictName', {'x': int})` to be used in arbitrary type positions, e.g. def foo() -> TypedDict('FooRet', {'a': int}): return {'a': 10} which isn't allowed today. Question: why is it not allowed? I'm guessing the problem is the irregular scoping and binding of the `DictName`, so the first step would be to fix any assumptions made here. 2. Allow anonymous typed dicts, e.g. `TypedDict({'x': int})` (no first argument for the name). Question: What's the repr? What's the type `__name__`? 3. Syntax e.g. `{'x': int}` (no `TypedDict`). Question: is this really needed if 2 is allowed? Question: for projects like pydantic/cattrs that want to make the TypedDict concrete at runtime, how do they do it? Can pass directly to `TypedDict(...)` but how are nested TypedDicts handled in this case? Question: I'm guessing `total=False` support is out in favor of explicit `NotRequired`? Question: how will `typing.is_typeddict` be handled? Ran
I'm in favor of this. This would also be very useful if Python ever gets something like TypeVarDict, which would be analogous type TypeVarTuple except that it's over typed dictionaries instead of tuples: TD = TypeVarDict("TD") class Table(Generic[TD]): def __init__(self, dict_: TD): self.dict_ = dict_ t: Table[{'key': int}] = Table({"key": 4}) The type `Table[{'key': int}]` couldn't be expressed without the syntax that is proposed here.
I actually have another comment: should this new syntax be for `TypedMapping` instead of `TypedDict`? `TypedMapping` was proposed in PEP 705: https://github.com/python/peps/pull/2997 I don't know how common it is to mutate these one-off TypedDicts that this is aimed at.
Currently, type annotation expressions are limited to a subset of expression forms. Type checkers will reject annotations that include call expressions or dictionary expressions, for example. There are multiple reasons why call expressions, in particular, are not allowed in annotations. Type annotations should be both unambiguous in their meaning and fast to evaluate (not only for type checking but also for language server features like completion suggestions). Call expressions are expensive to evaluate because they must handle all the complexities of TypeVar constraint solving, callback protocol evaluations, overloads, etc. Furthermore, call expression evaluation is under-specified in the Python typing standard, so different type checkers can produce different results based on design choices. For these reasons, I think it would be a bad idea to permit call expressions within type annotations. By contrast, dictionary expressions are relatively fast to evaluate, and their meaning within a type annotation could be made unambiguous if the feature were properly and thoroughly specified in a PEP. So, I'd be opposed to options 1 and 2 in Ran's list above because they involve call expressions, but option 3 (or some variant thereof) seems reasonable to me if the open questions could be resolved. I'll note that the dictionary syntax for a TypedDict definition would work only within type annotations, not within other locations where types are used. In those locations, it would be evaluated as a regular dictionary object, not a type definition. For example, you wouldn't be able to use this in a call to `cast`, `isinstance`, as a base class for a class declaration, as bounds for a `TypeVar`, within a traditional (pre-PEP 695) type alias definition, etc. I think there's a reasonable chance the steering council would reject such a proposal on these grounds. They have stated in the past that they prefer not to create syntax forms that are valid only for type annotations. It may be worth writing a PEP and trying to make a compelling enough case for the SC. -Eric -- Eric Traut Contributor to Pyright & Pylance
El sáb, 4 mar 2023 a las 11:44, Eric Traut (<eric@traut.com>) escribió:
I'll note that the dictionary syntax for a TypedDict definition would work only within type annotations, not within other locations where types are used. In those locations, it would be evaluated as a regular dictionary object, not a type definition. For example, you wouldn't be able to use this in a call to `cast`, `isinstance`, as a base class for a class declaration, as bounds for a `TypeVar`, within a traditional (pre-PEP 695) type alias definition, etc.
As I understand the proposal, no new syntax is being proposed: we would merely allow additional syntax in types that is currently rejected by type checkers, though it is already allowed at runtime. Therefore, many of the contexts you cite could be made to work without language-level changes. For example, `cast({"a": int, "b": str}, {})` is legal syntax today, and under the proposal we're discussing it would have a sensible meaning. The exceptions are `isinstance()` and base classes. However, `isinstance()` already does not work with TypedDicts. Inheriting from an anonymous TypedDict is a potentially meaningful operation, but it's unlikely to be useful, since inheriting from a TypedDict can only give you another TypedDict.
One argument against this that has been brought up in similar historical discussions is that TypedDict was introduced to be able to type legacy code bases, but shouldn't really be encouraged for new code. Introducing specialized syntax for TypedDict would be a 180 degree turn on that policy. Given the above, and that the more modern approach is typically to use dataclasses or similar, I think it's worth questioning if this syntax isn't better reserved for ad-hoc creation of protocols instead. That would also be more inline with TypeScript, where an "object literal type" describes an object, and not a mapping/dictionary.
This proposal is pretty wanted, but I have a trivial question: the syntax you proposed conflicts with the dict literal, but how to distinguish between TypedDict type literal and existing dict literal? Or do you mean to
I have a question: treat some kind of dict as a type? original post: https://mail.python.org/archives/list/typing-sig@python.org/thread/66EF7EWXT...
The new syntax would only be used in places where type annotations are expected, so there is no need to distinguish them from normal dictionaries because these will be used in different places.
I suppose you might be wrong. The distinction between TypedDict literals and ordinary dict literals is still significant. For example, the code shown below is legal for now: ``` def a(b: {'test': int}) -> {'foo': int}: ... ``` Now, with Python 3.11, `{'test': int}` and `{'foo': int}:` are treated as a dict literal, thus `a.__annotations__` is a dict `{'b': {'test': int}, 'return': {'foo': int}}` not any other types nor syntax error. Suppose you want to introduce the TypedDict type-literal. In that case, we must admit the special grammatical case; the literal is interpreted as a type representation when the literal is used as a type annotation. If we allow the particular kind of dict as a type, the above syntactical issue is no longer problematic, but we'll face the destructive changing of the typing system.
Wouldn’t the least-controversial change be to allow anonymous TypedDicts by taking a generic type parameter which was a dict literal, e.g. TypedDict[{"user": str}] and not bare dict-literal annotations? It has the disadvantage of still being verbose, but you could easily alias TypedDict as TD, for example, for brevity. I am not sure if there are any implementation complexities that arise when making TypedDicts generic in this way. This wouldn’t have the issues with resolving calls as TypedDict(…) in an annotation position. At the same time, it would not reserve the pure dict literal annotation syntax. - Seth
On 5 Mar 2023, at 16.42, Chihiro Sakai <sakaic2003@gmail.com> wrote:
I suppose you might be wrong. The distinction between TypedDict literals and ordinary dict literals is still significant. For example, the code shown below is legal for now:
``` def a(b: {'test': int}) -> {'foo': int}: ... ```
Now, with Python 3.11, `{'test': int}` and `{'foo': int}:` are treated as a dict literal, thus `a.__annotations__` is a dict `{'b': {'test': int}, 'return': {'foo': int}}` not any other types nor syntax error. Suppose you want to introduce the TypedDict type-literal. In that case, we must admit the special grammatical case; the literal is interpreted as a type representation when the literal is used as a type annotation.
If we allow the particular kind of dict as a type, the above syntactical issue is no longer problematic, but we'll face the destructive changing of the typing system. _______________________________________________ Typing-sig mailing list -- typing-sig@python.org To unsubscribe send an email to typing-sig-leave@python.org https://mail.python.org/mailman3/lists/typing-sig.python.org/ Member address: syastrov@gmail.com
Python 3.10.9 (main, Dec 19 2022, 17:35:49) [GCC 12.2.0] on linux Type "help", "copyright", "credits" or "license" for more information.
from typing import TypedDict TD = TypedDict("TD", {"user": str}) TD.__annotations__ {'user': <class 'str'>}
On Sun, 2023-03-05 at 19:16 +0100, syastrov@gmail.com wrote:
Wouldn’t the least-controversial change be to allow anonymous TypedDicts by taking a generic type parameter which was a dict literal, e.g. TypedDict[{"user": str}] and not bare dict-literal annotations?
It has the disadvantage of still being verbose, but you could easily alias TypedDict as TD, for example, for brevity. I am not sure if there are any implementation complexities that arise when making TypedDicts generic in this way.
This wouldn’t have the issues with resolving calls as TypedDict(…) in an annotation position. At the same time, it would not reserve the pure dict literal annotation syntax.
- Seth
On 5 Mar 2023, at 16.42, Chihiro Sakai <sakaic2003@gmail.com> wrote:
I suppose you might be wrong. The distinction between TypedDict literals and ordinary dict literals is still significant. For example, the code shown below is legal for now:
``` def a(b: {'test': int}) -> {'foo': int}: ... ```
Now, with Python 3.11, `{'test': int}` and `{'foo': int}:` are treated as a dict literal, thus `a.__annotations__` is a dict `{'b': {'test': int}, 'return': {'foo': int}}` not any other types nor syntax error. Suppose you want to introduce the TypedDict type- literal. In that case, we must admit the special grammatical case; the literal is interpreted as a type representation when the literal is used as a type annotation.
If we allow the particular kind of dict as a type, the above syntactical issue is no longer problematic, but we'll face the destructive changing of the typing system. _______________________________________________ Typing-sig mailing list -- typing-sig@python.org To unsubscribe send an email to typing-sig-leave@python.org https://mail.python.org/mailman3/lists/typing-sig.python.org/ Member address: syastrov@gmail.com
Typing-sig mailing list -- typing-sig@python.org To unsubscribe send an email to typing-sig-leave@python.org https://mail.python.org/mailman3/lists/typing-sig.python.org/ Member address: pbryan@anode.ca
I suppose the `dict` may accept a specific dict as a generic parameter. ```python def foo(arg: dict[{'bar': int}]) -> dict[{'baz': str}]: ... ``` This wouldn't be illegal even for now. Type-checkers should check whether or not the dict passed as a generic type parameter is proper.
I think I now also prefer this dict[{'x': int, 'y': str}] syntax over the plain {'x': int, 'y': str} because 1. It will work better with new-style unions: dict[{'x': int, 'y': str}] | None This will work out of the box because dict[] already supports it. 2. It can also be used with Mapping: Mapping[{'x': int, 'y': str}] which would then be equivalent to the TypedMapping from PEP 705. Often you actually want Mapping for a function argument because Mapping is covariant which will also allow you to pass in subtypes without any problems. It's generally a good idea not to mutate function parameters, and using Mapping makes it explicit that the parameter won't be mutated.
In my practice I very often create type aliases even for more trivial type hints. This allows them to be more clear and reusable (fore example in tests). The second point is that typeddict is shouldn't be used to often: in most of code I prefer using dataclasses as they are more extendable and behave like real objects (e.g can contain `__post_init__` with validators and properties). And finally, inlining real typeddict with multiple fields will transform function signature into a mess. In python we do not have multiline lambdas - here is the same situation.
Python allows breaking lines in certain expressions, even in type annotations, e.g.: ```python def foo(a: dict[{ 'bar': int, 'baz': str, }]) -> dict[{ 'bar': str, 'baz': str, }]: ... ```
I like the proposed `dict[{ <typed dict definition> }]` form. It's quite elegant. I've added experimental support for this proposal in the latest version of pyright (1.1.297), which I just published. The implementation was relatively straightforward, so I don't think it would be much of a burden for other type checkers to implement. The only issue I discovered is that `typing.Dict` raises an exception if it receives only a single type argument. That means this form can be used only with `builtins.dict` and not `typing.Dict`. I think that limitation is reasonable. Type checkers can flag it as an error if someone attempts to use `typing.Dict`. I've spent some time playing with this support, and I like it even more than I had anticipated. The language server support (completion suggestions, etc.) for TypedDict types in pyright make this especially compelling. I think this proposal is worth moving forward as a PEP. It should be a relatively short one. There's a good chance the SC will approve it because it doesn't require any changes to the runtime, it doesn't introduce any new syntax, and it composes well with existing type features. In the meantime, I'll leave the experimental support in pyright. It can be treated as a reference implementation for the PEP. If a PEP is not pursued or is eventually rejected by the SC, I'll remove the experimental support at that time. -- Eric Traut Contributor to Pyright & Pylance
There are some additional runtime complications with typing.py, because there are some internal caches that rely on types being hashable, and `dict[{"a": int}]` is not hashable. This is observable in the speed of creating instances: In [9]: td = dict[{"a": 4}] In [10]: d = dict[str, int] In [11]: %timeit Iterable[td] 4.48 µs ± 53.8 ns per loop (mean ± std. dev. of 7 runs, 100,000 loops each) In [12]: %timeit Iterable[d] 126 ns ± 1.19 ns per loop (mean ± std. dev. of 7 runs, 10,000,000 loops each) Iterable[d] is about 35x slower with the newly proposed syntax than with `dict[str, int]`. But this is typing.Iterable only; collections.abc.Iterable doesn't have this cache and both variants are equally fast. El mar, 7 mar 2023 a las 18:02, Eric Traut (<eric@traut.com>) escribió:
I like the proposed `dict[{ <typed dict definition> }]` form. It's quite elegant.
I've added experimental support for this proposal in the latest version of pyright (1.1.297), which I just published. The implementation was relatively straightforward, so I don't think it would be much of a burden for other type checkers to implement.
The only issue I discovered is that `typing.Dict` raises an exception if it receives only a single type argument. That means this form can be used only with `builtins.dict` and not `typing.Dict`. I think that limitation is reasonable. Type checkers can flag it as an error if someone attempts to use `typing.Dict`.
I've spent some time playing with this support, and I like it even more than I had anticipated. The language server support (completion suggestions, etc.) for TypedDict types in pyright make this especially compelling.
I think this proposal is worth moving forward as a PEP. It should be a relatively short one. There's a good chance the SC will approve it because it doesn't require any changes to the runtime, it doesn't introduce any new syntax, and it composes well with existing type features.
In the meantime, I'll leave the experimental support in pyright. It can be treated as a reference implementation for the PEP. If a PEP is not pursued or is eventually rejected by the SC, I'll remove the experimental support at that time.
-- Eric Traut Contributor to Pyright & Pylance _______________________________________________ Typing-sig mailing list -- typing-sig@python.org To unsubscribe send an email to typing-sig-leave@python.org https://mail.python.org/mailman3/lists/typing-sig.python.org/ Member address: jelle.zijlstra@gmail.com
For the "Alternatives considered" section of the PEP, can the TypedDict["TD", {"a": str, "b": int}] option be addressed? On Wed, 2023-03-08 at 02:01 +0000, Eric Traut wrote:
I like the proposed `dict[{ <typed dict definition> }]` form. It's quite elegant.
I've added experimental support for this proposal in the latest version of pyright (1.1.297), which I just published. The implementation was relatively straightforward, so I don't think it would be much of a burden for other type checkers to implement.
The only issue I discovered is that `typing.Dict` raises an exception if it receives only a single type argument. That means this form can be used only with `builtins.dict` and not `typing.Dict`. I think that limitation is reasonable. Type checkers can flag it as an error if someone attempts to use `typing.Dict`.
I've spent some time playing with this support, and I like it even more than I had anticipated. The language server support (completion suggestions, etc.) for TypedDict types in pyright make this especially compelling.
I think this proposal is worth moving forward as a PEP. It should be a relatively short one. There's a good chance the SC will approve it because it doesn't require any changes to the runtime, it doesn't introduce any new syntax, and it composes well with existing type features.
In the meantime, I'll leave the experimental support in pyright. It can be treated as a reference implementation for the PEP. If a PEP is not pursued or is eventually rejected by the SC, I'll remove the experimental support at that time.
-- Eric Traut Contributor to Pyright & Pylance _______________________________________________ Typing-sig mailing list -- typing-sig@python.org To unsubscribe send an email to typing-sig-leave@python.org https://mail.python.org/mailman3/lists/typing-sig.python.org/ Member address: pbryan@anode.ca
Yes, this indeed looks amazing and quite simple. (Link for the context: https://github.com/microsoft/pyright/commit/8e562156619b1d5b4aa0fb9e1bb16977... ) `dict[{...}]` form also removes the mutability issue. Jelle, do you think we need a PEP for this? I think so. If I am correct, I would be happy to work on it and formalize all the details. Best, Nikita ср, 8 мар. 2023 г. в 05:02, Eric Traut <eric@traut.com>:
I like the proposed `dict[{ <typed dict definition> }]` form. It's quite elegant.
I've added experimental support for this proposal in the latest version of pyright (1.1.297), which I just published. The implementation was relatively straightforward, so I don't think it would be much of a burden for other type checkers to implement.
The only issue I discovered is that `typing.Dict` raises an exception if it receives only a single type argument. That means this form can be used only with `builtins.dict` and not `typing.Dict`. I think that limitation is reasonable. Type checkers can flag it as an error if someone attempts to use `typing.Dict`.
I've spent some time playing with this support, and I like it even more than I had anticipated. The language server support (completion suggestions, etc.) for TypedDict types in pyright make this especially compelling.
I think this proposal is worth moving forward as a PEP. It should be a relatively short one. There's a good chance the SC will approve it because it doesn't require any changes to the runtime, it doesn't introduce any new syntax, and it composes well with existing type features.
In the meantime, I'll leave the experimental support in pyright. It can be treated as a reference implementation for the PEP. If a PEP is not pursued or is eventually rejected by the SC, I'll remove the experimental support at that time.
-- Eric Traut Contributor to Pyright & Pylance _______________________________________________ Typing-sig mailing list -- typing-sig@python.org To unsubscribe send an email to typing-sig-leave@python.org https://mail.python.org/mailman3/lists/typing-sig.python.org/ Member address: n.a.sobolev@gmail.com
I suppose this suggestion is reasonably simple and offers us great benefits. Also, Pyright's experimental support for this suggestion is an excellent tentative implementation for Request for Comments. We should stand for drafting up a new PEP. Still, we have a few issues with this inline syntax for TypedDict. 1. dict is not an immutable type; thus `dict[{ <type definitions> }].__args__[0]` can be mutated. 2. dict is not a hashable type; thus `hash(dict[{ <type definitions> }])` will raise a TypeError. Maybe some libraries break with those types.
El mié, 8 mar 2023 a las 4:13, Никита Соболев (<n.a.sobolev@gmail.com>) escribió:
Yes, this indeed looks amazing and quite simple. (Link for the context: https://github.com/microsoft/pyright/commit/8e562156619b1d5b4aa0fb9e1bb16977... )
`dict[{...}]` form also removes the mutability issue.
Jelle, do you think we need a PEP for this? I think so.
Yes, this will need a PEP.
If I am correct, I would be happy to work on it and formalize all the details.
Best, Nikita
ср, 8 мар. 2023 г. в 05:02, Eric Traut <eric@traut.com>:
I like the proposed `dict[{ <typed dict definition> }]` form. It's quite elegant.
I've added experimental support for this proposal in the latest version of pyright (1.1.297), which I just published. The implementation was relatively straightforward, so I don't think it would be much of a burden for other type checkers to implement.
The only issue I discovered is that `typing.Dict` raises an exception if it receives only a single type argument. That means this form can be used only with `builtins.dict` and not `typing.Dict`. I think that limitation is reasonable. Type checkers can flag it as an error if someone attempts to use `typing.Dict`.
I've spent some time playing with this support, and I like it even more than I had anticipated. The language server support (completion suggestions, etc.) for TypedDict types in pyright make this especially compelling.
I think this proposal is worth moving forward as a PEP. It should be a relatively short one. There's a good chance the SC will approve it because it doesn't require any changes to the runtime, it doesn't introduce any new syntax, and it composes well with existing type features.
In the meantime, I'll leave the experimental support in pyright. It can be treated as a reference implementation for the PEP. If a PEP is not pursued or is eventually rejected by the SC, I'll remove the experimental support at that time.
-- Eric Traut Contributor to Pyright & Pylance _______________________________________________ Typing-sig mailing list -- typing-sig@python.org To unsubscribe send an email to typing-sig-leave@python.org https://mail.python.org/mailman3/lists/typing-sig.python.org/ Member address: n.a.sobolev@gmail.com
_______________________________________________ Typing-sig mailing list -- typing-sig@python.org To unsubscribe send an email to typing-sig-leave@python.org https://mail.python.org/mailman3/lists/typing-sig.python.org/ Member address: jelle.zijlstra@gmail.com
[...] and it composes well with existing type features.
I don't think it composes particularly well, actually. For example, what would it mean to do def f(d: dict[T]) -> T: ... f({'x': 42}) # 1 f({'x': 42}. {'x': 'foo', 'y': 24}) # 2 d: dict[str, int] f(d) # 3 Should that be allowed? If yes, what is the type bound to T in calls 1, 2 and 3? What about def f(d: dict[{'x': T]}]) -> T: ... ? Overall it feels like this change makes dict a special form, like tuple, as opposed to a normal type. I would be very careful about adding more special forms to an already quite complicated and nuanced type system. Sergei On Wed, Mar 8, 2023 at 2:02 AM Eric Traut <eric@traut.com> wrote:
I like the proposed `dict[{ <typed dict definition> }]` form. It's quite elegant.
I've added experimental support for this proposal in the latest version of pyright (1.1.297), which I just published. The implementation was relatively straightforward, so I don't think it would be much of a burden for other type checkers to implement.
The only issue I discovered is that `typing.Dict` raises an exception if it receives only a single type argument. That means this form can be used only with `builtins.dict` and not `typing.Dict`. I think that limitation is reasonable. Type checkers can flag it as an error if someone attempts to use `typing.Dict`.
I've spent some time playing with this support, and I like it even more than I had anticipated. The language server support (completion suggestions, etc.) for TypedDict types in pyright make this especially compelling.
I think this proposal is worth moving forward as a PEP. It should be a relatively short one. There's a good chance the SC will approve it because it doesn't require any changes to the runtime, it doesn't introduce any new syntax, and it composes well with existing type features.
In the meantime, I'll leave the experimental support in pyright. It can be treated as a reference implementation for the PEP. If a PEP is not pursued or is eventually rejected by the SC, I'll remove the experimental support at that time.
-- Eric Traut Contributor to Pyright & Pylance _______________________________________________ Typing-sig mailing list -- typing-sig@python.org To unsubscribe send an email to typing-sig-leave@python.org https://mail.python.org/mailman3/lists/typing-sig.python.org/ Member address: sergei.a.lebedev@gmail.com
I don't really like the idea of repurposing `dict` either. But I can think of an explanation. `dict` is a special form which can be indexed by either a pair of types or a dict literal whose keys are literal strings and whose values are types. `dict` indexed by a pair of types (or raw `dict` implicitly instantiated to `dict[Any, Any]` is a dict type. `dict` indexed by a dict literal is a typed dict type. `def f(d: dict[T]) -> T: ...` is not allowed because dict is indexed with a single type. `def f(d: dict[{'x': T}]) -> T: ...` is allowed by this rule. It's a generic function that takes a typed dict. On Thu, Mar 9, 2023 at 11:48 AM Sergei Lebedev <sergei.a.lebedev@gmail.com> wrote:
[...] and it composes well with existing type features.
I don't think it composes particularly well, actually. For example, what would it mean to do
def f(d: dict[T]) -> T: ... f({'x': 42}) # 1 f({'x': 42}. {'x': 'foo', 'y': 24}) # 2 d: dict[str, int] f(d) # 3
Should that be allowed? If yes, what is the type bound to T in calls 1, 2 and 3?
What about
def f(d: dict[{'x': T]}]) -> T: ...
?
Overall it feels like this change makes dict a special form, like tuple, as opposed to a normal type. I would be very careful about adding more special forms to an already quite complicated and nuanced type system.
Sergei
On Wed, Mar 8, 2023 at 2:02 AM Eric Traut <eric@traut.com> wrote:
I like the proposed `dict[{ <typed dict definition> }]` form. It's quite elegant.
I've added experimental support for this proposal in the latest version of pyright (1.1.297), which I just published. The implementation was relatively straightforward, so I don't think it would be much of a burden for other type checkers to implement.
The only issue I discovered is that `typing.Dict` raises an exception if it receives only a single type argument. That means this form can be used only with `builtins.dict` and not `typing.Dict`. I think that limitation is reasonable. Type checkers can flag it as an error if someone attempts to use `typing.Dict`.
I've spent some time playing with this support, and I like it even more than I had anticipated. The language server support (completion suggestions, etc.) for TypedDict types in pyright make this especially compelling.
I think this proposal is worth moving forward as a PEP. It should be a relatively short one. There's a good chance the SC will approve it because it doesn't require any changes to the runtime, it doesn't introduce any new syntax, and it composes well with existing type features.
In the meantime, I'll leave the experimental support in pyright. It can be treated as a reference implementation for the PEP. If a PEP is not pursued or is eventually rejected by the SC, I'll remove the experimental support at that time.
-- Eric Traut Contributor to Pyright & Pylance _______________________________________________ Typing-sig mailing list -- typing-sig@python.org To unsubscribe send an email to typing-sig-leave@python.org https://mail.python.org/mailman3/lists/typing-sig.python.org/ Member address: sergei.a.lebedev@gmail.com
_______________________________________________ Typing-sig mailing list -- typing-sig@python.org To unsubscribe send an email to typing-sig-leave@python.org https://mail.python.org/mailman3/lists/typing-sig.python.org/ Member address: kmillikin@google.com
For the `{“x”: 42}` syntax as well as the `dict[{“x”: 42}]` syntax, what will this be at runtime? Ideally it would be as similar as possible to a non-inline TypedDict.
Inline TypedDict syntax has been on my wish list since TypedDict was released. As stated in a few other comments having to name and define intermediary TypedDicts is lots of the time unwanted. Given the following `TypedDict`'s stripped and slimed down from a codebase at work: ```python from typing import TypedDict class Author(TypedDict): name: str email: str class AlternativeID(TypedDict): type: str id: str class Document(TypedDict): id: str authors: list[Author] alternative_ids: list[AlternativeID] ``` The `Author` and `AlternativeID` declarations are not used anywhere else in the codebase so in this case they are visually distracting and it is annoying when you have to jump to a definition much further up the page to see it (the original code contains a MUCH larger TypedDict definition. I think a nested syntax support would be great but I don't know which syntax would be the most suitable, the `dict[{}]` syntax has been suggested further up: ```python from typing import TypedDict Document = dict[ { "id": str, "authors": list[dict[{"name": str, "email": str}]], "alternative_ids": list[dict[{"type": str, "id": str}]], } ] ``` I think it looks okay to me apart from the additional nesting within the `list`, in my opinion TypeScript in comparison is easier to parse visually: ```ts type Document = { id: string; authors: Array<{ name: string; email: string }>; alternative_ids: Array<{ type: string; id: string }>; }; ``` Also being able to mix and match between inline definitions and separate classes would be a requirement so the choice is then left to the user to decide.
It already supports an inline syntax: TypedDict("TD", {"s": str, "i": int}) On Fri, 2023-06-09 at 15:10 +0000, michaeloliver__@outlook.com wrote:
Inline TypedDict syntax has been on my wish list since TypedDict was released. As stated in a few other comments having to name and define intermediary TypedDicts is lots of the time unwanted.
Given the following `TypedDict`'s stripped and slimed down from a codebase at work: ```python from typing import TypedDict
class Author(TypedDict): name: str email: str
class AlternativeID(TypedDict): type: str id: str
class Document(TypedDict): id: str authors: list[Author] alternative_ids: list[AlternativeID] ```
The `Author` and `AlternativeID` declarations are not used anywhere else in the codebase so in this case they are visually distracting and it is annoying when you have to jump to a definition much further up the page to see it (the original code contains a MUCH larger TypedDict definition.
I think a nested syntax support would be great but I don't know which syntax would be the most suitable, the `dict[{}]` syntax has been suggested further up:
```python from typing import TypedDict
Document = dict[ { "id": str, "authors": list[dict[{"name": str, "email": str}]], "alternative_ids": list[dict[{"type": str, "id": str}]], } ] ```
I think it looks okay to me apart from the additional nesting within the `list`, in my opinion TypeScript in comparison is easier to parse visually: ```ts type Document = { id: string; authors: Array<{ name: string; email: string }>; alternative_ids: Array<{ type: string; id: string }>; }; ```
Also being able to mix and match between inline definitions and separate classes would be a requirement so the choice is then left to the user to decide. _______________________________________________ Typing-sig mailing list -- typing-sig@python.org To unsubscribe send an email to typing-sig-leave@python.org https://mail.python.org/mailman3/lists/typing-sig.python.org/ Member address: pbryan@anode.ca
Sorry if I didn't make it clear, I would want to inlinenested dictionaries and inline the TypedDict in a function return type etc.
participants (14)
-
17＠itishka.org -
Adrian Garcia Badaracco -
Anton Agestam -
Chihiro Sakai -
Eric Traut -
Jelle Zijlstra -
Kevin Millikin -
michaeloliver__＠outlook.com -
Paul Bryan -
Ran Benita -
Sergei Lebedev -
syastrov＠gmail.com -
Thomas Kehrenberg -
Никита Соболев