Accessing non-required fields of a TypedDict
A Pyright user just posted a feature request [here](https://github.com/microsoft/pyright/issues/1584). He's suggesting that Pyright (and type checkers in general) should flag as an error a subscripted access to a non-required field within a TypedDict. Here's an example (using the new PEP 655 notation for simplicity). ```python from typing import TypedDict class MyDict(TypedDict): fieldA: str fieldB: NotRequired[int] def func(x: MyDict): a = x['fieldA'] # OK b = x['fieldB'] # Should generate an error b = x.get('fieldB') # OK ``` I noticed that mypy does not flag x['fieldB'] as an error even though the field is potentially not present. I presume that code should always use the `get` method when attempting to access such fields. I'm curious why the TypedDict support in mypy doesn't emit an error in this case. Was that an oversight, or perhaps there was there a concern about generating false positives? My inclination is to always emit an error when code attempts to access a non-required TypedDict field using a subscript. Thoughts? -- Eric Traut Contributor to Pyright & Pylance Microsoft Corp.
On Sun, Mar 7, 2021 at 5:02 PM Eric Traut <eric@traut.com> wrote:
I noticed that mypy does not flag x['fieldB'] as an error even though the field is potentially not present. I presume that code should always use the `get` method when attempting to access such fields. I'm curious why the TypedDict support in mypy doesn't emit an error in this case. Was that an oversight, or perhaps there was there a concern about generating false positives?
We decided to allow x['field'] to avoid false positives when annotating legacy code. We've seen code that first checks whether a key exists and then uses x[...], and various variants of this where a technically non-required key is always present. Also users tend to just use total=False even if some keys are always present, because currently it's tedious to mark a subset of keys as not required. Users may then access the implicitly required keys using x[...], and there may even not be a reasonable default to use with get().
My inclination is to always emit an error when code attempts to access a non-required TypedDict field using a subscript. Thoughts?
This would probably cause quite a few false positives in some existing codebases, but it could be nice for new code, at least once it's easier to declare a subset of keys as non-required. In mypy we'd probably add a strictness flag to enable this. Jukka
On Tue, Mar 9, 2021 at 8:05 AM Jukka Lehtosalo <jlehtosalo@gmail.com> wrote:
On Sun, Mar 7, 2021 at 5:02 PM Eric Traut <eric@traut.com> wrote:
I noticed that mypy does not flag x['fieldB'] as an error even though the field is potentially not present. I presume that code should always use the `get` method when attempting to access such fields. I'm curious why the TypedDict support in mypy doesn't emit an error in this case. Was that an oversight, or perhaps there was there a concern about generating false positives?
We decided to allow x['field'] to avoid false positives when annotating legacy code. We've seen code that first checks whether a key exists and then uses x[...], and various variants of this where a technically non-required key is always present. Also users tend to just use total=False even if some keys are always present, because currently it's tedious to mark a subset of keys as not required. Users may then access the implicitly required keys using x[...], and there may even not be a reasonable default to use with get().
Were any of those variants of a dynamic nature, or were those patterns only checking a static field. I mean: if 'some_key' in typed_dict: ...typed_dict['some_key'] vs if check_keys(typed_dict, keys): ...typed_dict[k] # k in keys I'm asking because in the first case you could use the in expression as a "narrowing"-like guard. Having inside the if a TypedDict with the same fields, but some_key being a required one.
My inclination is to always emit an error when code attempts to access a non-required TypedDict field using a subscript. Thoughts?
This would probably cause quite a few false positives in some existing codebases, but it could be nice for new code, at least once it's easier to declare a subset of keys as non-required. In mypy we'd probably add a strictness flag to enable this.
Jukka _______________________________________________ Typing-sig mailing list -- typing-sig@python.org To unsubscribe send an email to typing-sig-leave@python.org https://mail.python.org/mailman3/lists/typing-sig.python.org/ Member address: skreft@gmail.com
-- Sebastian Kreft
Thanks for the additional context, Jukka. Here's what I've implemented in the latest release of pyright: 1. A subscripted access to a non-required TypedDict key generates an error unless it is guarded by a conditional check of the form `"some_key" in x`. The error is generated in normal type checking mode, not only in "strict" mode. 2. The `get` method for a non-required TypedDict key returns an `Optional[T]` type, whereas the `get` method for a required TypedDict key returns a `T` type (non-optional). -- Eric Traut Contributor to Pyright & Pylance Microsoft Corp.
participants (4)
-
Eric Traut -
Jukka Lehtosalo -
Sebastian Kreft -
Yurij Mikhalevich