I agree with Eric that `**kwargs: Foo` means that `kwargs` is a dictionary
of Foo values, as per PEP 484. Changing that would be
backward-incompatible, and special-casing it for TypedDict alone would be
clumsy.
However, it *is* useful to be able to specify the individual types of the
keyword-only parameters we want to accept.
*Proposal*: We allow typing `**kwargs: **KwargsTypedDict`.
Required keyword-only parameters will be required fields of the TypedDict.
Keyword-only parameters with default values will be non-required fields of
the TypedDict.
I wanted a real-world example to motivate this and thought of `json.loads`
and co:
```python
# Simplified and modified slightly from `json.loads` and `json.load`, which
# share the same keyword-only parameters.
# Before.
def loads(
s: Union[str, bytes],
*,
# No default value for this one.
json_decoder: Type[JSONDecoder],
# Has a default value.
parse_int: Optional[Callable[[str], Any]] = ...,
# And a bunch of other keyword-only parameters.
) -> JSON:
...
def load(
fp: SupportsRead[Union[str, bytes]],
*,
json_decoder: Type[JSONDecoder],
parse_int: Optional[Callable[[str], Any]] = ...,
) -> JSON:
...
class JSONDecoder:
def __init__(
self,
*,
parse_int: Optional[Callable[[str], Any]] = ...,
) -> None: ...
```
Note that `loads` and `load` internally just construct a `JSONDecoder` by
passing on their kwargs to the given `json_decoder`. Clearly, these
functions share the same keyword-only parameters.
```python
# These keyword-only parameters had default values. So, they are
non-required
# fields in the TypedDict.
class JSONDecoderKwargs(TypedDict, total=False):
parse_int: Callable[[str], Any]
# And a bunch of others.
# `json_decoder` was a required keyword-only parameter (in my example
above).
# So, it goes in a total TypedDict.
# We also inherit the other fields (preserving their non-requiredness).
class LoadKwargs(JSONDecoderKwargs, total=True):
json_decoder: Type[JSONDecoder]
# After.
def loads(
s: Union[str, bytes],
**kwargs: **LoadKwargs,
) -> JSON:
...
def load(
fp: SupportsRead[Union[str, bytes]],
**kwargs: **LoadKwargs,
) -> JSON:
...
class JSONDecoder:
def __init__(
self,
**kwargs: **JSONDecoderKwargs,
) -> None: ...
# valid
loads(s, json_decoder=MyJsonDecoder)
loads(s, json_decoder=MyJsonDecoder, parse_int=my_parse_int)
# invalid: missing keyword-only parameter `class_info`.
loads(s)
# invalid: type mismatch
loads(s, json_decoder=1)
# invalid: unexpected argument `foo`.
loads(s, json_decoder=MyJsonDecoder, foo=2)
```
Other such cases off the top of my head include
+ `subprocess.run`, `Popen.__init__`, and friends.
+ `sort` and `sorted`, which accept the same keyword-only parameters
Things to consider:
1. This is backward-compatible because it preserves the `**kwargs: int`
behavior of PEP 484.
2. `**kwargs: **KwargsTypedDict` requires changes to the parser. If that's
not worth it, we could settle for something like `**kwargs:
UnpackTypedDict[KwargsTypedDict]`. This is analogous to PEP 646 making
`Unpack[Ts]` a synonym for `*Ts` until syntax support lands.
3. We may also want to allow arbitrary keyword parameters beyond the ones
specifically named.
That is, how would we type the following using the above TypedDict proposal?
```
def foo(*, required: int, non_required: str=..., **kwargs: int) -> None: ...
# valid.
foo(required=1)
# valid.
foo(required=1, extra=7)
# invalid: expected int, got str.
foo(required=1, extra="wrong type")
```
One option is to simply require users to type out the named keyword-only
parameters by hand (as done above for `foo`). This wouldn't allow multiple
such functions to share the same keyword-only parameters, but would not
require any other changes.
A more long-term option is to allow open-ended TypedDicts - ones that
allows arbitrary fields other than the named fields. I believe there was
some discussion about this earlier, but there was no resulting PEP there:
https://mail.python.org/archives/list/typing-sig@python.org/thread/66RITIHDQ....
This might be impractical to wait for.
Yet another option is to always allow arbitrary kwargs beyond the fields in
the TypedDict. I'm against this because it won't let us specify that we
want a finite set of keyword-only parameters, like in the `json.loads`
example above.
In any case, this is a backward-compatible feature that we can defer for
this discussion.
On Sun, Feb 7, 2021 at 3:08 PM Eric Traut
PEP 484 says that a type argument on a **kwargs parameter annotates the _value_ type of the dictionary. In other words, `**kwargs: X` indicates that kwargs is of type `Dict[str, X]`. If X is a TypedDict, then this would indicate that kwargs is a dictionary whose keys are strings and values are all typed dictionaries. I don't see any ambiguity here, but it's clearly not the behavior that you want.
I think you're proposing to make an exception to the PEP 484 rule specifically in the case that X evaluates to a TypedDict. I think that's an ugly inconsistency, and it sets a dangerous precedent. One could make the argument that `Dict` should be exempt from the normal PEP 484 rules here as well. It would mean that there's no way to specify the case where you intend for kwargs to be a dictionary whose values are all type dictionaries. It's also a change that would break backward compatibility, since the rules for PEP 484 are well established. It also opens up questions like what if X is a union that includes a TypedDict or multiple TypedDicts? For all of these reasons, I think this proposal is a no-go.
If I understand your motivation correctly, you are designing an interface where you have (presumably a large number of) keyword parameters that are shared across many methods. Have you considered changing your interface such that you don't expose individual keyword parameters and instead expose a single parameter that accepts a TypedDict? I realize this would change the way callers invoke these methods (e.g. `foo(a=3, b=5) would need to be changed to `foo({"a": 1, "b": 5})`), but it would be type safe and would work with existing Python type checkers.
-- Eric Traut Contributor to Pyright and Pylance Microsoft Corp. _______________________________________________ Typing-sig mailing list -- typing-sig@python.org To unsubscribe send an email to typing-sig-leave@python.org https://mail.python.org/mailman3/lists/typing-sig.python.org/ Member address: gohanpra@gmail.com
-- S Pradeep Kumar