Type checking for **kwargs based on its use
Look at the following example: ``` def some_function(f1: int): assert isinstance(f1, int) def other_function(f0: str, **kwargs): assert isinstance(f0, str) some_function(**kwargs) other_function(f0='a', f1='b') ``` I would expect a static type checker to warn that f1='b' is wrong because the type should be int. There shouldn't be a need to add a type to **kwargs since what is accepted can be deduced from how it is used. Note that some_function could be defined in another module and be used in multiple places, thus the "don't repeat yourself" and "separation of concerns" principles apply. Better to have the type for f1 be only in the definition of some_function and not in the **kwargs. I created a github issue in pyright (https://github.com/microsoft/pyright/issues/3583) and the response was that PEP 484 forbids this. I have seen that there are discussions about TypedDict for more precise typing of **kwargs. However, requiring to use TypedDict for the cases in which it is possible to derive what **kwargs accepts based on its use, seems over-complicating things, which is yet another principle "keep it simple, stupid". I would think that it wouldn't be too problematic to have a PEP stating that static type checkers when **kwargs has not type annotation are allowed to analyze the source code to identify what **kwargs accepts. I would even say that when **kwargs has a type, checkers are allowed to analyze how it is used. Also that the type annotation for **kwargs be not necessary if it is possible to deduce what it accepts. Anyway, I am just giving out this idea to see what happens. As a side note, I have been developing jsonargparse which can be used for automatically creating complex parsers based on signatures (https://jsonargparse.readthedocs.io/en/stable/#classes-methods-and-functions). I do plan to support identifying what **kwargs accepts based on its use. See all the cases currently supported https://jsonargparse.readthedocs.io/en/latest/#ast-resolver.
Hi Mauricio, I'd advise starting a discussion on typing-sig ( https://mail.python.org/archives/list/typing-sig@python.org/) or creating an issue on the python/typing repo (https://github.com/python/typing). The folks interested in typing generally hang out in those places, so those are usually the best places to discuss new typing features. Best wishes, Alex On Fri, Jun 17, 2022 at 2:18 PM Mauricio Villegas via Python-ideas < python-ideas@python.org> wrote:
On Fri, 17 Jun 2022 at 14:19, Mauricio Villegas via Python-ideas <python-ideas@python.org> wrote:
The problem here is that type checkers are *required* to not check unannotated types (by PEP 484, as the pyright maintainers pointed out). That's fundamental to the idea of "gradual typing" and can't be changed without impacting anyone who doesn't fully annotate their code (and many such people exist - not everyone is willing to embrace type checking 100%). The SC have publicly stated that type annotations will remain optional. What you *could* do, I guess, is propose a new annotation that says "allow any type, but do type inference". I don't know whether type checkers have the ability to statically infer the sort of constraint you're suggesting here, but that's a question for the implementers of those checkers, and not a language matter. As a thought, maybe an existing type would work: def other_function(f0: str, **kwargs: dict): or def other_function(f0: str, **kwargs: object): Those don't constrain kwargs beyond what the language mandates, and you don't need to know about how kwargs is used to add them. But if a type checker *can* infer a tighter constraint based on analysis of how kwargs is used in the rest of the program, then presumably it will. Have you tried that?
I would think that it wouldn't be too problematic to have a PEP stating that static type checkers when **kwargs has not type annotation are allowed to analyze the source code to identify what **kwargs accepts. I would even say that when **kwargs has a type, checkers are allowed to analyze how it is used. Also that the type annotation for **kwargs be not necessary if it is possible to deduce what it accepts. Anyway, I am just giving out this idea to see what happens.
It would be problematic to have such a PEP, as I've said - it contradicts the guarantee that type checking is optional. But as Alex Waygood said, this is better raised on the typing SIG, as that's where the typing experts hang out (they don't, to my knowledge, read python-ideas at all). Paul
Hi Mauricio, I'd advise starting a discussion on typing-sig ( https://mail.python.org/archives/list/typing-sig@python.org/) or creating an issue on the python/typing repo (https://github.com/python/typing). The folks interested in typing generally hang out in those places, so those are usually the best places to discuss new typing features. Best wishes, Alex On Fri, Jun 17, 2022 at 2:18 PM Mauricio Villegas via Python-ideas < python-ideas@python.org> wrote:
On Fri, 17 Jun 2022 at 14:19, Mauricio Villegas via Python-ideas <python-ideas@python.org> wrote:
The problem here is that type checkers are *required* to not check unannotated types (by PEP 484, as the pyright maintainers pointed out). That's fundamental to the idea of "gradual typing" and can't be changed without impacting anyone who doesn't fully annotate their code (and many such people exist - not everyone is willing to embrace type checking 100%). The SC have publicly stated that type annotations will remain optional. What you *could* do, I guess, is propose a new annotation that says "allow any type, but do type inference". I don't know whether type checkers have the ability to statically infer the sort of constraint you're suggesting here, but that's a question for the implementers of those checkers, and not a language matter. As a thought, maybe an existing type would work: def other_function(f0: str, **kwargs: dict): or def other_function(f0: str, **kwargs: object): Those don't constrain kwargs beyond what the language mandates, and you don't need to know about how kwargs is used to add them. But if a type checker *can* infer a tighter constraint based on analysis of how kwargs is used in the rest of the program, then presumably it will. Have you tried that?
I would think that it wouldn't be too problematic to have a PEP stating that static type checkers when **kwargs has not type annotation are allowed to analyze the source code to identify what **kwargs accepts. I would even say that when **kwargs has a type, checkers are allowed to analyze how it is used. Also that the type annotation for **kwargs be not necessary if it is possible to deduce what it accepts. Anyway, I am just giving out this idea to see what happens.
It would be problematic to have such a PEP, as I've said - it contradicts the guarantee that type checking is optional. But as Alex Waygood said, this is better raised on the typing SIG, as that's where the typing experts hang out (they don't, to my knowledge, read python-ideas at all). Paul
participants (3)
-
Alex Waygood -
Mauricio Villegas -
Paul Moore