Mailman 3 January 2022 - Typing-sig

Exhaustiveness checking in match statements
by Henry F. Schreiner 22 Jan '22

22 Jan '22

I’d like to gather some opinions or suggestions on exhaustiveness checking on match statements, stemming from https://github.com/python/mypy/pull/10191 (though most future discussion should happen on https://github.com/python/mypy/issues/12010). I’m especially interested in any experience from working with this already, such as in pyright. # case _: assert False would trigger a type check error if reachable. This would allow users to mark match statements as exhaustive. This would require assert False to be treated specially by type checkers in a “match all” case statement, but it’s inline with a common usage of assert false at the end of an if chain today, and wouldn’t introduce new constructs. # --exhaustive-match flag for checkers like mypy This would be a new “strict' flag added to mypy “strict” that would force match statements to be statically exhaustive - this is not (and maybe could not) be enforced at runtime by Python, but could for a static type checker, and would force case _: pass to be added if fallthrough was expected - to make it explicit, and avoid forgetting to add case _: assert False. At some some codebases would likely benefit from this flag. I assume this could be done by other checkers too. It would be a normal mypy strictness flag, so could be enabled/disabled per module, etc. # Current workaround The current workaround for this is to have a function assert_never that takes a NoReturn - NoReturn happens to be (the only) way to spell an empty Union (at least in mypy?). This also works with type narrowing in if statements, so is usable before requiring 3.10+. Until one or both of the above are implemented, it’s the only way to check pattern matching for exhaustiveness, too. Personally, I’d like a slightly nicer way to write this as well, with assert_unreabable or something, or at least a Nothing/EmptyUnion spelling for NoReturn. I think the current leaning is to implement the first two, and do nothing on the third. Thoughts? Especially on the exhaustive-match flag. - Henry

8 9

dataclass_transform support for converter field descriptor parameter
by Erik De Bonte 18 Jan '22

18 Jan '22

Thanks to everyone that has provided feedback on the dataclass_transform PEP so far! There's one open issue<https://github.com/debonte/peps/blob/dataclass_transforms/pep-9999.rst#conv…> right now, and I wondered if anyone had feedback specifically on it: converter field descriptor parameter The attrs library supports a converter field descriptor parameter, which is passed a callable that is called by the generated __init__ method to convert the supplied value to some other desired value. This is tricky to support since the parameter type in the synthesized __init__ method needs to accept uncovered values, but the resulting field is typed according to the output of the converter. There may be no good way to support this because there's not enough information to derive the type of the input parameter. We currently have two ideas: 1. Add support for a converter field descriptor parameter but then use the Any type for the corresponding parameter in the __init__ method. 2. Say that converters are unsupported and recommend that attrs users avoid them. Some aspects of this issue are discussed on the pyright GitHub site. [https://github.com/microsoft/pyright/discussions/1782?sort=old#discussionco…] Thanks, Erik

4 3

Opened and changed typing issues week 2022-W02
by Typing Bot 17 Jan '22

17 Jan '22

The following is an overview of all issues and pull requests in the typing repository on GitHub with the label 'topic: feature' that were opened or updated last week, excluding closed issues. --------------------------------------------------- The following issues and pull requests were updated last week: #241 Allow subclassing without supertyping opened by @dmoisset https://github.com/python/typing/issues/241 #996 TypeGuard type narrowing on returning False opened by @ikamensh https://github.com/python/typing/issues/996 #1000 ParamSpec: Allow only `P.args` opened by @cdce8p https://github.com/python/typing/issues/1000 --------------------------------------------------- All issues and pull requests with the label 'topic: feature' can be viewed under the following URL: https://github.com/python/typing/issues?q=label%3A%22topic%3A+feature%22

1 0

Proposal: Type evaluation functions
by Jelle Zijlstra 15 Jan '22

15 Jan '22

I recently implemented a new feature in pyanalyze that unlocks many things that aren't possible with the current type system and provides a simpler alternative for most use cases of overloads. The idea is that users can specify a special function, marked with the `@evaluated` decorator, written in a (very) restricted subset of Python, to indicate how a type checker should evaluate a function call. For example, this is an implementation of a simplified version of the `open()` builtin: @evaluated def open(mode: str): if is_of_type(mode, Literal["r", "w"]): return TextIO elif is_of_type(mode, Literal["rb", "wb"]): return BinaryIO else: return IO[Any] Compared to overloads, this is shorter (the real `open()` takes lots of other parameters, which have to be repeated with every overload), is easier to read (you don't have to stare at a series of overloads to find the difference), and gives the stub author more precise control over how the code is interpreted (overloads are underspecified). I wrote a detailed spec at https://github.com/quora/pyanalyze/blob/master/docs/type_evaluation.md, but here I'll just briefly cover a few more use cases. 1. Customized errors The specification includes a `show_error()` intrinsic that allows users to emit a specific error message. For example, we may want to warn about the deprecated `U` mode: @evaluated def open(mode: str): if is_of_type(mode, Literal["rU"]): show_error("The U open mode is deprecated", argument=mode) ... 2. Overlapping types Some type checkers implement custom checks for cases where two types do not overlap but are compared. Type evaluation functions provide a way to do this in the stub itself: @evaluated def safe_contains(elt: T1, container: Container[T2]) -> bool: if not is_of_type(elt, T2) and not is_of_type(container, Container[T1]): show_error("Element cannot be a member of container") lst: List[int] safe_contains("x", lst) # error safe_contains(True, lst) # ok (bool is a subclass of int) safe_contains(object(), lst) # ok (List[int] is a subclass of Container[object]) 3. Detecting invalid combinations of arguments `open()` takes an `encoding` argument that must be None if the file is opened in binary mode. But the current typeshed overloads ( https://github.com/python/typeshed/blob/f42de016b8443bea012d06d02c88d41e0bd…) do not catch this case, because `open(..., "rb", encoding="x")` matches the fallback overload with `mode: str`. A full implementation of `open()` is in the pyanalyze test suite: https://github.com/quora/pyanalyze/blob/234625d88aa4c0b2f9ffc76cc9a0ec7b3f0… . I'd love to hear any feedback: Is this worth pursuing as a PEP? Are there more use cases that could be covered? Should the spec be changed?

4 8

How to type a deprecated argument?
by Sebastian Berg 14 Jan '22

14 Jan '22

Hi all, a (hopefully) very brief question: How would you type a deprecated API, in particular a kwarg that is in the process of renaming. [1] In such cases NumPy currently makes the stubs "live in the future": They indicate that the old name is not a valid argument. [2] Now there is a request that the stubs should accept both until the deprecation is finalized. Is that the "obviously" preferred option or is it reasonable to ask users to just use `# type: ignore `? I am more than happy if the answer is just a shrug ;). Cheers, Sebastian [1] I found some discussions about supporting a Deprecated type-hint, but not on a clear preference here. [2] This may partially come from the fact that we have API that should never be used and I think some of that is intentionally not typed.

2 1

Re: RFC on Arbitrary Literal Strings PEP
by Arie Bovenberg 14 Jan '22

14 Jan '22

Hi all, The motivation in the PEP mentions SQL and shell scripts, but logging may also be a potential use-case to mention. Logging does string formatting. So, if you feed it an arbitrary string, you can get into trouble: logger.info(f'look: {untrusted_string}', some_dict) # DANGER if untrusted_string = %(somekey)999999999s IMHO this seemed like a good place to make use of Literal[str]. I’ve opened a discussion at https://bugs.python.org/issue46200, and would be interested to hear some opinions from this crowd.

1 0

Next tensor typing meeting: Monday 17th of January?
by Matthew Rahtz 14 Jan '22

14 Jan '22

Hey everyone, Happy new year! Hope everyone managed to get some proper rest and relaxation over the holidays. We'd tentatively set the 17 of January as the date for our next tensor typing meeting, but once again I'm not sure we have anything to discuss. Does anyone have something they'd like to present or talk about? In terms of progress updates from our end: we've resubmitted PEP 646 to the steering council, and finished a first draft of the CPython implementation: https://github.com/python/cpython/pull/30398 So now we're mainly waiting on word from the steering council about whether they're happy with our revisions. Thanks! Matthew

2 2

Opened and changed typing issues week 2022-W01
by Typing Bot 10 Jan '22

10 Jan '22

The following is an overview of all issues and pull requests in the typing repository on GitHub with the label 'topic: feature' that were opened or updated last week, excluding closed issues. --------------------------------------------------- The following issues and pull requests were opened last week: #1009 ParamSpec: No support for adding keyword-only argument opened by @ItsDrike https://github.com/python/typing/issues/1009 --------------------------------------------------- The following issues and pull requests were updated last week: #182 Define a JSON type opened by @brettcannon https://github.com/python/typing/issues/182 #213 Introduce an Intersection opened by @ilevkivskyi https://github.com/python/typing/issues/213 #801 Introduce a Not type opened by @antonagestam https://github.com/python/typing/issues/801 #996 TypeGuard type narrowing on returning False opened by @ikamensh https://github.com/python/typing/issues/996 #1000 ParamSpec: Allow only `P.args` opened by @cdce8p https://github.com/python/typing/issues/1000 --------------------------------------------------- All issues and pull requests with the label 'topic: feature' can be viewed under the following URL: https://github.com/python/typing/issues?q=label%3A%22topic%3A+feature%22

1 0

`typing.get_type_hints` and implicit `Optional`
by Никита Соболев 09 Jan '22

09 Jan '22

Hi! I am posting this to get additional feedback on a proposed change to CPython. I propose removing the addition of implicit `Optional` type for cases like `def some(arg: int = None):`. Right now `get_type_hints(some)` will return `{'arg': Optional[int]}`. After https://github.com/python/cpython/pull/30304 it will return just `{'arg': int}`. Why? 1. This behavior has bugs. Like the one described in https://bugs.python.org/issue46195 2. Because PEP484 was changed: https://github.com/python/typing/issues/275 and https://github.com/python/peps/pull/689 Does anyone rely heavily on this behavior? Maybe we need to support some kind of optional bool argument like `add_implicit_optional` in `get_type_hints` for backward compat? Feedback on the PR is welcome 🙂 Link: https://github.com/python/cpython/pull/30304 Best, Nikita Sobolev

1 0

Please consider a decorator in typing to mark base class methods whose overrides must call super
by Neil Girdhar 07 Jan '22

07 Jan '22

Not calling super on __init__ is a common source of bugs. So much so, that some base classes explicitly check for it (e.g., https://github.com/deepmind/dm-haiku/blob/main/haiku/_src/module.py#L142). Besides the specific problem with __init__, this is a general pattern that some methods overrides must call super. I was wondering if we could get a a decorator in typing to mark base class methods whose overrides must call super. This suggestion is motivated by an issue I saw on pyright: https://github.com/microsoft/pyright/issues/2789 The basic idea would be something like this: from typing import augmentable class C: @aumentable def f(self, ...) -> None: ... class D(C): def f(self, ...) -> None: # Error: aumentable method override doesn't call super. print(10) By default, methods like __init__, __init_subclass__, __enter__, and __exit__ (any others?) should be "augmentable".

4 4