Mailman 3 February 2023 - Typing-sig

Why is TypeGuard not a subtype of bool?
by Kevin Millikin 22 Feb '23

22 Feb '23

PEP 647 (https://peps.python.org/pep-0647/) is very clear that TypeGuard is not a subtype of bool, but without motivation. I don't understand why. To me, it's a bool value with some extra static metadata (that it's a type guard for some type). We should always be able to statically forget about that metadata. A programmer who wants to use the result of a type guard as a bool value will have to write `True if is_str_list(xs) else False` which looks silly to me (and I would normally teach beginners is useless). From the PEP: """ This PEP introduces the symbol TypeGuard exported from the typing module. TypeGuard is a special form that accepts a single type argument. It is used to annotate the return type of a user-defined type guard function. Return statements within a type guard function should return bool values, and type checkers should verify that all return paths return a bool. In all other respects, TypeGuard is a distinct type from bool. It is not a subtype of bool. Therefore, Callable[..., TypeGuard[int]] is not assignable to Callable[..., bool]. """ I'm bothered by "in all other respects". That makes me think I could distinguish them by writing ``` def test(b: bool | TypeGuard[list[str]]): if not isinstance(b, bool): # This is reachable and b has type TypeGuard[list[str]]. ``` And I would expect it to work to distinguish type guards from "other" bool values at runtime: ``` guard = is_str_list(xs) test(guard) # Should hit the TypeGuard branch. ``` But that requires that an implementation will have to somehow tag type guards at runtime, and I don't think any implementation is actually going to want to do that. -- [image: logo] Kevin Millikin Software Engineer kmillikin(a)deepmind.com If you received this email in error, please do not forward it to anyone else (it may contain confidential or privileged information), erase all copies and attachments, and let me know that it has gone to the wrong person.

3 15

Compatibility rules for method overrides
by Steven Troxler 21 Feb '23

21 Feb '23

In PEP 698 (regarding @typing.override), we had a section clarifying that this PEP does *not* add any additional rules about ensuring that subclass methods have type signatures that are compatible with those of superclasses [1]. The way we phrased it was to say that this was already required. After we requested review from the SC, Brett Cannon pointed out, correctly, that in fact no existing PEP says this should be true [2]. All of the existing type checkers do require at least some compatibility, because to not do so would be unsound - for example the following code will fail and we would want type checkers to catch it: ```python class A: def f(self) -> int: return 5 class B: # incompatible override, return class is not a subtype of the return class `int` in parent method def f(self) -> str: return "5" a: A = B() x = 5 + a.f() # Runtime type error! ``` The rule of thumb for compatibility is that all legal function calls for the parent method should be legal, and the output of the subclass method should have a value that is of the type of the parent method return annotation. This implies several specific rules, depending exactly how many edge cases a particular type checker wants to catch - the ones I can think of off the top of my head being: (a) the return annotation on a subclass method should be a subtype of that on the parent class (b) for any parameter in the parent signature, the subclass parameter should be a supertype of what is declared in the parent method (c) named parameter names should not change, that would invalidate some calls (d) new positional or named parameters should have default values, otherwise you could get missing argument errors (e) if the parent type has a *args, then new positional-only / named params must be a supertype of the *args type (f) if the parent type has a *kwargs, then new named / keyword-only params must be a supertype of the *kwargs type I'm not sure whether all type checkers enforce all these rules - some of them (especially the *args and **kwargs limitations) might be obtrusive enough to merit trading off soundness. But my main question is simply: should we try to declare this behavior somewhere in the PEPs repo? We could potentially formalize the rule in a new PEP, or perhaps just add a note to PEP 484 (I believe this is Guido's preference, I'm not sure how clarifying existing PEPs works but that does seem better than a new PEP to me). -------------- [1] PEP 698 compatibility section, prior to Brett's objection: https://github.com/python/peps/blob/c34268dd5b7a2344437ea2785bbb2e1340226b0… [2] Discussion in steering-council issue regarding compatibility: https://github.com/python/steering-council/issues/160

3 2

Macros in type checking contexts
by Tin Tvrtković 21 Feb '23

21 Feb '23

Hello friends, I was curious whether macro systems have ever been brought up in type checking contexts for Python? Some cursory research turned up MacroPy and PEP 638 (Syntactic Macros), but these ideas, as far as I can tell, work inside the interpreter, at import time. But what if we had macros that did their processing before, at 'typechecking' time? The final, processed output is then fed into the typechecker. For example, if attrs and dataclasses were implemented as these 'typechecking' macros, they wouldn't need plugins or PEP 681. Typecheckers could implement support for the macro system and let the community innovate (as we do in the runtime context already). Ideally other tools, like Mypyc, could also operate on the output of this phase. Both MacroPy and PEP 638 claim to operate on AST nodes. Would that even be compatible with Mypy/Pyright/Pyre, or do these require a different representation? As I get ready to do some more work on the Mypy attrs plugin, I feel like we (the typing community) are stuck in a local maximum and unless we have a fundamental rethink, typed Python won't be what it could be. I'm curious to hear what the community thinks of an approach like this.

2 1

Re: TypeGuard is covariant, right?
by Morgan Bartholomew 21 Feb '23

21 Feb '23

Unfortunately `TypeGuard` is bivariant. https://github.com/python/mypy/issues/11230 https://github.com/python/mypy/issues/11229

1 0

Expressing type relationships between keys and values of a dict
by Oscar Benjamin 21 Feb '23

21 Feb '23

Hi all, I have some code that works relatively naturally as untyped Python code and is type safe at runtime but I can't see how to make that be understood by a static type checker. I'm just wondering if anyone has a better idea than just using Any or type:ignore. My actual code is much more complicated but I have a simple demonstration below. The crux of the issue is that I have things like dicts where there are typing relationships between the keys and the values like: function_map = { Key[T1]: Callable[[T1], R], Key[T2]: Callable[[T2], R], ... } The relationships between the keys and values of this dictionary mean that at runtime a lookup should return a function of the correct type. I can't seem to find any way to express this with type annotations though. Ideally I would want for: 1. A type checker to be able to verify the relationships between the keys and values of the dictionary. 2. A type checker to understand these relationships when function_map is used to lookup a function and call it. After looking a bit I'm fairly sure that it is not possible to make this work with an actual dict. I would be happy to try any suggestions though. Here is demonstration code: ================================== from typing import Generic, TypeVar, Hashable, Callable, Any, cast T = TypeVar('T', bound=Hashable, covariant=True) class Key(Generic[T]): def __init__(self, name: str): self.name = name class Atom(Generic[T]): def __init__(self, key: Key[T], value: T): self.key = key self.value = value # A key for Objects with int as internal value # and a function int -> float number_key: Key[int] = Key('number') number_func: Callable[[int], float] = lambda v: float(v)**2 # A key for Objects with str as internal value # and a function str -> float symbol_key: Key[str] = Key('symbol') symbol_func: Callable[[str], float] = lambda v: float(len(v)) # A different key for Objects also having str as internal value # but a different associated function. function_key: Key[str] = Key('function') function_func: Callable[[str], float] = lambda v: -1 # No way to annotate this because number_func and symbol_func # are incompatible. function_map = { number_key: number_func, symbol_key: symbol_func, function_key: function_func, } objects: list[Atom[Hashable]] = [Atom(number_key, 2), Atom(symbol_key, 'foo')] for obj in objects: func = function_map[obj.key] result: float = func(obj.value) print(result) ================================== Here mypy gives: $ mypy --strict demo2.py demo2.py:41:21: error: Cannot call function of unknown type [operator] result: float = func(obj.value) ^~~~~~~~~~~~~~~ Found 1 error in 1 file (checked 1 source file) The error is because I haven't annotated function_map and mypy cannot figure out a usable type for it. Using reveal_type(function_map) gives: builtins.dict[demo2.Key[builtins.object], builtins.function] There doesn't seem to be any way that I can annotate function_map without using Any and/or cast. Does anyone have a better idea how to annotate this? -- Oscar

2 2

TypeGuard is covariant, right?
by Kevin Millikin 20 Feb '23

20 Feb '23

PEP 647 (https://peps.python.org/pep-0647/) doesn't mention the variance of `TypeGuard` as far as I can tell. I think it's obviously covariant. If I want a `Callable[[object], TypeGuard[float]]` then you can give me a `Callable[[object], TypeGuard[int]]`. If it returns True then the argument is definitely a `float` (in fact, it's an `int`). Mypy and pyright appear to agree. This is the simplest explanation of why `TypeGuard` doesn't narrow in the negative case. Your `float` type guard might actually be an `int` one. -- [image: logo] Kevin Millikin Software Engineer kmillikin(a)deepmind.com If you received this email in error, please do not forward it to anyone else (it may contain confidential or privileged information), erase all copies and attachments, and let me know that it has gone to the wrong person.

1 0

Opened and changed typing issues week 2023-W07
by Typing Bot 20 Feb '23

20 Feb '23

The following is an overview of all issues and pull requests in the typing repository on GitHub with the label 'topic: feature' that were opened or updated last week, excluding closed issues. --------------------------------------------------- The following issues and pull requests were updated last week: #213 Introduce an Intersection opened by @ilevkivskyi https://github.com/python/typing/issues/213 #256 Possible to distinguish between Sequence[str]/Iterable[str] and str? opened by @jtatum https://github.com/python/typing/issues/256 --------------------------------------------------- All issues and pull requests with the label 'topic: feature' can be viewed under the following URL: https://github.com/python/typing/issues?q=label%3A%22topic%3A+feature%22

1 0

typing-extensions 4.5.0 released
by Jelle Zijlstra 15 Feb '23

15 Feb '23

I just released typing-extensions 4.5.0. The release adds support for @deprecated (PEP 702) and includes various other bug fixes and improvements: - Runtime support for PEP 702, adding `typing_extensions.deprecated`. Patch by Jelle Zijlstra. - Add better default value for TypeVar `default` parameter, PEP 696. Enables runtime check if `None` was passed as default. Patch by Marc Mueller (@cdce8p). - The `(a)typing_extensions.override` decorator now sets the `.__override__` attribute. Patch by Steven Troxler. - Fix `get_type_hints()` on cross-module inherited `TypedDict` in 3.9 and 3.10. Patch by Carl Meyer. - Add `frozen_default` parameter on `dataclass_transform`. Patch by Erik De Bonte.

1 0

Could we expose DataclassInstance?
by Neil Girdhar 14 Feb '23

14 Feb '23

Currently, pyright checks that only dataclasses are passed to `dataclasses.field`. A dataclass is defined to match the protocol: ``` class DataclassInstance(Protocol): __dataclass_fields__: ClassVar[dict[str, dataclasses.Field[Any]]] ``` This is good enough for users, but would it be more convenient to expose `DataclassInstance` in typing-extensions and ultimately typing?

3 4

Opened and changed typing issues week 2023-W06
by Typing Bot 13 Feb '23

13 Feb '23

The following is an overview of all issues and pull requests in the typing repository on GitHub with the label 'topic: feature' that were opened or updated last week, excluding closed issues. --------------------------------------------------- The following issues and pull requests were updated last week: #213 Introduce an Intersection opened by @ilevkivskyi https://github.com/python/typing/issues/213 #256 Possible to distinguish between Sequence[str]/Iterable[str] and str? opened by @jtatum https://github.com/python/typing/issues/256 #685 Record Types opened by @saulshanabrook https://github.com/python/typing/issues/685 --------------------------------------------------- All issues and pull requests with the label 'topic: feature' can be viewed under the following URL: https://github.com/python/typing/issues?q=label%3A%22topic%3A+feature%22

1 0