Mailman 3 August 2023 - Typing-sig

Re: Typing Meetup
by S Pradeep Kumar 19 Sep '23

19 Sep '23

Updating the Typing Meetup date to Wednesday as there were some conflicts. Let me know if there are any major time conflicts. Date: *Wednesday, November 03, 2021.*Time: 10 am San Francisco time (UTC-7) / 5 pm London time (UTC) Duration: 1 hour Potential discussion topics: + Typing **kwargs using a TypedDict (Franek Magiera?, Pradeep) [1] + Remaining questions about Callable syntax (Steven) + Non-required Protocol fields (Sebastian Rittau?) [2] + Type support for sealed classes (Adrian Freund?) [3] + Safe, typed JSON validation + Type system documentation (Shannon?) If you're working on these or any other topics and would like to start a discussion at the meetup, reach out to me. If you have interesting topics you'd like to see discussed, share them too. Once we have topics, we can always start a discussion even if there's no proposer yet. We will probably have time for 3-4 topics in this session. On Fri, Oct 15, 2021 at 12:55 PM S Pradeep Kumar <gohanpra(a)gmail.com> wrote: >>> >>>> Hello all, >>>> >>>> We plan to hold a virtual Typing Meetup to discuss key typing-related >>>> issues in real-time. The idea is that we can make progress on important or >>>> controversial discussions, get early feedback on in-progress PEPs, and >>>> gauge interest in new proposals. >>>> >>>> A virtual meetup will be more accessible to people in other timezones >>>> than Bay Area in-person meetups. It should also help consolidate >>>> discussions that are often spread out across mailing lists or Github repos. >>>> The tentative plan is to have a monthly meetup, but we can change that as >>>> needed. >>>> >>>> Date: Monday, November 01, 2021. >>>> Time: 10 am San Francisco time (UTC-7) / 5 pm London time (UTC) >>>> Duration: 1 hour >>>> >>>> Time and date are flexible. I'll share the zoom link and agenda the >>>> week before. I'll also record the meetup and share afterwards. >>>> >>>> If you're interested in presenting a new proposal or your work on some >>>> existing proposal, please respond here. We can shoot for 15 minutes per >>>> presentation. I know there are quite a few interesting threads on >>>> typing-sig that haven't seen much activity in a while :) >>>> -- >>>> S Pradeep Kumar >>>> _______________________________________________ >>>> Typing-sig mailing list -- typing-sig(a)python.org >>>> To unsubscribe send an email to typing-sig-leave(a)python.org >>>> https://mail.python.org/mailman3/lists/typing-sig.python.org/ >>>> Member address: guido(a)python.org >>> >>> Links: [1] Typing **kwargs using TypedDict: https://mail.python.org/archives/list/typing-sig@python.org/thread/IUCXT2IQ… [2] Non-required Protocol fields: https://mail.python.org/archives/list/typing-sig@python.org/thread/P6FCGRJU… [3] Sealed classes: https://mail.python.org/archives/list/typing-sig@python.org/thread/QSCT2N4R… -- S Pradeep Kumar

10 60

Heterogeneous, mutable, type-safe mapping
by Ran Benita 31 Aug '23

31 Aug '23

This post doesn't have any particular aim, just a design report. I'm curious if anyone else had done this and has experience to share, or maybe there's a nicer solution for this in Python. ### Motivation In pytest we have plugin system where external plugins may implement various hooks. For the sake of example let's say a plugin wants to do something on each test ("item") setup and teardown: def pytest_runtest_setup(item: pytest.Item) -> None: ... def pytest_runtest_teardown(item: pytest.Item) -> None: ... One thing plugins often want to do is to store/associate some data with the item on setup, then use it on teardown. The traditional way plugins did this was to use an attribute on the `item`: def pytest_runtest_setup(item: pytest.Item) -> None: item.my_plugin_attr = MyPluginAttrType(...) def pytest_runtest_teardown(item: pytest.Item) -> None: use_my_attr(item.my_plugin_attr) # Maybe del item.my_plugin_attr The problem with this is that type checkers are not aware of `my_plugin_attr` and they complain. To make things clean for type checkers, and type safe (no casts/asserts) an alternative solution is needed. ### Rejected solution 1 We can add a dict to each item on the pytest side: class Item: def __init__(self, ...) -> None: ... self.store: Final[Dict[str, ???]] = {} which plugins will use as def pytest_runtest_setup(item: pytest.Item) -> None: item.store["my_plugin_attr"] = MyPluginAttrType(...) This fixes the "undefined attribute" issue, but as the `???` indicates, it's still not good. We can either use `object` and force casts/asserts, or `Any` and lose type safety. ### Rejected solution 2 We can imagine a typing feature which allows a 3rd-party to "overlay" a type it did not define with extra fields/methods. I think some languages support this (TypeScript?). But it does not seem like a desirable feature to me. ### Solution we ended up with We define a `Stash` type (because plugins "stash" their data there), which is used as follows: # In pytest class Item: def __init__(self, ...) -> None: ... self.stash: Final[Stash] = Stash() # In plugin my_plugin_attr: Final = pytest.StashKey[MyPluginAttrType]() def pytest_runtest_setup(item: pytest.Item) -> None: item.stash[my_plugin_attr] = MyPluginAttrType(...) def pytest_runtest_teardown(item: pytest.Item) -> None: use_my_attr(item.stash[my_plugin_attr]) # Maybe del item.stash[my_plugin_attr] Each plugin defines `StashKey`s for its attributes, and the stash key carries the attribute type which makes things type safe. The `Stash` is defined like this (see [0] for full implementation in pytest): T = TypeVar("T") D = TypeVar("D") class StashKey(Generic[T]): __slots__ = () class Stash: __slots__ = ("_storage",) def __init__(self) -> None: self._storage: Dict[StashKey[Any], object] = {} def __setitem__(self, key: StashKey[T], value: T) -> None: self._storage[key] = value def __getitem__(self, key: StashKey[T]) -> T: return cast(T, self._storage[key]) def __delitem__(self, key: StashKey[T]) -> None: del self._storage[key] # .. other mutable mapping methods -- same idea. [0] https://github.com/pytest-dev/pytest/blob/7.4.0/src/_pytest/stash.py ### Advantages No undefined attributes. Type safe. The `StashKey`s are namespaced objects, which makes conflicts between plugins impossible, unlike with string keys. ### Disadvantages Not standard, users are not familiar with idea and are confused, try to use it as a dict. Less ergonomic, need to define the `StashKey`s at the module level.

6 12

Opened and changed typing issues week 2023-W34
by Typing Bot 28 Aug '23

28 Aug '23

The following is an overview of all issues and pull requests in the typing repository on GitHub with the label 'topic: feature' that were opened or updated last week, excluding closed issues. --------------------------------------------------- The following issues and pull requests were updated last week: #256 Possible to distinguish between Sequence[str]/Iterable[str] and str? opened by @jtatum https://github.com/python/typing/issues/256 --------------------------------------------------- All issues and pull requests with the label 'topic: feature' can be viewed under the following URL: https://github.com/python/typing/issues?q=label%3A%22topic%3A+feature%22

1 0

Reliable Analysis of `Annotated`
by Alex Porter 21 Aug '23

21 Aug '23

When using `Annotated`, there does not appear to be a reliable way to analyze what is being annotated. Consider the following: ```python from typing import Annotated, _AnnotatedAlias, reveal_type, get_type_hints from dataclasses import dataclass @dataclass class Foo: bar: Annotated[int, "anno bar"] foo_inst = Foo(1) reveal_type(foo_inst.bar) # Runtime type is 'int' print(get_type_hints(Foo, include_extras=True)) # {'bar': typing.Annotated[int, 'anno bar']} bar_hint = get_type_hints(Foo, include_extras=True)["bar"] print(bar_hint) # typing.Annotated[int, 'anno bar'] print(type(bar_hint)) # <class 'typing._AnnotatedAlias'> print(bar_hint is Annotated) # False print(isinstance(bar_hint, Annotated)) # False print(isinstance(bar_hint, _AnnotatedAlias)) # True print(hasattr(bar_hint, "__metadata__")) # True print(bar_hint.__metadata__) # ('anno bar',) ``` I can programmatically identify that a type has been annotated only by either using `hasattr` or checking that it is an instance of `_AnnotatedAlias`, which is a private type, but not by checking it is an instance of `Annotated`. I can use `hasattr` and `getattr` or `bar_hint.__metadata__`, but type checking doesn't like those options as much. I would prefer something like `get_annotations(obj) -> Tuple[Any, ...] | None: ...`. Does something like this already exist in `typing_extensions` or somewhere else?

2 1

Opened and changed typing issues week 2023-W33
by Typing Bot 21 Aug '23

21 Aug '23

The following is an overview of all issues and pull requests in the typing repository on GitHub with the label 'topic: feature' that were opened or updated last week, excluding closed issues. --------------------------------------------------- The following issues and pull requests were updated last week: #801 Introduce a Not type opened by @antonagestam https://github.com/python/typing/issues/801 #1216 TypeVarTuple Transformations Before Unpack opened by @llchan https://github.com/python/typing/issues/1216 #1273 Type mappings for generics opened by @jonathanslenders https://github.com/python/typing/issues/1273 --------------------------------------------------- All issues and pull requests with the label 'topic: feature' can be viewed under the following URL: https://github.com/python/typing/issues?q=label%3A%22topic%3A+feature%22

1 0

typing.Any subclasses
by Sergei Lebedev 19 Aug '23

19 Aug '23

Hi typing-sig@, Python 3.11 came out with a seemingly minor change allowing typing.Any to be subclassed <https://github.com/python/cpython/issues/91154>, So, for example, from typing import Any class A(Any): x: int is now valid Python. The intent of the change was to have a way of expressing mock-like types which are assignable to and from any type, in the same way typing.Any is. I think this behavior needs to be deprecated and removed. typing.Any is a gradual type with special properties, which is distinct from class types introduced via class definitions. Allowing the two to mix complicates the Python type system in unspecified (and not fully understood) ways. For example, we now need to figure out * What does it mean to subclass typing.Any in a) a generic class b) a protocol c) a typing.NamedTuple d) a typing.TypedDict? * What does it mean for a typing.Any subclass to have attributes? * Are Any subclasses plain subtypes of typing.Any? Does this question even make sense? * How do typing.Any subclasses interact with typing.Any? Are they consistent with typing.Any? Are they consistent subtypes of typing.Any? * etc Sergei

6 11

Opened and changed typing issues week 2023-W32
by Typing Bot 14 Aug '23

14 Aug '23

The following is an overview of all issues and pull requests in the typing repository on GitHub with the label 'topic: feature' that were opened or updated last week, excluding closed issues. --------------------------------------------------- The following issues and pull requests were opened last week: #1445 Should operator `|` on `TypedDict` allow for creating intersection-like dicts? opened by @bluenote10 https://github.com/python/typing/issues/1445 --------------------------------------------------- The following issues and pull requests were updated last week: #213 Introduce an Intersection opened by @ilevkivskyi https://github.com/python/typing/issues/213 #256 Possible to distinguish between Sequence[str]/Iterable[str] and str? opened by @jtatum https://github.com/python/typing/issues/256 --------------------------------------------------- All issues and pull requests with the label 'topic: feature' can be viewed under the following URL: https://github.com/python/typing/issues?q=label%3A%22topic%3A+feature%22

1 0

Opened and changed typing issues week 2023-W31
by Typing Bot 07 Aug '23

07 Aug '23

The following is an overview of all issues and pull requests in the typing repository on GitHub with the label 'topic: feature' that were opened or updated last week, excluding closed issues. --------------------------------------------------- The following issues and pull requests were updated last week: #1043 TypingError: type checker will emit error with a message specified in typing stubs opened by @Akuli https://github.com/python/typing/issues/1043 --------------------------------------------------- All issues and pull requests with the label 'topic: feature' can be viewed under the following URL: https://github.com/python/typing/issues?q=label%3A%22topic%3A+feature%22

1 0

Type narrowing for TypeGuard in the negative case
by Eric Traut 04 Aug '23

04 Aug '23

Since the introduction of PEP 647 (User-defined Type Guards), we've received a steady stream of input from users saying that they don't like the limitation that type narrowing is applied only in the positive case and is not applied in the negative case. In general, type narrowing is not safe to perform if a user-defined type guard returns False, so I've stood by the original decision not to provide type narrowing in the negative case, but there are cases where such narrowing is safe and desirable. Here's a recent thread where this was discussed in some detail: https://github.com/python/typing/issues/996#issuecomment-1002716830 In this discussion, Ilya Kamen proposed a solution. The proposal is to extend the existing `TypeGuard` to support an optional second type argument. If present, the second argument indicates the narrowed type in the negative type narrowing situation. Here's a simple (admittedly contrived) example: ```python def is_str(val: str | int) -> TypeGuard[str, int]: return isinstance(val, str) def func(val: str | int): if is_str(val): reveal_type(val) # str else: reveal_type(val) # int ``` I've implemented this proposal in the latest published version of pyright (1.1.202) so folks can experiment with it and see if they like it. If there's general consensus that it's the right approach, I can file an amendment for the existing PEP 647 to include this functionality. It was trivial to add to pyright, so I'm optimistic that it would likewise be easy to add to the other type checkers. Another common request for PEP 647 is the desire for an "assert" form of TypeGuard — a way to indicate that a function performs runtime validation of a type, raising an exception if the type is incorrect. It occurred to me that this two-argument form of `TypeGuard` could also be used to handle this use case. The second argument would be specified as a `NoReturn`. I've provisionally implemented this in pyright as well (although it's not in the currently-published version — you'll need to wait until 1.1.203 is published). Here's what this would look like: ```python def validate_sequence(val: Sequence[_T | None]) -> TypeGuard[Sequence[_T], NoReturn]: """Verifies that the sequence contains no None values""" if len([x for x in val if x is not None]) > 0: raise Exception() return True def func(val: Sequence[int | None]): validate_sequence(val) reveal_type(val) # Sequence[int] ``` I'm interested in input on these proposals. -Eric -- Eric Traut Contributor to Pyright & Pylance Microsoft

7 8