Mailman 3 December 2022 - Typing-sig

Re: Typing Meetup
by S Pradeep Kumar 19 Sep '23

19 Sep '23

Updating the Typing Meetup date to Wednesday as there were some conflicts. Let me know if there are any major time conflicts. Date: *Wednesday, November 03, 2021.*Time: 10 am San Francisco time (UTC-7) / 5 pm London time (UTC) Duration: 1 hour Potential discussion topics: + Typing **kwargs using a TypedDict (Franek Magiera?, Pradeep) [1] + Remaining questions about Callable syntax (Steven) + Non-required Protocol fields (Sebastian Rittau?) [2] + Type support for sealed classes (Adrian Freund?) [3] + Safe, typed JSON validation + Type system documentation (Shannon?) If you're working on these or any other topics and would like to start a discussion at the meetup, reach out to me. If you have interesting topics you'd like to see discussed, share them too. Once we have topics, we can always start a discussion even if there's no proposer yet. We will probably have time for 3-4 topics in this session. On Fri, Oct 15, 2021 at 12:55 PM S Pradeep Kumar <gohanpra(a)gmail.com> wrote: >>> >>>> Hello all, >>>> >>>> We plan to hold a virtual Typing Meetup to discuss key typing-related >>>> issues in real-time. The idea is that we can make progress on important or >>>> controversial discussions, get early feedback on in-progress PEPs, and >>>> gauge interest in new proposals. >>>> >>>> A virtual meetup will be more accessible to people in other timezones >>>> than Bay Area in-person meetups. It should also help consolidate >>>> discussions that are often spread out across mailing lists or Github repos. >>>> The tentative plan is to have a monthly meetup, but we can change that as >>>> needed. >>>> >>>> Date: Monday, November 01, 2021. >>>> Time: 10 am San Francisco time (UTC-7) / 5 pm London time (UTC) >>>> Duration: 1 hour >>>> >>>> Time and date are flexible. I'll share the zoom link and agenda the >>>> week before. I'll also record the meetup and share afterwards. >>>> >>>> If you're interested in presenting a new proposal or your work on some >>>> existing proposal, please respond here. We can shoot for 15 minutes per >>>> presentation. I know there are quite a few interesting threads on >>>> typing-sig that haven't seen much activity in a while :) >>>> -- >>>> S Pradeep Kumar >>>> _______________________________________________ >>>> Typing-sig mailing list -- typing-sig(a)python.org >>>> To unsubscribe send an email to typing-sig-leave(a)python.org >>>> https://mail.python.org/mailman3/lists/typing-sig.python.org/ >>>> Member address: guido(a)python.org >>> >>> Links: [1] Typing **kwargs using TypedDict: https://mail.python.org/archives/list/typing-sig@python.org/thread/IUCXT2IQ… [2] Non-required Protocol fields: https://mail.python.org/archives/list/typing-sig@python.org/thread/P6FCGRJU… [3] Sealed classes: https://mail.python.org/archives/list/typing-sig@python.org/thread/QSCT2N4R… -- S Pradeep Kumar

10 60

Type narrowing for TypeGuard in the negative case
by Eric Traut 04 Aug '23

04 Aug '23

Since the introduction of PEP 647 (User-defined Type Guards), we've received a steady stream of input from users saying that they don't like the limitation that type narrowing is applied only in the positive case and is not applied in the negative case. In general, type narrowing is not safe to perform if a user-defined type guard returns False, so I've stood by the original decision not to provide type narrowing in the negative case, but there are cases where such narrowing is safe and desirable. Here's a recent thread where this was discussed in some detail: https://github.com/python/typing/issues/996#issuecomment-1002716830 In this discussion, Ilya Kamen proposed a solution. The proposal is to extend the existing `TypeGuard` to support an optional second type argument. If present, the second argument indicates the narrowed type in the negative type narrowing situation. Here's a simple (admittedly contrived) example: ```python def is_str(val: str | int) -> TypeGuard[str, int]: return isinstance(val, str) def func(val: str | int): if is_str(val): reveal_type(val) # str else: reveal_type(val) # int ``` I've implemented this proposal in the latest published version of pyright (1.1.202) so folks can experiment with it and see if they like it. If there's general consensus that it's the right approach, I can file an amendment for the existing PEP 647 to include this functionality. It was trivial to add to pyright, so I'm optimistic that it would likewise be easy to add to the other type checkers. Another common request for PEP 647 is the desire for an "assert" form of TypeGuard — a way to indicate that a function performs runtime validation of a type, raising an exception if the type is incorrect. It occurred to me that this two-argument form of `TypeGuard` could also be used to handle this use case. The second argument would be specified as a `NoReturn`. I've provisionally implemented this in pyright as well (although it's not in the currently-published version — you'll need to wait until 1.1.203 is published). Here's what this would look like: ```python def validate_sequence(val: Sequence[_T | None]) -> TypeGuard[Sequence[_T], NoReturn]: """Verifies that the sequence contains no None values""" if len([x for x in val if x is not None]) > 0: raise Exception() return True def func(val: Sequence[int | None]): validate_sequence(val) reveal_type(val) # Sequence[int] ``` I'm interested in input on these proposals. -Eric -- Eric Traut Contributor to Pyright & Pylance Microsoft

7 8

Any way to express that a dict is expected to have a key(s) but don't care about other keys?
by Brett Cannon 20 Jun '23

20 Jun '23

Looking at TypedDict it seems that it does not like the idea of superfluous keys existing in a dict. In my case I have a REST call returning JSON for an error and all I'm guaranteed is to get an "error" key. But there may be extra keys that provide more details about the specific error. Unfortunately I don't know the exhaustive list of keys so I can't use a TypedDict with `totality == False` to try to be complete. I can't think of any way to somehow say "require/expect this one key, but other keys are OK". Am I missing anything that would let me do this?

17 35

Runtime Access to Type Parameters
by Saul Shanabrook 07 Jun '23

07 Jun '23

I would like to propose being able to access, at runtime, the type parameters in classmethods and in constructors. I wrote up a draft PEP on this to capture the thinking here: https://github.com/saulshanabrook/peps/blob/master/pep-9999.md It seems easier to add them to classmethods, so even that addition alone would be very welcome! It could be added to Guido's existing draft PR for pep 585, by making a small change in the getattr function (https://github.com/python/cpython/pull/18239/files#diff-a34ae826f869897e56e…), to first do a `__getattribute__` and check if the result is a descriptor (has a `__get__` attribute), and if so passing the generic class itself in as the class, instead of the origin class. I wrote up a little example of that in the PEP. To make it available in the constructor, I proposed adding a global function `get_parameters` to `typing` which returns the parameters in the constructor. I also provide a little mock implementation of that. Note that work is only for custom classes which inherit from `typing.Generic` not for any built in collections.

5 6

Extending published stubs
by Irv Lustig 26 Apr '23

26 Apr '23

I'm the leader of the pandas-stubs project. In pandas, users can extend the library by creating their own accessor via `(a)pd.api.extensions.register_dataframe_accessor` (see https://pandas.pydata.org/docs/reference/api/pandas.api.extensions.register…). So for a `DataFrame`, if you do `(a)pd.api.extensions.register_dataframe_accessor("xyz")`, you now have `DataFrame.xyz` returning the class you annotated. Now suppose you want to create a stub for your new accessor, that would extend the existing pandas-stubs. AFAIK, there isn't a way to do something like this in a stub: ``` from pandas import DataFrame @typing.extend class DataFrame: def xyz(self) -> MyXyzClass: ... ``` The idea here is that there is a class (in this case, `DataFrame`) that already has stubs, and you want to add a method to it. Has this idea been discussed anywhere?

6 11

Support Algebraic Data Types with @sealed
by John Hagen 20 Mar '23

20 Mar '23

The original (superseded) PEP 622 proposed adding a @sealed decorator so that users could define algebraic data types (ADTs) that static type checkers could verify are `match`ed exhaustively: - https://peps.python.org/pep-0622/#sealed-classes-as-algebraic-data-types There is a somewhat fairly-upvoted question about ADTs in Python, indicating interest in this area: - https://stackoverflow.com/questions/16258553 @sealed would be very valuable towards making ADT usage within Python more type-safe. Is there any historical discussion about why @sealed was dropped from the final pattern matching PEP? What would be the next steps needed to push this forward? Would writing a new PEP be the next step?

15 32

PEP 698 typing.override
by Steven Troxler 02 Feb '23

02 Feb '23

A few weeks back we merged PEP 698 for `typing.override` as a draft PEP, with no significant changes since the typing-sig discussion in [1] One of the things we decided for the initial proposal was to avoid any runtime behavior; we considered setting `__override__` but decided not to because (a) it is hard to get the ordering of decorators right for runtime behavior, it depends on which decorators are descriptor- versus wrapper-function-based (b) some objects have fixed slots and therefore cannot have `__override__` set at all, there's no way to produce runtime behavior in these cases I think we should reconsider for two reasons: (a) The author of the overrides library got back to us and specifically requested this [2], because he believes there's potential for libraries to make runtime use of @override, for example automatically applying docstrings from parent classes to child classes which could be useful for notebook users (b) We do indeed set a `__final__` attribute in the @final decorator [3], and after some more thought I think consistency is important My current opinion is that we should make a best-effort attempt to set the attribute (as we do in `(a)typing.final`) and document edge cases that can impact runtime behavior, while ignoring those edge cases in static checkers. Are there any objections before I start working on these changes? ------------------- [1] Previous thread about typing.override proposal: https://mail.python.org/archives/list/typing-sig@python.org/message/7JDW2PK… [2] Discussion in peps repo regarding `__override__` https://github.com/python/peps/pull/2830 [3] @typing.final logic today: https://github.com/python/cpython/blob/main/Lib/typing.py#L2630-L2636

4 7

PEP for dataclass_transform support for converter field descriptor parameter
by joshdcannon＠gmail.com 28 Dec '22

28 Dec '22

Howdy y'all! Picking up on PEP 681, and more specifically this post: https://mail.python.org/archives/list/typing-sig@python.org/thread/5XZQK7GV… I'm interested to hear thoughts on how a PEP to add this functionality would be received. My usage doesn't come from attrs (but shares the problem-space) in that our library defines dataclasses that clients instantiate (which all happen to be frozen and only contain immutable types). Currently to be type-compliant without mypy plugins, we have to define __init__ and do the conversion inside it. Example: @dataclass(frozen=True) class MyData: values: tuple[int, ...] number_map: FrozenOrderedDict[str, int] # a frozen dict, essentially def __init__(self, values: Iterable[int], number_map: Mapping[str, int]): ... However this has a few drawbacks: - Extremely verbose: certainly not as ideal as `values: tuple[int, ...] = my_field(..., converter=tuple)` without the init method - Prone to overload explosion: dicts have an overloaded init with 2 unary-constructors. To be most correct, we'd need to overload our init with each possible cross-product of value unary constructors I think this is in the spirit of attrs' converter field, and I don't feel like forcing our clients to convert is really in the spirit of Python. Additionally, I agree with Jelle Zijlstra in https://mail.python.org/archives/list/typing-sig@python.org/message/7SDEVKQ… that support for this should used typed-converters (instead of using Any, as suggested elsewhere). So far, I can think of a few complexities to document, but otherwise would love to hear if this feature is prohibitively difficult or just doesn't fit. Challenges: - Overloaded converters (such as dict's __init__): Would need to overload the dataclass __init__, and generate a cross-product of overloads if multiple converters - Converters accepting varargs: The type-checker needs to treat the callable's possible first arg as the lone arg (and not error on variable arg functions, again consider dict's constructor) - Generic converters (such as the value `tuple`): There's no great way for Python users pre Py 3.9 to use built-in generics as converters, however users could roll their own converter easily. In Py 3.9+, `tuple[int]` should work at type-checking-time and at runtime. (Whether it would work "out of the box" at type-check-time is up for debate) Thanks for your time and thoughts!

3 2

PEP 696 - default inferred Generic
by Sam Bull 27 Dec '22

27 Dec '22

I don't see an example in the PEP that shows using a default inferred through an __init__() method, and just wanted to check if this would work as expected. Specifically, if I want to have a default value for a Generic, currently I need to use overloads like this: ResponseT = TypeVar("ResponseT", bound=AbstractClientResponse) class ClientSession(Generic[ResponseT]): @overload def __init__(self, response_class: type[ResponseT]): ... @overload def __init__(self: ClientSession[ClientResponse]): ... def __init__(self, response_class: type[ResponseT] = ClientResponse): ... This becomes unwieldy when you have many extra arguments to be maintained in the overloads and exponentially more complex when you add additional types to be generic over (e.g. 8 overloads are needed for a 3 item Generic). Both of these cases are true of aiohttp's ClientSession, which we are looking at improving typing for. With default, I'm hoping that this will work without overloads with just: ResponseT = TypeVar("ResponseT", bound=AbstractClientResponse, default=ClientResponse) class ClientSession(Generic[ResponseT]): def __init__(self, response_class: type[ResponseT] = ClientResponse): ... ClientSession(CustomResponse) # -> ClientSession[CustomResponse] ClientSession() # -> ClientSession[ClientResponse] Can anyone confirm if that is expected to work or not? Might be good if the type checker could even verify the parameter default matches the TypeVar default (though there's probably cases where you have None for the parameter and want it to default to another type...). Thanks, Sam

2 1

Opened and changed typing issues week 2022-W51
by Typing Bot 26 Dec '22

26 Dec '22

The following is an overview of all issues and pull requests in the typing repository on GitHub with the label 'topic: feature' that were opened or updated last week, excluding closed issues. --------------------------------------------------- The following issues and pull requests were opened last week: #1319 Type hinting a non-iterator (non-consumable) iterable opened by @GideonBear https://github.com/python/typing/issues/1319 #1321 Typing decorators which enrich a class opened by @lijok https://github.com/python/typing/issues/1321 --------------------------------------------------- The following issues and pull requests were updated last week: #912 allow `...` in place of generic parameters opened by @DetachHead https://github.com/python/typing/issues/912 #1304 range typing opened by @Godwhitelight https://github.com/python/typing/issues/1304 --------------------------------------------------- All issues and pull requests with the label 'topic: feature' can be viewed under the following URL: https://github.com/python/typing/issues?q=label%3A%22topic%3A+feature%22

1 0