Mailman 3 February 2022 - Typing-sig

Re: Typing Meetup
by S Pradeep Kumar 19 Sep '23

19 Sep '23

Updating the Typing Meetup date to Wednesday as there were some conflicts. Let me know if there are any major time conflicts. Date: *Wednesday, November 03, 2021.*Time: 10 am San Francisco time (UTC-7) / 5 pm London time (UTC) Duration: 1 hour Potential discussion topics: + Typing **kwargs using a TypedDict (Franek Magiera?, Pradeep) [1] + Remaining questions about Callable syntax (Steven) + Non-required Protocol fields (Sebastian Rittau?) [2] + Type support for sealed classes (Adrian Freund?) [3] + Safe, typed JSON validation + Type system documentation (Shannon?) If you're working on these or any other topics and would like to start a discussion at the meetup, reach out to me. If you have interesting topics you'd like to see discussed, share them too. Once we have topics, we can always start a discussion even if there's no proposer yet. We will probably have time for 3-4 topics in this session. On Fri, Oct 15, 2021 at 12:55 PM S Pradeep Kumar <gohanpra(a)gmail.com> wrote: >>> >>>> Hello all, >>>> >>>> We plan to hold a virtual Typing Meetup to discuss key typing-related >>>> issues in real-time. The idea is that we can make progress on important or >>>> controversial discussions, get early feedback on in-progress PEPs, and >>>> gauge interest in new proposals. >>>> >>>> A virtual meetup will be more accessible to people in other timezones >>>> than Bay Area in-person meetups. It should also help consolidate >>>> discussions that are often spread out across mailing lists or Github repos. >>>> The tentative plan is to have a monthly meetup, but we can change that as >>>> needed. >>>> >>>> Date: Monday, November 01, 2021. >>>> Time: 10 am San Francisco time (UTC-7) / 5 pm London time (UTC) >>>> Duration: 1 hour >>>> >>>> Time and date are flexible. I'll share the zoom link and agenda the >>>> week before. I'll also record the meetup and share afterwards. >>>> >>>> If you're interested in presenting a new proposal or your work on some >>>> existing proposal, please respond here. We can shoot for 15 minutes per >>>> presentation. I know there are quite a few interesting threads on >>>> typing-sig that haven't seen much activity in a while :) >>>> -- >>>> S Pradeep Kumar >>>> _______________________________________________ >>>> Typing-sig mailing list -- typing-sig(a)python.org >>>> To unsubscribe send an email to typing-sig-leave(a)python.org >>>> https://mail.python.org/mailman3/lists/typing-sig.python.org/ >>>> Member address: guido(a)python.org >>> >>> Links: [1] Typing **kwargs using TypedDict: https://mail.python.org/archives/list/typing-sig@python.org/thread/IUCXT2IQ… [2] Non-required Protocol fields: https://mail.python.org/archives/list/typing-sig@python.org/thread/P6FCGRJU… [3] Sealed classes: https://mail.python.org/archives/list/typing-sig@python.org/thread/QSCT2N4R… -- S Pradeep Kumar

10 60

Type narrowing for TypeGuard in the negative case
by Eric Traut 04 Aug '23

04 Aug '23

Since the introduction of PEP 647 (User-defined Type Guards), we've received a steady stream of input from users saying that they don't like the limitation that type narrowing is applied only in the positive case and is not applied in the negative case. In general, type narrowing is not safe to perform if a user-defined type guard returns False, so I've stood by the original decision not to provide type narrowing in the negative case, but there are cases where such narrowing is safe and desirable. Here's a recent thread where this was discussed in some detail: https://github.com/python/typing/issues/996#issuecomment-1002716830 In this discussion, Ilya Kamen proposed a solution. The proposal is to extend the existing `TypeGuard` to support an optional second type argument. If present, the second argument indicates the narrowed type in the negative type narrowing situation. Here's a simple (admittedly contrived) example: ```python def is_str(val: str | int) -> TypeGuard[str, int]: return isinstance(val, str) def func(val: str | int): if is_str(val): reveal_type(val) # str else: reveal_type(val) # int ``` I've implemented this proposal in the latest published version of pyright (1.1.202) so folks can experiment with it and see if they like it. If there's general consensus that it's the right approach, I can file an amendment for the existing PEP 647 to include this functionality. It was trivial to add to pyright, so I'm optimistic that it would likewise be easy to add to the other type checkers. Another common request for PEP 647 is the desire for an "assert" form of TypeGuard — a way to indicate that a function performs runtime validation of a type, raising an exception if the type is incorrect. It occurred to me that this two-argument form of `TypeGuard` could also be used to handle this use case. The second argument would be specified as a `NoReturn`. I've provisionally implemented this in pyright as well (although it's not in the currently-published version — you'll need to wait until 1.1.203 is published). Here's what this would look like: ```python def validate_sequence(val: Sequence[_T | None]) -> TypeGuard[Sequence[_T], NoReturn]: """Verifies that the sequence contains no None values""" if len([x for x in val if x is not None]) > 0: raise Exception() return True def func(val: Sequence[int | None]): validate_sequence(val) reveal_type(val) # Sequence[int] ``` I'm interested in input on these proposals. -Eric -- Eric Traut Contributor to Pyright & Pylance Microsoft

7 8

Any way to express that a dict is expected to have a key(s) but don't care about other keys?
by Brett Cannon 20 Jun '23

20 Jun '23

Looking at TypedDict it seems that it does not like the idea of superfluous keys existing in a dict. In my case I have a REST call returning JSON for an error and all I'm guaranteed is to get an "error" key. But there may be extra keys that provide more details about the specific error. Unfortunately I don't know the exhaustive list of keys so I can't use a TypedDict with `totality == False` to try to be complete. I can't think of any way to somehow say "require/expect this one key, but other keys are OK". Am I missing anything that would let me do this?

17 35

Runtime Access to Type Parameters
by Saul Shanabrook 07 Jun '23

07 Jun '23

I would like to propose being able to access, at runtime, the type parameters in classmethods and in constructors. I wrote up a draft PEP on this to capture the thinking here: https://github.com/saulshanabrook/peps/blob/master/pep-9999.md It seems easier to add them to classmethods, so even that addition alone would be very welcome! It could be added to Guido's existing draft PR for pep 585, by making a small change in the getattr function (https://github.com/python/cpython/pull/18239/files#diff-a34ae826f869897e56e…), to first do a `__getattribute__` and check if the result is a descriptor (has a `__get__` attribute), and if so passing the generic class itself in as the class, instead of the origin class. I wrote up a little example of that in the PEP. To make it available in the constructor, I proposed adding a global function `get_parameters` to `typing` which returns the parameters in the constructor. I also provide a little mock implementation of that. Note that work is only for custom classes which inherit from `typing.Generic` not for any built in collections.

5 6

Explicit Type Aliases
by Shannon Zhu 17 Aug '22

17 Aug '22

Hi everyone, I’d like to continue the discussion from our last typing summit on explicit type aliases. A quick summary of the current state and the proposal -- Current state: ``` from typing import List x = List[int] # considered a type alias y : Type[List[int]] = List[int] # considered an expression z : Any # considered a type alias z : x = [] # fine z : y = [] # invalid type error ``` Proposal (with explicit aliases): ``` from typing import List, TypeAlias x = TypeAlias[List[int]] # considered a type alias y = List[int] # considered an expression z : Any # considered an expression z = TypeAlias[Any] # considered a type alias reveal_type(x) # Type[List[int]] reveal_type(y) # Any (return type of __getitem__) z : x = [] # fine z : y = [] # invalid type error ``` Some of the benefits we’re hoping to gain from explicit type aliases: * Clearly distinguish between an unannotated global assignment and a type alias, especially when parsing forward-referencing string annotations. * Avoid the confusing case of type aliases in which some part of the type is invalid or undefined. This would facilitate warnings on invalid types at the alias definition rather than later on when the alias is used. * Make valid and invalid types more intuitive to Python programmers by shifting from delineation of non-aliases with a meta annotation toward delineation of aliases with `TypeAlias`. * Remove special treatment of values annotated as `Any`, which currently break all type aliasing rules. This will also allow us to clean up some of the distinctions we maintain between unannotated values and explicit Anys. Note: We also considered denoting the alias as an annotation (e.g. `x: TypeAlias = int`) as an alternative to the syntax above. I’m interested to hear your thoughts and suggestions on making type aliasing explicit going forward – we’d like to start implementing a version of this in Pyre soon! Shannon

7 10

Dataclass Transform draft PEP
by Erik De Bonte 21 Apr '22

21 Apr '22

I've created a draft PEP for the "Dataclass Transforms" concept that Eric Traut presented at the December 1st Typing SIG meetup. Feedback is welcome! https://github.com/debonte/peps/blob/dataclass_transforms/pep-9999.rst Thanks, Erik

12 52

PEP 681: Descriptor fields with dataclass_transform
by Erik De Bonte 06 Apr '22

06 Apr '22

We're considering an addition to PEP 681 (dataclass_transform) [1] to better support classes with fields that are descriptors. Setting a new dataclass_transform parameter named "transform_descriptor_types" to True would indicate that __init__ parameters corresponding to descriptor fields have the type of the descriptor's setter value parameter rather than the descriptor type. Here's an example: @dataclass_transform(transform_descriptor_types=True) def decorator() -> Callable[[Type[T]], Type[T]]: ... class Descriptor(Generic[_T]): def __get__(self, instance: object, owner: Any) -> Any: ... def __set__(self, instance: object, value: T | None): ... @decorator class InventoryItem: quantity_on_hand: Descriptor[int] In this case, type checkers would understand that the quantity_on_hand parameter of InventoryItem's __init__ method is of type "int | None" rather than "Descriptor[int]". This change was driven by a need in SQLAlchemy, and we'd appreciate any thoughts on broader applicability. Feedback is welcome either via email or on GitHub! [2] A reference implementation is available in Pyright 1.1.222 and Pylance 2022.2.3. [1] https://www.python.org/dev/peps/pep-0681/ [2] https://github.com/debonte/peps/pull/3/files

5 13

Proposal: typing.assert_type() and with typing.assert_error()
by Jelle Zijlstra 17 Mar '22

17 Mar '22

Don't worry, this is the last one for today, but probably the trickiest. These two functions are intended to help users who want to check how their API behaves with a type checker: they want to assert that some type is inferred correctly, or that some code that they want the type checker to catch is rejected. # Specification The signature of typing.assert_type() is: def assert_type(obj: T, typ: TypeForm[T], /) -> T: ... (Using TypeForm from David Foster's pending proposal.) At runtime, this simply returns the first argument unchanged. As with my proposed behavior for reveal_type(), this allows using the function within an expression. A static type checker will emit an error if it encounters a call to assert_type() and the inferred type for the obj argument is not exactly equal to the provided type. The type passed in as the `typ` argument is interpreted the same way the type checker would interpret a type annotation, just as with the first argument to the existing `typing.cast()` function. For example: def f(x: int): assert_type(x, int) # ok assert_type(x, str) # error: type of x is int, not str assert_type(x, Any) # error: type of x is int, not Any The signature of typing.assert_error() is: def assert_error() -> ContextManager[None]: ... At runtime, this returns a context manager that does nothing. At type checking time, the context asserts that an error is found within the block. A static type checker will handle a `with assert_error():` block as follows: - Type check the block as normal, but don't emit any errors to the user - If there were no errors, emit an error - If there were errors, continue as normal For example: with assert_error(): # ok 1 + "x" with assert_error(): # error 1 + 1 # Motivation There has been a longstanding need for a way to check that typed code is understood correctly by the type checker: - https://github.com/python/typeshed/issues/1339 - typeshed issue looking for a way to test typeshed stubs - https://github.com/python/typing/discussions/1030 - recent discussion looking for the same thing in user code - https://pypi.org/project/pytest-mypy-plugins/ provides a mypy-specific way to test inferred types - https://google.github.io/pytype/faq.html#can-i-find-out-what-pytype-thinks-… - pytype added an assert_type() function as proposed here based on user feedback Adding these two functions to the standard library with defined semantics will create a portable, well-documented way to test type checker behavior. This will be useful for typeshed and for any library author who needs to support multiple type checkers. # Use case As a motivating example, I'd like to go over a recent change I made to typeshed: https://github.com/python/typeshed/pull/6941. I wanted to make it so that you cannot call `dict.setdefault()` with one argument if the dict's value type does not include `None`, so I wrote some tricky overloads. With `assert_type()` and `assert_error()`, I could have written tests like these to make sure my stubs work with all type checkers: int_int: dict[int, int] = {} int_any: dict[int, Any] = {} int_optional: dict[int, int | None] = {} with assert_error(): int_int.setdefault(1) # should error because this may set the value to None assert_type(int_int.setdefault(1, 1), int) assert_type(int_any.setdefault(1), Any) assert_type(int_optional.setdefault(1), int | None) What I did instead was run some samples manually with mypy, but that of course doesn't guarantee that the sample continues to work. This example comes from typeshed, but the utility isn't limited to typeshed; it can help any user who wants to make sure the types they wrote behave as they expect. # Open issues The assert_error() context manager is unlike other existing special forms in the typing module, since no other context manager currently has special meaning to the type checker. I'd be happy to hear any alternative ideas for covering this use case. Here are two alternative approaches I considered: - A magical comment, like TypeScript's "// $ExpectError". But a magical comment is not as explicit, is harder to understand for users, and may cause problems with autoformatters. - A function call: perhaps `typing.assert_type(expr, typing.Error)` would assert that evaluating `expr` produces an error. But this wouldn't allow asserting that an error occurs in a statement as opposed to an expression. In https://github.com/python/typing/discussions/1030, we explored several variants for the behavior of assert_type(), such as allowing an explicit way to provide the type as a string. I think the behavior specified here is the most intuitive and consistent with the behavior of other typing constructs, but I'm happy to discuss this further.

11 26

PEP draft: Using TypedDict for more precise **kwargs typing
by Franek Magiera 15 Mar '22

15 Mar '22

Hello, I'd like to share a PEP draft regarding more precise **kwargs typing. It is available at [1]. A quick summary: - There is a proposition to introduce a double asterisk syntax to support this feature, i.e. `def foo(**kwargs: **Movie): ...` - For backwards compatibility the proposition is to reuse PEP 646's Unpack, i.e. `def foo(**kwargs: Unpack[Movie]): ...` - Using total TypedDicts makes all the keywords necessary - Using non-total TypedDicts makes all the keywords optional (I think it makes sense for type checkers to encourage the use of kwargs.get('key') in those cases although it's not written there in the PEP draft) - Passing additional keywords should not raise any errors (the advantage of using **kwargs in the first place is that any dictionary can be passed in and the function can pick and use any keyword arguments it needs) One more thing that is missing from the PEP itself at the moment - keyword collisions should be handled the same way as it is currently described in python docs, that is: - For `def foo(name, **kwargs): ...` kwargs should not contain a 'name' keyword as it will always bind to the first parameter - For `def foo(name, /, **kwargs): ...` kwargs could contain a 'name' keyword as name is a positional argument Please let me know what you think. Franek [1] https://github.com/franekmagiera/peps/blob/more-precise-kwargs-typing/pep-9… [2] https://docs.python.org/3/tutorial/controlflow.html#function-examples

4 11

Should typing.Annotated disallow certain typeforms?
by Gregory Beauregard 28 Feb '22

28 Feb '22

Recently, we fixed Annotated so that it is allowed to wrap Final and ClassVar[1]. Since then I have found a number of other places where Annotated cannot wrap typeforms at runtime. In an issue addressing that ParamSpec P.args and P.kwargs cannot be wrapped at runtime [2], it was brought up that one might not consider these types and thus they might contravene the PEP 593 (Annotated)[3] statement "The first argument to Annotated must be a valid type" and thus we should discuss it publicly. It is my position that the PEP never intended to disallow using Annotated on top-level typeforms (like P.args and P.kwargs), and that moreover the intention either always was or should be that if an Annotated contains something that would be a valid typeform in that location, it's a valid place to use Annotated. If we look back on the typing issue [4] that discussed adding Annotated, the requirement is worded that the first Annotated argument must be a "valid `typing` type" as opposed to a type in some stricter understanding of the term. A stronger argument I think is in the impetus of the PEP itself: the PEP is intended to allow typing and non-typing users of annotation space in Python to coexist. If the PEP intended to completely disallow Annotated to be used in certain annotation spots, the PEP isn't accomplishing one of its primary stated intentions. Is there agreement that Annotated should be allowed in these locations? I will submit appropriate doc/runtime changes given agreement. [1] https://bugs.python.org/issue46491 [2] https://bugs.python.org/issue46643 [3] https://www.python.org/dev/peps/pep-0593/ [4] https://github.com/python/typing/issues/600

7 7