Mailman 3 March 2022 - Typing-sig

PEP 681: Descriptor fields with dataclass_transform
by Erik De Bonte 05 Apr '22

05 Apr '22

We're considering an addition to PEP 681 (dataclass_transform) [1] to better support classes with fields that are descriptors. Setting a new dataclass_transform parameter named "transform_descriptor_types" to True would indicate that __init__ parameters corresponding to descriptor fields have the type of the descriptor's setter value parameter rather than the descriptor type. Here's an example: @dataclass_transform(transform_descriptor_types=True) def decorator() -> Callable[[Type[T]], Type[T]]: ... class Descriptor(Generic[_T]): def __get__(self, instance: object, owner: Any) -> Any: ... def __set__(self, instance: object, value: T | None): ... @decorator class InventoryItem: quantity_on_hand: Descriptor[int] In this case, type checkers would understand that the quantity_on_hand parameter of InventoryItem's __init__ method is of type "int | None" rather than "Descriptor[int]". This change was driven by a need in SQLAlchemy, and we'd appreciate any thoughts on broader applicability. Feedback is welcome either via email or on GitHub! [2] A reference implementation is available in Pyright 1.1.222 and Pylance 2022.2.3. [1] https://www.python.org/dev/peps/pep-0681/ [2] https://github.com/debonte/peps/pull/3/files

5 13

Opened and changed typing issues week 2022-W12
by Typing Bot 27 Mar '22

27 Mar '22

The following is an overview of all issues and pull requests in the typing repository on GitHub with the label 'topic: feature' that were opened or updated last week, excluding closed issues. --------------------------------------------------- The following issues and pull requests were updated last week: #182 Define a JSON type opened by @brettcannon https://github.com/python/typing/issues/182 #548 Higher-Kinded TypeVars opened by @tek https://github.com/python/typing/issues/548 --------------------------------------------------- All issues and pull requests with the label 'topic: feature' can be viewed under the following URL: https://github.com/python/typing/issues?q=label%3A%22topic%3A+feature%22

1 0

Accepting PEP 675 - Arbitrary Literal String Type
by Gregory P. Smith 21 Mar '22

21 Mar '22

On behalf of the Python Steering Council, we are accepting PEP 675 - Arbitrary Literal String Type <https://peps.python.org/pep-0675/>. TL;DR - PEP 675 allows type checkers to help prevent bugs allowing attacker-controlled data to be passed to APIs that declare themselves as requiring literal, in-code strings. This is a very thorough PEP with a compelling and highly relevant set of use cases. If I tried to call out all the things we like about it, it’d turn into a table of contents. It is long, but everything has a reason to be there. :) Once implemented, we expect it to be a challenge to tighten widely used existing APIs that accept str today to be LiteralString for practical reasons of what existing code calling unrestricted APIs naturally does. The community would benefit from anyone who attempts to move a widely used existing str API to LiteralString sharing their experiences, successful or not. -gps for the steering council

2 1

Accepting PEP 655 - Marking individual TypedDict items as required or potentially-missing
by Gregory P. Smith 21 Mar '22

21 Mar '22

On behalf of the Python Steering Council, we are pleased to accept PEP 655 - Marking individual TypedDict items as required or potentially-missing <https://peps.python.org/pep-0655/>. Thanks for considering the potential for confusion with Optional during the design and recommending best practices in the “how to teach” section. A couple SC members have tried using TypedDict and found it painful, this PEP helps. -gps for the steering council

1 0

Opened and changed typing issues week 2022-W11
by Typing Bot 20 Mar '22

20 Mar '22

The following is an overview of all issues and pull requests in the typing repository on GitHub with the label 'topic: feature' that were opened or updated last week, excluding closed issues. --------------------------------------------------- The following issues and pull requests were updated last week: #307 Allow specifying a default for omitted type parameters opened by @refi64 https://github.com/python/typing/issues/307 #548 Higher-Kinded TypeVars opened by @tek https://github.com/python/typing/issues/548 --------------------------------------------------- All issues and pull requests with the label 'topic: feature' can be viewed under the following URL: https://github.com/python/typing/issues?q=label%3A%22topic%3A+feature%22

1 0

Seeking examples of runtime type checkers
by David Foster 18 Mar '22

18 Mar '22

Hi typing-sig, I'm looking at giving a presentation at the Typing Summit about type checkers that operate at *runtime*, so I'd like to survey what type checkers of this kind folks already use/maintain. For example, here are a few projects I know about: * pydantic: Typechecked dataclasses. [1] * trycast: Typechecked cast of a value against a type annotation object. [2] * isassignable: Extended form of isinstance() that also accepts non-class type forms. [3] * pytypes.typechecked: Checks the type of function arguments & return value at runtime against type annotations. [4] * enforce: Checks the type of function arguments & return value at runtime against type annotations. [5] Do you have any favorite projects that use type annotation objects at runtime? [1]: https://pydantic-docs.helpmanual.io/ [2]: https://github.com/davidfstr/trycast#trycast-1 [3]: https://github.com/davidfstr/trycast#isassignable [4]: https://github.com/Stewori/pytypes [5]: https://github.com/RussBaz/enforce -- David Foster | Seattle, WA, USA Contributor to TypedDict, mypy, and Python's typing system

11 10

Proposal: typing.assert_type() and with typing.assert_error()
by Jelle Zijlstra 16 Mar '22

16 Mar '22

Don't worry, this is the last one for today, but probably the trickiest. These two functions are intended to help users who want to check how their API behaves with a type checker: they want to assert that some type is inferred correctly, or that some code that they want the type checker to catch is rejected. # Specification The signature of typing.assert_type() is: def assert_type(obj: T, typ: TypeForm[T], /) -> T: ... (Using TypeForm from David Foster's pending proposal.) At runtime, this simply returns the first argument unchanged. As with my proposed behavior for reveal_type(), this allows using the function within an expression. A static type checker will emit an error if it encounters a call to assert_type() and the inferred type for the obj argument is not exactly equal to the provided type. The type passed in as the `typ` argument is interpreted the same way the type checker would interpret a type annotation, just as with the first argument to the existing `typing.cast()` function. For example: def f(x: int): assert_type(x, int) # ok assert_type(x, str) # error: type of x is int, not str assert_type(x, Any) # error: type of x is int, not Any The signature of typing.assert_error() is: def assert_error() -> ContextManager[None]: ... At runtime, this returns a context manager that does nothing. At type checking time, the context asserts that an error is found within the block. A static type checker will handle a `with assert_error():` block as follows: - Type check the block as normal, but don't emit any errors to the user - If there were no errors, emit an error - If there were errors, continue as normal For example: with assert_error(): # ok 1 + "x" with assert_error(): # error 1 + 1 # Motivation There has been a longstanding need for a way to check that typed code is understood correctly by the type checker: - https://github.com/python/typeshed/issues/1339 - typeshed issue looking for a way to test typeshed stubs - https://github.com/python/typing/discussions/1030 - recent discussion looking for the same thing in user code - https://pypi.org/project/pytest-mypy-plugins/ provides a mypy-specific way to test inferred types - https://google.github.io/pytype/faq.html#can-i-find-out-what-pytype-thinks-… - pytype added an assert_type() function as proposed here based on user feedback Adding these two functions to the standard library with defined semantics will create a portable, well-documented way to test type checker behavior. This will be useful for typeshed and for any library author who needs to support multiple type checkers. # Use case As a motivating example, I'd like to go over a recent change I made to typeshed: https://github.com/python/typeshed/pull/6941. I wanted to make it so that you cannot call `dict.setdefault()` with one argument if the dict's value type does not include `None`, so I wrote some tricky overloads. With `assert_type()` and `assert_error()`, I could have written tests like these to make sure my stubs work with all type checkers: int_int: dict[int, int] = {} int_any: dict[int, Any] = {} int_optional: dict[int, int | None] = {} with assert_error(): int_int.setdefault(1) # should error because this may set the value to None assert_type(int_int.setdefault(1, 1), int) assert_type(int_any.setdefault(1), Any) assert_type(int_optional.setdefault(1), int | None) What I did instead was run some samples manually with mypy, but that of course doesn't guarantee that the sample continues to work. This example comes from typeshed, but the utility isn't limited to typeshed; it can help any user who wants to make sure the types they wrote behave as they expect. # Open issues The assert_error() context manager is unlike other existing special forms in the typing module, since no other context manager currently has special meaning to the type checker. I'd be happy to hear any alternative ideas for covering this use case. Here are two alternative approaches I considered: - A magical comment, like TypeScript's "// $ExpectError". But a magical comment is not as explicit, is harder to understand for users, and may cause problems with autoformatters. - A function call: perhaps `typing.assert_type(expr, typing.Error)` would assert that evaluating `expr` produces an error. But this wouldn't allow asserting that an error occurs in a statement as opposed to an expression. In https://github.com/python/typing/discussions/1030, we explored several variants for the behavior of assert_type(), such as allowing an explicit way to provide the type as a string. I think the behavior specified here is the most intuitive and consistent with the behavior of other typing constructs, but I'm happy to discuss this further.

11 26

Collecting edge cases revolving around the various PEPs on storing type annotations
by Brett Cannon 15 Mar '22

15 Mar '22

https://discuss.python.org/t/finding-edge-cases-for-peps-484-563-and-649-ty… If you know of any way the three PEPs fail at some edge case, please leave a comment on the above topic so we can gather a corpus of issues. The hope is this will help the Python steering council get an overall view of the known drawbacks between the various PEPs of 484, 563, and 649. FYI I am not subscribed to typing-sig, so I will not see replies, so please leave responses on the discuss.python.org topic.

1 0

PEP draft: Using TypedDict for more precise **kwargs typing
by Franek Magiera 15 Mar '22

15 Mar '22

Hello, I'd like to share a PEP draft regarding more precise **kwargs typing. It is available at [1]. A quick summary: - There is a proposition to introduce a double asterisk syntax to support this feature, i.e. `def foo(**kwargs: **Movie): ...` - For backwards compatibility the proposition is to reuse PEP 646's Unpack, i.e. `def foo(**kwargs: Unpack[Movie]): ...` - Using total TypedDicts makes all the keywords necessary - Using non-total TypedDicts makes all the keywords optional (I think it makes sense for type checkers to encourage the use of kwargs.get('key') in those cases although it's not written there in the PEP draft) - Passing additional keywords should not raise any errors (the advantage of using **kwargs in the first place is that any dictionary can be passed in and the function can pick and use any keyword arguments it needs) One more thing that is missing from the PEP itself at the moment - keyword collisions should be handled the same way as it is currently described in python docs, that is: - For `def foo(name, **kwargs): ...` kwargs should not contain a 'name' keyword as it will always bind to the first parameter - For `def foo(name, /, **kwargs): ...` kwargs could contain a 'name' keyword as name is a positional argument Please let me know what you think. Franek [1] https://github.com/franekmagiera/peps/blob/more-precise-kwargs-typing/pep-9… [2] https://docs.python.org/3/tutorial/controlflow.html#function-examples

4 11

Typed Tensor Shapes
by MarkSaroufim 13 Mar '22

13 Mar '22

Hi folks, I work on PyTorch at Meta and have been desperately needing to use more expressive types for my work. I came across your PEP which mentioned that numeric code requirements were put out of scope <https://peps.python.org/pep-0586/#rejected-or-out-of-scope-ideas> but I'd like to revisit this work and contribute my engineering time to make it happen. Here's my reasoning as to why this problem is important: So the first application is, I maintain torchserve <https://github.com/pytorch/serve> which is a tool to help people deploy PyTorch models in production. A common need people have there is preparing their models for inference by running all sorts of optimizations and profiles. I have a CLI based tool called torchprpep <https://github.com/pytorch/serve/tree/master/experimental/torchprep> which makes this clearer torchprep quantize models/resnet152.pt int8 --input-shape 64,3,7,7 What this command does is change the precision of a model to int8 and then profiles it with a random matrix of shape 64,3,7,7 but there are a few problems with setting up the input format in this manner 1. It needs a way to work with arbitrary dimensions like batch size which could go from 1 to whatever 2. Support a range of shapes 3. Support for multiple inputs like language models that take in a mask and sentence 4. A way to set a dtype for each input or even a range of dtypes Things get complicated which is why similar projects like NVIDIA inference server came up with their own YAML format <https://gist.github.com/msaroufim/0fbacbb659e03bf22661dddbc027a558> which is kinda convoluted considering users could potentially annotate their PyTorch model.forward() calls to get something. I'm not sure if this is the right interface or possible with Python syntax but figured it can showcase the requirements I'm looking for. forward(input : TensorType [Shape [32, 32, -1 ], Dtype [Union[Int8, Float16], Sparse ]) -> bool: I figure in principle this is possible, here's a project called torchtyping <https://github.com/patrick-kidger/torchtyping> to verify tensor correctness which works but isn't integrated into PyTorch. The second application is composing models from model stores, nowadays it's become more common for people to compose pre-trained models instead of training models from scratch thanks to projects like the HuggingFace Hub <https://huggingface.co/models> so now imagine you have - Model 1: Takes in sentences of length up to 50 tokens and generates an image of size 32x32 - Model 2: Takes in an image size 64x64 and generates a binary classification 0/1 - Models 1 and 2 do not compose but if you know the types you can create an adapter to convert images from 32x32 to 64x64 and now they do So I have one very immediate need for dependent types in Python and one future-looking one I'm very excited about. I hope this is interesting to you and if it is please let me know how I can get more involved in helping you. Thanks, Mark

3 2

Typing-sig March 2022