Mailman 3 February 2022 - Typing-sig

Re: Typing Meetup
by S Pradeep Kumar 19 Sep '23

19 Sep '23

Updating the Typing Meetup date to Wednesday as there were some conflicts. Let me know if there are any major time conflicts. Date: *Wednesday, November 03, 2021.*Time: 10 am San Francisco time (UTC-7) / 5 pm London time (UTC) Duration: 1 hour Potential discussion topics: + Typing **kwargs using a TypedDict (Franek Magiera?, Pradeep) [1] + Remaining questions about Callable syntax (Steven) + Non-required Protocol fields (Sebastian Rittau?) [2] + Type support for sealed classes (Adrian Freund?) [3] + Safe, typed JSON validation + Type system documentation (Shannon?) If you're working on these or any other topics and would like to start a discussion at the meetup, reach out to me. If you have interesting topics you'd like to see discussed, share them too. Once we have topics, we can always start a discussion even if there's no proposer yet. We will probably have time for 3-4 topics in this session. On Fri, Oct 15, 2021 at 12:55 PM S Pradeep Kumar <gohanpra(a)gmail.com> wrote: >>> >>>> Hello all, >>>> >>>> We plan to hold a virtual Typing Meetup to discuss key typing-related >>>> issues in real-time. The idea is that we can make progress on important or >>>> controversial discussions, get early feedback on in-progress PEPs, and >>>> gauge interest in new proposals. >>>> >>>> A virtual meetup will be more accessible to people in other timezones >>>> than Bay Area in-person meetups. It should also help consolidate >>>> discussions that are often spread out across mailing lists or Github repos. >>>> The tentative plan is to have a monthly meetup, but we can change that as >>>> needed. >>>> >>>> Date: Monday, November 01, 2021. >>>> Time: 10 am San Francisco time (UTC-7) / 5 pm London time (UTC) >>>> Duration: 1 hour >>>> >>>> Time and date are flexible. I'll share the zoom link and agenda the >>>> week before. I'll also record the meetup and share afterwards. >>>> >>>> If you're interested in presenting a new proposal or your work on some >>>> existing proposal, please respond here. We can shoot for 15 minutes per >>>> presentation. I know there are quite a few interesting threads on >>>> typing-sig that haven't seen much activity in a while :) >>>> -- >>>> S Pradeep Kumar >>>> _______________________________________________ >>>> Typing-sig mailing list -- typing-sig(a)python.org >>>> To unsubscribe send an email to typing-sig-leave(a)python.org >>>> https://mail.python.org/mailman3/lists/typing-sig.python.org/ >>>> Member address: guido(a)python.org >>> >>> Links: [1] Typing **kwargs using TypedDict: https://mail.python.org/archives/list/typing-sig@python.org/thread/IUCXT2IQ… [2] Non-required Protocol fields: https://mail.python.org/archives/list/typing-sig@python.org/thread/P6FCGRJU… [3] Sealed classes: https://mail.python.org/archives/list/typing-sig@python.org/thread/QSCT2N4R… -- S Pradeep Kumar

10 60

Type narrowing for TypeGuard in the negative case
by Eric Traut 04 Aug '23

04 Aug '23

Since the introduction of PEP 647 (User-defined Type Guards), we've received a steady stream of input from users saying that they don't like the limitation that type narrowing is applied only in the positive case and is not applied in the negative case. In general, type narrowing is not safe to perform if a user-defined type guard returns False, so I've stood by the original decision not to provide type narrowing in the negative case, but there are cases where such narrowing is safe and desirable. Here's a recent thread where this was discussed in some detail: https://github.com/python/typing/issues/996#issuecomment-1002716830 In this discussion, Ilya Kamen proposed a solution. The proposal is to extend the existing `TypeGuard` to support an optional second type argument. If present, the second argument indicates the narrowed type in the negative type narrowing situation. Here's a simple (admittedly contrived) example: ```python def is_str(val: str | int) -> TypeGuard[str, int]: return isinstance(val, str) def func(val: str | int): if is_str(val): reveal_type(val) # str else: reveal_type(val) # int ``` I've implemented this proposal in the latest published version of pyright (1.1.202) so folks can experiment with it and see if they like it. If there's general consensus that it's the right approach, I can file an amendment for the existing PEP 647 to include this functionality. It was trivial to add to pyright, so I'm optimistic that it would likewise be easy to add to the other type checkers. Another common request for PEP 647 is the desire for an "assert" form of TypeGuard — a way to indicate that a function performs runtime validation of a type, raising an exception if the type is incorrect. It occurred to me that this two-argument form of `TypeGuard` could also be used to handle this use case. The second argument would be specified as a `NoReturn`. I've provisionally implemented this in pyright as well (although it's not in the currently-published version — you'll need to wait until 1.1.203 is published). Here's what this would look like: ```python def validate_sequence(val: Sequence[_T | None]) -> TypeGuard[Sequence[_T], NoReturn]: """Verifies that the sequence contains no None values""" if len([x for x in val if x is not None]) > 0: raise Exception() return True def func(val: Sequence[int | None]): validate_sequence(val) reveal_type(val) # Sequence[int] ``` I'm interested in input on these proposals. -Eric -- Eric Traut Contributor to Pyright & Pylance Microsoft

7 8

Any way to express that a dict is expected to have a key(s) but don't care about other keys?
by Brett Cannon 20 Jun '23

20 Jun '23

Looking at TypedDict it seems that it does not like the idea of superfluous keys existing in a dict. In my case I have a REST call returning JSON for an error and all I'm guaranteed is to get an "error" key. But there may be extra keys that provide more details about the specific error. Unfortunately I don't know the exhaustive list of keys so I can't use a TypedDict with `totality == False` to try to be complete. I can't think of any way to somehow say "require/expect this one key, but other keys are OK". Am I missing anything that would let me do this?

17 35

Runtime Access to Type Parameters
by Saul Shanabrook 07 Jun '23

07 Jun '23

I would like to propose being able to access, at runtime, the type parameters in classmethods and in constructors. I wrote up a draft PEP on this to capture the thinking here: https://github.com/saulshanabrook/peps/blob/master/pep-9999.md It seems easier to add them to classmethods, so even that addition alone would be very welcome! It could be added to Guido's existing draft PR for pep 585, by making a small change in the getattr function (https://github.com/python/cpython/pull/18239/files#diff-a34ae826f869897e56e…), to first do a `__getattribute__` and check if the result is a descriptor (has a `__get__` attribute), and if so passing the generic class itself in as the class, instead of the origin class. I wrote up a little example of that in the PEP. To make it available in the constructor, I proposed adding a global function `get_parameters` to `typing` which returns the parameters in the constructor. I also provide a little mock implementation of that. Note that work is only for custom classes which inherit from `typing.Generic` not for any built in collections.

5 6

Explicit Type Aliases
by Shannon Zhu 17 Aug '22

17 Aug '22

Hi everyone, I’d like to continue the discussion from our last typing summit on explicit type aliases. A quick summary of the current state and the proposal -- Current state: ``` from typing import List x = List[int] # considered a type alias y : Type[List[int]] = List[int] # considered an expression z : Any # considered a type alias z : x = [] # fine z : y = [] # invalid type error ``` Proposal (with explicit aliases): ``` from typing import List, TypeAlias x = TypeAlias[List[int]] # considered a type alias y = List[int] # considered an expression z : Any # considered an expression z = TypeAlias[Any] # considered a type alias reveal_type(x) # Type[List[int]] reveal_type(y) # Any (return type of __getitem__) z : x = [] # fine z : y = [] # invalid type error ``` Some of the benefits we’re hoping to gain from explicit type aliases: * Clearly distinguish between an unannotated global assignment and a type alias, especially when parsing forward-referencing string annotations. * Avoid the confusing case of type aliases in which some part of the type is invalid or undefined. This would facilitate warnings on invalid types at the alias definition rather than later on when the alias is used. * Make valid and invalid types more intuitive to Python programmers by shifting from delineation of non-aliases with a meta annotation toward delineation of aliases with `TypeAlias`. * Remove special treatment of values annotated as `Any`, which currently break all type aliasing rules. This will also allow us to clean up some of the distinctions we maintain between unannotated values and explicit Anys. Note: We also considered denoting the alias as an annotation (e.g. `x: TypeAlias = int`) as an alternative to the syntax above. I’m interested to hear your thoughts and suggestions on making type aliasing explicit going forward – we’d like to start implementing a version of this in Pyre soon! Shannon

7 10

Dataclass Transform draft PEP
by Erik De Bonte 21 Apr '22

21 Apr '22

I've created a draft PEP for the "Dataclass Transforms" concept that Eric Traut presented at the December 1st Typing SIG meetup. Feedback is welcome! https://github.com/debonte/peps/blob/dataclass_transforms/pep-9999.rst Thanks, Erik

12 52

PEP 681: Descriptor fields with dataclass_transform
by Erik De Bonte 06 Apr '22

06 Apr '22

We're considering an addition to PEP 681 (dataclass_transform) [1] to better support classes with fields that are descriptors. Setting a new dataclass_transform parameter named "transform_descriptor_types" to True would indicate that __init__ parameters corresponding to descriptor fields have the type of the descriptor's setter value parameter rather than the descriptor type. Here's an example: @dataclass_transform(transform_descriptor_types=True) def decorator() -> Callable[[Type[T]], Type[T]]: ... class Descriptor(Generic[_T]): def __get__(self, instance: object, owner: Any) -> Any: ... def __set__(self, instance: object, value: T | None): ... @decorator class InventoryItem: quantity_on_hand: Descriptor[int] In this case, type checkers would understand that the quantity_on_hand parameter of InventoryItem's __init__ method is of type "int | None" rather than "Descriptor[int]". This change was driven by a need in SQLAlchemy, and we'd appreciate any thoughts on broader applicability. Feedback is welcome either via email or on GitHub! [2] A reference implementation is available in Pyright 1.1.222 and Pylance 2022.2.3. [1] https://www.python.org/dev/peps/pep-0681/ [2] https://github.com/debonte/peps/pull/3/files

5 13

Proposal: typing.assert_type() and with typing.assert_error()
by Jelle Zijlstra 17 Mar '22

17 Mar '22

Don't worry, this is the last one for today, but probably the trickiest. These two functions are intended to help users who want to check how their API behaves with a type checker: they want to assert that some type is inferred correctly, or that some code that they want the type checker to catch is rejected. # Specification The signature of typing.assert_type() is: def assert_type(obj: T, typ: TypeForm[T], /) -> T: ... (Using TypeForm from David Foster's pending proposal.) At runtime, this simply returns the first argument unchanged. As with my proposed behavior for reveal_type(), this allows using the function within an expression. A static type checker will emit an error if it encounters a call to assert_type() and the inferred type for the obj argument is not exactly equal to the provided type. The type passed in as the `typ` argument is interpreted the same way the type checker would interpret a type annotation, just as with the first argument to the existing `typing.cast()` function. For example: def f(x: int): assert_type(x, int) # ok assert_type(x, str) # error: type of x is int, not str assert_type(x, Any) # error: type of x is int, not Any The signature of typing.assert_error() is: def assert_error() -> ContextManager[None]: ... At runtime, this returns a context manager that does nothing. At type checking time, the context asserts that an error is found within the block. A static type checker will handle a `with assert_error():` block as follows: - Type check the block as normal, but don't emit any errors to the user - If there were no errors, emit an error - If there were errors, continue as normal For example: with assert_error(): # ok 1 + "x" with assert_error(): # error 1 + 1 # Motivation There has been a longstanding need for a way to check that typed code is understood correctly by the type checker: - https://github.com/python/typeshed/issues/1339 - typeshed issue looking for a way to test typeshed stubs - https://github.com/python/typing/discussions/1030 - recent discussion looking for the same thing in user code - https://pypi.org/project/pytest-mypy-plugins/ provides a mypy-specific way to test inferred types - https://google.github.io/pytype/faq.html#can-i-find-out-what-pytype-thinks-… - pytype added an assert_type() function as proposed here based on user feedback Adding these two functions to the standard library with defined semantics will create a portable, well-documented way to test type checker behavior. This will be useful for typeshed and for any library author who needs to support multiple type checkers. # Use case As a motivating example, I'd like to go over a recent change I made to typeshed: https://github.com/python/typeshed/pull/6941. I wanted to make it so that you cannot call `dict.setdefault()` with one argument if the dict's value type does not include `None`, so I wrote some tricky overloads. With `assert_type()` and `assert_error()`, I could have written tests like these to make sure my stubs work with all type checkers: int_int: dict[int, int] = {} int_any: dict[int, Any] = {} int_optional: dict[int, int | None] = {} with assert_error(): int_int.setdefault(1) # should error because this may set the value to None assert_type(int_int.setdefault(1, 1), int) assert_type(int_any.setdefault(1), Any) assert_type(int_optional.setdefault(1), int | None) What I did instead was run some samples manually with mypy, but that of course doesn't guarantee that the sample continues to work. This example comes from typeshed, but the utility isn't limited to typeshed; it can help any user who wants to make sure the types they wrote behave as they expect. # Open issues The assert_error() context manager is unlike other existing special forms in the typing module, since no other context manager currently has special meaning to the type checker. I'd be happy to hear any alternative ideas for covering this use case. Here are two alternative approaches I considered: - A magical comment, like TypeScript's "// $ExpectError". But a magical comment is not as explicit, is harder to understand for users, and may cause problems with autoformatters. - A function call: perhaps `typing.assert_type(expr, typing.Error)` would assert that evaluating `expr` produces an error. But this wouldn't allow asserting that an error occurs in a statement as opposed to an expression. In https://github.com/python/typing/discussions/1030, we explored several variants for the behavior of assert_type(), such as allowing an explicit way to provide the type as a string. I think the behavior specified here is the most intuitive and consistent with the behavior of other typing constructs, but I'm happy to discuss this further.

11 26

PEP draft: Using TypedDict for more precise **kwargs typing
by Franek Magiera 15 Mar '22

15 Mar '22

Hello, I'd like to share a PEP draft regarding more precise **kwargs typing. It is available at [1]. A quick summary: - There is a proposition to introduce a double asterisk syntax to support this feature, i.e. `def foo(**kwargs: **Movie): ...` - For backwards compatibility the proposition is to reuse PEP 646's Unpack, i.e. `def foo(**kwargs: Unpack[Movie]): ...` - Using total TypedDicts makes all the keywords necessary - Using non-total TypedDicts makes all the keywords optional (I think it makes sense for type checkers to encourage the use of kwargs.get('key') in those cases although it's not written there in the PEP draft) - Passing additional keywords should not raise any errors (the advantage of using **kwargs in the first place is that any dictionary can be passed in and the function can pick and use any keyword arguments it needs) One more thing that is missing from the PEP itself at the moment - keyword collisions should be handled the same way as it is currently described in python docs, that is: - For `def foo(name, **kwargs): ...` kwargs should not contain a 'name' keyword as it will always bind to the first parameter - For `def foo(name, /, **kwargs): ...` kwargs could contain a 'name' keyword as name is a positional argument Please let me know what you think. Franek [1] https://github.com/franekmagiera/peps/blob/more-precise-kwargs-typing/pep-9… [2] https://docs.python.org/3/tutorial/controlflow.html#function-examples

4 11

Should typing.Annotated disallow certain typeforms?
by Gregory Beauregard 28 Feb '22

28 Feb '22

Recently, we fixed Annotated so that it is allowed to wrap Final and ClassVar[1]. Since then I have found a number of other places where Annotated cannot wrap typeforms at runtime. In an issue addressing that ParamSpec P.args and P.kwargs cannot be wrapped at runtime [2], it was brought up that one might not consider these types and thus they might contravene the PEP 593 (Annotated)[3] statement "The first argument to Annotated must be a valid type" and thus we should discuss it publicly. It is my position that the PEP never intended to disallow using Annotated on top-level typeforms (like P.args and P.kwargs), and that moreover the intention either always was or should be that if an Annotated contains something that would be a valid typeform in that location, it's a valid place to use Annotated. If we look back on the typing issue [4] that discussed adding Annotated, the requirement is worded that the first Annotated argument must be a "valid `typing` type" as opposed to a type in some stricter understanding of the term. A stronger argument I think is in the impetus of the PEP itself: the PEP is intended to allow typing and non-typing users of annotation space in Python to coexist. If the PEP intended to completely disallow Annotated to be used in certain annotation spots, the PEP isn't accomplishing one of its primary stated intentions. Is there agreement that Annotated should be allowed in these locations? I will submit appropriate doc/runtime changes given agreement. [1] https://bugs.python.org/issue46491 [2] https://bugs.python.org/issue46643 [3] https://www.python.org/dev/peps/pep-0593/ [4] https://github.com/python/typing/issues/600

7 7

Do any type checkers support annotations by decorator?
by Ethan Furman 28 Feb '22

28 Feb '22

As you can guess, I don't know how type checkers work -- so my question: given the following annotated function: def foobar(spam, eggs, ham='tasty'): pass If it were annotated via decorator [^1]: @annotate( spam=str, eggs=int, ham=str, ) Would mypy and the others recognize those annotations? -- ~Ethan~ [^1]: the decorator would populate the function's `__annotations__` variable.

3 2

Opened and changed typing issues week 2022-W08
by Typing Bot 28 Feb '22

28 Feb '22

The following is an overview of all issues and pull requests in the typing repository on GitHub with the label 'topic: feature' that were opened or updated last week, excluding closed issues. --------------------------------------------------- The following issues and pull requests were updated last week: #593 Buffer protocol types opened by @srittau https://github.com/python/typing/issues/593 --------------------------------------------------- All issues and pull requests with the label 'topic: feature' can be viewed under the following URL: https://github.com/python/typing/issues?q=label%3A%22topic%3A+feature%22

1 0

Adding support for a Pick type
by Patrick Arminio 27 Feb '22

27 Feb '22

Hi everyone! Recently I felt the need for a `Pick` type in Python. The name is "inspired" from TypeScript's `Pick` type (see https://www.typescriptlang.org/docs/handbook/utility-types.html#picktype-ke…). Here's an example of how it work, let' say that we have a `User` model that represents a table in a database and we have a `fetch_user` function that returns a `User` with only `id` and `name`. Here's how the code could look like: ```python from typing import Pick class User: id: int name: str password: str email: str def fetch_user(name: str) -> Pick[User, "id" | "name"]: return fetch_from_db("user", name=name, fields=["id", "name"]) ``` Note: I'm making use of the union syntax on strings, which I don't think it is supported yet. I've done so in order to keep the code a bit simpler. I wanted to see what's the feeling for a feature like this. I think it might be quite useful in the scenario I described above, I guess it could make some ORM's code more type safe 😊 I can try to write a PEP for this and if anyone is interested in helping I'd happy to work with you 😊

8 13

Clarification about PEP 681 statement
by Neil Girdhar 26 Feb '22

26 Feb '22

Hello, I'm reading over PEP 681, and it says here (https://www.python.org/dev/peps/pep-0681/#kw-only-field-descriptor-parameter) that "The attrs library supports a kw_only parameter for individual fields. We chose not to support a kw_only parameter, since it is specific to attrs." Am I misunderstanding this? In Python 3.10, we can do `dataclasses.field(kw_only=True)`. Thanks, Neil

2 1

Making ``cast`` a built-in/keyword?
by Jonathan Slenders 25 Feb '22

25 Feb '22

Hi all, Right now, I'm struggling a bit with the new Union syntax. It seems very important to keep in mind where it can be used and where it shouldn't be used. Some places I've seen where it can't be used: - In type aliases: `T = A | B` - When using a cast: `x = cast(A | B, y)` - In Pydantic models. - In NewType definitions: `T = NewType('T', A | B)` This means, that we have to teach our users to be very careful about this, and know when to use `Union[...]` or `Optional[...]` instead. So, about `cast()` specifically, I wonder whether it would make sense to turn that into a keyword, so that it doesn't evaluate the first argument at runtime, and is completely transparent. Would that be desirable? For the other situations, I'm not sure what we could do about those, beside quoting the type. Sorry if this has been brought up before. I wonder whether there's any consensus about this and what direction we want to go in the future? Thanks a lot, Jonathan

3 3

PEP 655: Marking individual TypedDict items as required or potentially-missing
by David Foster 23 Feb '22

23 Feb '22

The new syntax Required[...] and NotRequired[...] for marking individual keys of a TypedDict has now been formally proposed as a draft PEP. Please see the latest text at the bottom of this email, or online at: https://www.python.org/dev/peps/pep-0655/ Comments welcome. :) Meanwhile I'll proceed to implement this syntax in mypy. -- David Foster | Seattle, WA, USA Contributor to TypedDict support for mypy >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> PEP: 655 Title: Marking individual TypedDict items as required or potentially-missing Author: David Foster <david at dafoster.net> Sponsor: Guido van Rossum <guido at python.org> Discussions-To: typing-sig at python.org Status: Draft Type: Standards Track Content-Type: text/x-rst Requires: 604 Created: 30-Jan-2021 Python-Version: 3.10 Post-History: 31-Jan-2021, 11-Feb-2021, 20-Feb-2021, 26-Feb-2021 Abstract ======== `PEP 589 <https://www.python.org/dev/peps/pep-0589/>`__ defines syntax for declaring a TypedDict with all required keys and syntax for defining a TypedDict with `all potentially-missing keys <https://www.python.org/dev/peps/pep-0589/#totality>`__ however it does not provide any syntax to declare some keys as required and others as potentially-missing. This PEP introduces two new syntaxes: ``Required[...]`` which can be used on individual items of a TypedDict to mark them as required, and ``NotRequired[...]`` which can be used on individual items to mark them as potentially-missing. Motivation ========== It is not uncommon to want to define a TypedDict with some keys that are required and others that are potentially-missing. Currently the only way to define such a TypedDict is to declare one TypedDict with one value for ``total`` and then inherit it from another TypedDict with a different value for ``total``: :: class _MovieBase(TypedDict): # implicitly total=True title: str class Movie(_MovieBase, total=False): year: int Having to declare two different TypedDict types for this purpose is cumbersome. Rationale ========= One might think it unusual to propose syntax that prioritizes marking *required* keys rather than syntax for *potentially-missing* keys, as is customary in other languages like TypeScript: :: interface Movie { title: string; year?: number; // ? marks potentially-missing keys } The difficulty is that the best word for marking a potentially-missing key, ``Optional[...]``, is already used in Python for a completely different purpose: marking values that could be either of a particular type or ``None``. In particular the following does not work: :: class Movie(TypedDict): ... year: Optional[int] # means int|None, not potentially-missing! Attempting to use any synonym of “optional” to mark potentially-missing keys (like ``Missing[...]``) would be too similar to ``Optional[...]`` and be easy to confuse with it. Thus it was decided to focus on positive-form phrasing for required keys instead, which is straightforward to spell as ``Required[...]``. Nevertheless it is common for folks wanting to extend a regular (``total=True``) TypedDict to only want to add a small number of potentially-missing keys, which necessitates a way to mark keys that are *not* required and potentially-missing, and so we also allow the ``NotRequired[...]`` form for that case. Specification ============= The ``typing.Required`` type qualifier is used to indicate that a variable declared in a TypedDict definition is a required key: :: class Movie(TypedDict, total=False): title: Required[str] year: int Additionally the ``typing.NotRequired`` type qualifier is used to indicate that a variable declared in a TypedDict definition is a potentially-missing key: :: class Movie(TypedDict): # implicitly total=True title: str year: NotRequired[int] It is an error to use ``Required[...]`` or ``NotRequired[...]`` in any location that is not an item of a TypedDict. It is valid to use ``Required[...]`` and ``NotRequired[...]`` even for items where it is redundant, to enable additional explicitness if desired: :: class Movie(TypedDict): title: Required[str] # redundant year: NotRequired[int] Backwards Compatibility ======================= No backward incompatible changes are made by this PEP. How to Teach This ================= To define a TypedDict where most keys are required and some are potentially-missing, define a single TypedDict as normal and mark those few keys that are potentially-missing with ``NotRequired[...]``. To define a TypedDict where most keys are potentially-missing and a few are required, define a ``total=False`` TypedDict and mark those few keys that are required with ``Required[...]``. If some items accept ``None`` in addition to a regular value, it is recommended that the ``TYPE|None`` syntax be preferred over ``Optional[TYPE]`` for marking such item values, to avoid using ``Required[...]`` or ``NotRequired[...]`` alongside ``Optional[...]`` within the same TypedDict definition: Yes: :: from __future__ import annotations # for Python 3.7-3.9 class Dog(TypedDict): name: str owner: NotRequired[str|None] Avoid (unless Python 3.5-3.6): :: class Dog(TypedDict): name: str # ick; avoid using both Optional and NotRequired owner: NotRequired[Optional[str]] Reference Implementation ======================== The goal is to be able to make the following statement: The `mypy <http://www.mypy-lang.org/>`__ type checker supports ``Required`` and ``NotRequired``. A reference implementation of the runtime component is provided in the `typing_extensions <https://github.com/python/typing/tree/master/typing_extensions>`__ module. The mypy implementation is currently still being worked on. Rejected Ideas ============== Special syntax around the *key* of a TypedDict item --------------------------------------------------- :: class MyThing(TypedDict): opt1?: str # may not exist, but if exists, value is string opt2: Optional[str] # always exists, but may have null value or: :: class MyThing(TypedDict): Optional[opt1]: str # may not exist, but if exists, value is string opt2: Optional[str] # always exists, but may have null value These syntaxes would require Python grammar changes and it is not believed that marking TypedDict items as required or potentially-missing would meet the high bar required to make such grammar changes. Also, “let’s just not put funny syntax before the colon.” [1]_ Marking required or potentially-missing keys with an operator ------------------------------------------------------------- We could use unary ``+`` as shorthand to mark a required key, unary ``-`` to mark a potentially-missing key, or unary ``~`` to mark a key with opposite-of-normal totality: :: class MyThing(TypedDict, total=False): req1: +int # + means a required key, or Required[...] opt1: str req2: +float class MyThing(TypedDict): req1: int opt1: -str # - means a potentially-missing key, or NotRequired[...] req2: float class MyThing(TypedDict): req1: int opt1: ~str # ~ means a opposite-of-normal-totality key req2: float Such operators could be implemented on ``type`` via the ``__pos__``, ``__neg__`` and ``__invert__`` special methods without modifying the grammar. It was decided that it would be prudent to introduce longform syntax (i.e. ``Required[...]`` and ``NotRequired[...]``) before introducing any shortform syntax. Future PEPs may reconsider introducing this or other shortform syntax options. Marking absence of a value with a special constant -------------------------------------------------- We could introduce a new type-level constant which signals the absence of a value when used as a union member, similar to JavaScript’s ``undefined`` type, perhaps called ``Missing``: :: class MyThing(TypedDict): req1: int opt1: str|Missing req2: float Such a ``Missing`` constant could also be used for other scenarios such as the type of a variable which is only conditionally defined: :: class MyClass: attr: int|Missing def __init__(self, set_attr: bool) -> None: if set_attr: self.attr = 10 :: def foo(set_attr: bool) -> None: if set_attr: attr = 10 reveal_type(attr) # int|Missing Misalignment with how unions apply to values '''''''''''''''''''''''''''''''''''''''''''' However this use of ``...|Missing``, equivalent to ``Union[..., Missing]``, doesn’t align well with what a union normally means: ``Union[...]`` always describes the type of a *value* that is present. By contrast missingness or non-totality is a property of a *variable* instead. Current precedent for marking properties of a variable include ``Final[...]`` and ``ClassVar[...]``, which the proposal for ``Required[...]`` is aligned with. Misalignment with how unions are subdivided ''''''''''''''''''''''''''''''''''''''''''' Furthermore the use of ``Union[..., Missing]`` doesn’t align with the usual ways that union values are broken down: Normally you can eliminate components of a union type using ``isinstance`` checks: :: class Packet: data: Union[str, bytes] def send_data(packet: Packet) -> None: if isinstance(packet.data, str): reveal_type(packet.data) # str packet_bytes = packet.data.encode('utf-8') else: reveal_type(packet.data) # bytes packet_bytes = packet.data socket.send(packet_bytes) However if we were to allow ``Union[..., Missing]`` you’d either have to eliminate the ``Missing`` case with ``hasattr`` for object attributes: :: class Packet: data: Union[str, Missing] def send_data(packet: Packet) -> None: if hasattr(packet, 'data'): reveal_type(packet.data) # str packet_bytes = packet.data.encode('utf-8') else: reveal_type(packet.data) # Missing? error? packet_bytes = b'' socket.send(packet_bytes) or a check against ``locals()`` for local variables: :: def send_data(packet_data: Optional[str]) -> None: packet_bytes: Union[str, Missing] if packet_data is not None: packet_bytes = packet.data.encode('utf-8') if 'packet_bytes' in locals(): reveal_type(packet_bytes) # bytes socket.send(packet_bytes) else: reveal_type(packet_bytes) # Missing? error? or a check via other means, such as against ``globals()`` for global variables: :: warning: Union[str, Missing] import sys if sys.version_info < (3, 6): warning = 'Your version of Python is unsupported!' if 'warning' in globals(): reveal_type(warning) # str print(warning) else: reveal_type(warning) # Missing? error? Weird and inconsistent. ``Missing`` is not really a value at all; it’s an absence of definition and such an absence should be treated specially. Difficult to implement '''''''''''''''''''''' Eric Traut from the Pyright type checker team has stated that implementing a ``Union[..., Missing]``-style syntax would be difficult. [2]_ Introduces a second null-like value into Python ''''''''''''''''''''''''''''''''''''''''''''''' Defining a new ``Missing`` type-level constant would be very close to introducing a new ``Missing`` value-level constant at runtime, creating a second null-like runtime value in addition to ``None``. Having two different null-like constants in Python (``None`` and ``Missing``) would be confusing. Many newcomers to JavaScript already have difficulty distinguishing between its analogous constants ``null`` and ``undefined``. Replace Optional with Nullable. Repurpose Optional to mean “optional item”. --------------------------------------------------------------------------- ``Optional[...]`` is too ubiquitous to deprecate. Although use of it *may* fade over time in favor of the ``T|None`` syntax specified by `PEP 604 <https://www.python.org/dev/peps/pep-0604/>`__. Change Optional to mean “optional item” in certain contexts instead of “nullable” --------------------------------------------------------------------------------- Consider the use of a special flag on a TypedDict definition to alter the interpretation of ``Optional`` inside the TypedDict to mean “optional item” rather than its usual meaning of “nullable”: :: class MyThing(TypedDict, optional_as_missing=True): req1: int opt1: Optional[str] or: :: class MyThing(TypedDict, optional_as_nullable=False): req1: int opt1: Optional[str] This would add more confusion for users because it would mean that in *some* contexts the meaning of ``Optional[...]`` is different than in other contexts, and it would be easy to overlook the flag. Various synonyms for “potentially-missing item” ----------------------------------------------- - Omittable – too easy to confuse with optional - OptionalItem, OptionalKey – two words; too easy to confuse with optional - MayExist, MissingOk – two words - Droppable – too similar to Rust’s ``Drop``, which means something different - Potential – too vague - Open – sounds like applies to an entire structure rather then to an item - Excludable - Checked References ========== .. [1] https://mail.python.org/archives/list/typing-sig@python.org/message/4I3GPIW… .. [2] https://mail.python.org/archives/list/typing-sig@python.org/message/S2VJSVG… Copyright ========= This document is placed in the public domain or under the CC0-1.0-Universal license, whichever is more permissive. .. Local Variables: mode: indented-text indent-tabs-mode: nil sentence-end-double-space: t fill-column: 70 coding: utf-8 End: <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<

10 31

Re: Thoughts on a NotType?
by Anton Agestam 22 Feb '22

22 Feb '22

(Hoping I figured out how to reply in the thread now... ) Yes, sorry for the cryptic message but it looks like you got it right. The example isn't impressive but just like Steven is saying the value comes in larger applications. In a web application for instance only the parsing layer will have to deal with the runtime validation of the type. The business logic and database layers can use static type checking to make sure only the narrowed value is allowed. So it's kind of like a NewType plus an "instantiation predicate" (I'm failing to find the correct term, nothing is actually instantiated). I thought it's relevant here since I imagine a NotType would also have to rely a lot on runtime narrowing. If a downstream API takes Sequence & Not[str] and we have a Sequence available, we'd have to narrow it with something like assert not isinstance(..., str).

5 4

Opened and changed typing issues week 2022-W07
by Typing Bot 21 Feb '22

21 Feb '22

No issues or pull requests with the label 'topic: feature' were opened or updated last week in the typing repository on GitHub. All issues and pull requests with the label 'topic: feature' can be viewed under the following URL: https://github.com/python/typing/issues?q=label%3A%22topic%3A+feature%22

1 0

Should enum member aliases have the same type?
by martin＠push-f.com 19 Feb '22

19 Feb '22

Currently both mypy and pyright complain about the assignment in the following code: ```python import enum from typing import Literal class Foo(enum.Enum): A = 1 B = A _: Literal[Foo.A] = Foo.B ``` I would argue that since enum member aliases behave exactly alike they should have the same type, for example: ``` >>> Foo.B <Foo.A: 1> >>> str(Foo.B) 'Foo.A' >>> Foo.B.name 'A' ``` What do you think about this? - Martin

1 0

PEP 544 - Protocols subclassing normal classes
by Randolf Scholz 18 Feb '22

18 Feb '22

In the resolution of PEP 544 this idea was discarded in order to preserve transitivity of subtyping: https://www.python.org/dev/peps/pep-0544/#protocols-subclassing-normal-clas… However, the provided example is weird, because there is another seemingly straightforward and logical solution: `C` should not be a subtype of `Proto`! This behaviour would be useful, because it would allow to define Protocols that match any **subclass** of an existing class that also implement some extra feature. As a concrete, useful example this would allow for @runtime_checkable class namedtuple(Protocol, tuple): def _fields(self) -> tuple[str, ...]: ... which could be used to isinstance check for arbitrary namedtuples.

1 0

Debrief for Callable syntax (PEP 677)
by S Pradeep Kumar 16 Feb '22

16 Feb '22

Hello all, I know many of us have been privately discussing the PEP 677 (Callable syntax) rejection and what it might mean for future typing efforts. I wanted to start a thread so that we can pool our thoughts and converge on some lessons for the future. ("We" below means various people from typing-sig - Steven, Eric, Guido, me, and everybody else who participated.) Timeline: + May 2021: We started discussing callable syntax, among other syntax proposals, at the Typing Summit + June 2021: Steven kicked off a typing-sig thread about callable syntax [2] + September 2021: We all had a Zoom meetup about Callable syntax [3]. The main outcome was that, as a group, we dropped the def-style syntax proposal because it was too verbose. We didn't reach consensus about shorthand vs extended style. + early October 2021: We mailed python-dev to get preliminary feedback about new callable syntax [4]. There was no strong pushback at that point, as far as I can see. People pointed out edge cases such as trailing commas and proposed alternatives such as function-name-as-a-type. The Steering Council asked us to explicitly discuss the syntax changes. + November 2021: We had another heated Zoom discussion about shorthand vs extended syntax. [5] We still didn't reach consensus. Given that shorthand syntax was forward-compatible with extended syntax, Steven and I drafted a PEP with shorthand syntax, hoping to get it in before the deadline for 3.11. + December 2021: We put up the draft PEP in python-dev [6]. There was some strong pushback, especially in this thread [7] - "tl;dr: I find it very troubling that we are going on a path where we need to increase the language complexity (syntax) only in the cause 'easier' typing. So I am opposed to this change". + December 2021: We discovered some wrinkles in the proposal: readers could find it hard to distinguish the multiple `->`s in a function signature (this was point (4) in the SC rejection notice); readers could get confusing errors because of the relative binding of `->` and `|` [8]. + February 2022: The SC rejected PEP 677 [1]. The reasons were: (1) "A feature for use only in a fraction of type annotations ... did not feel like a strong enough reason to justify the complexity needed to parse this new syntax and provide meaningful error messages" (2) "While the current Callable[x, y] syntax is not loved, it does work." (3) "Each time we add syntax solely for typing it shifts us further in the direction of typing being its own mini-language so we aim to tread lightly in what gets added here." (4) "We did not like the visual and cognitive consequence of multiple `->` tokens in a def." + February 2022: SC clarification in that thread based on Shantanu's question: "If the 677 proposed syntax had been expanded to include more def features instead of being conservative I think it would've been an even easier rejection for many of us". My takeaways: 1. We could have tested the syntax in real-world code a lot earlier. That would have caught the readability issue with multiple arrows and the relative binding of `->` and `|`. This can be a lesson for future syntax changes. 2. It would have been good to hear the December python-dev pushback earlier. If I'd heard that in the early-October python-dev thread, I would have been much more pessimistic about the PEP's chances. Maybe instead of the informal mail in October, we could have put up a draft PEP right away. That might have gotten more eyeballs in python-dev. 3. The def vs shorthand discussion was useful and helped us decisively eliminate a proposal. 4. In retrospect, the shorthand vs extended discussion seems less useful. From the SC clarification above, it looks like it was always going to favor a simpler proposal over a more complicated proposal. Once we realized that we weren't going to reach consensus anytime soon, maybe we could have avoided the extended discussions (but maybe not). 5. It was a good call to defer the full CPython implementation until the SC verdict. That would have been a month or two of work that went nowhere. 6. Finally, on a positive note, this syntax rejection doesn't mean we can't have any typing-related Python language changes. For example, I'd be interested in seeing builtins for `mapping[str, int]` and `sequence[int]`, similar to `dict` and `list`. We could even consider `callable[[int, str], bool]`. If there's enough interest in such ideas, we could have a discussion session for this at the PyCon Typing Summit. Eric Traut had mentioned that he'd provisionally implemented callable syntax in Pyright's parser. With this rejection, I'm guessing he'd need to roll that back. Let me know if anyone has any objection. Overall, I think this PEP was still worth submitting. It got us valuable feedback from python-dev and the SC about future syntax changes. Special thanks to Steven Troxler, who put in a *lot* of work writing the PEP, understanding the CPython code, and implementing the syntax changes. Thanks also to everyone who participated in the vigorous debates, particularly Eric Traut, who presented alternative proposals, and Guido van Rossum, who provided much wisdom and guidance behind the scenes. What are your takeaways? What could we (typing-sig as a whole or individual people) have done differently? Also feel free to vent your disappointment, surprise, or worry :) -- S Pradeep Kumar References: [1]: https://mail.python.org/archives/list/python-dev@python.org/thread/NHCLHCU2… [2]: https://mail.python.org/archives/list/typing-sig@python.org/thread/3JNXLYH5… [3]: https://docs.google.com/document/d/17iqV7WWvB0IwA43EPlIqlUS6Xuvk08X3sEudAA-… [4]: https://mail.python.org/archives/list/python-dev@python.org/thread/VBHJOS3L… [5]: https://docs.google.com/document/d/17iqV7WWvB0IwA43EPlIqlUS6Xuvk08X3sEudAA-… [6]: https://mail.python.org/archives/list/python-dev@python.org/thread/OGACYN2X… [7]: https://mail.python.org/archives/list/python-dev@python.org/thread/FI4AFU3I… [8]: https://mail.python.org/archives/list/typing-sig@python.org/thread/DWIKV5XS…

5 4

Allow subclassing from Any at runtime
by Shantanu Jain 15 Feb '22

15 Feb '22

Currently subclassing from Any gives you `TypeError: Cannot subclass typing.Any` at runtime. You can currently work around this in .py with something like: ``` if TYPE_CHECKING: AnyBaseClass = Any else: AnyBaseClass = object class X(AnyBaseClass): ... ``` As far as I'm aware, type checkers already support this. We inherit from Any in typeshed in a couple places. The need to support Any inheritance also arises naturally as a result of gradual typing, e.g. if your base class comes from an untyped library or the import cannot be resolved. I've used this trick myself a couple times and have seen use cases for this a couple times on gitter.im. It's a neat way to silence many errors from classes that do super dynamic things or to allow for classes that can duck type anywhere (e.g. mocks).

4 5

PyCon 2022 Typing Summit Interest Form
by Shannon Zhu 14 Feb '22

14 Feb '22

Hi everyone, PyCon 2022 is just a few months out, and Alfonso and I are looking forward to organizing another Typing Summit this year! We plan to be in-person in Salt Lake City during PyCon US 2022, on Thursday April 28th from 1-5pm. For those who haven’t been before, the summit is a 3-4 hour event organized around a series of presentation and discussion topics related to the type system or type checkers. Some topics are presentation heavy, some have just a few slides and then focus most of the time on group discussions of details or related questions. You can find a recording with time stamps of last year’s (virtual) typing summit here<https://pyvideo.org/pycon-us-2021/summit-typing.html>. Please fill out the PyCon 2022 Typing Summit Interest Form<https://docs.google.com/forms/d/e/1FAIpQLSdsgrGXrKZUXv9KMBwyYJkJ6L3CL_Os4Yg…> if you are interested in attending, so we can get a sense of expected size and potential presentation topics! Let me or Alfonso know if you have any questions! :) Shannon

2 1

Phantom types via NewType?
by David Foster 14 Feb '22

14 Feb '22

Hi Anton, I was just reviewing your slides on "phantom types" from the Jan 13, 2022 Typing Meetup [1], and eyeballed your related phantom-types library [2]. It seems to me that a lightweight way to implement a phantom type would be to just use a NewType. For example: ``` IntStr = NewType('IntStr', str) def parse_int_str(s: str) -> Optional[IntStr]: try: int(s) except: return None else: return cast(IntStr, s) ``` Does this seem correct? Am I understanding the idea of a "phantom type" correctly? [1]: https://docs.google.com/document/d/17iqV7WWvB0IwA43EPlIqlUS6Xuvk08X3sEudAA-… [2]: https://phantom-types.readthedocs.io/en/stable/pages/getting-started.html -- David Foster | Seattle, WA, USA Contributor to TypedDict support for mypy

2 1

Opened and changed typing issues week 2022-W06
by Typing Bot 14 Feb '22

14 Feb '22

The following is an overview of all issues and pull requests in the typing repository on GitHub with the label 'topic: feature' that were opened or updated last week, excluding closed issues. --------------------------------------------------- The following issues and pull requests were updated last week: #593 Buffer protocol types opened by @srittau https://github.com/python/typing/issues/593 #685 Record Types opened by @saulshanabrook https://github.com/python/typing/issues/685 #735 Exhaustiveness checking util opened by @Phlogistique https://github.com/python/typing/issues/735 #920 Consider loosening restrictions of `Final` opened by @DetachHead https://github.com/python/typing/issues/920 --------------------------------------------------- All issues and pull requests with the label 'topic: feature' can be viewed under the following URL: https://github.com/python/typing/issues?q=label%3A%22topic%3A+feature%22

1 0

Release: typing_extensions 4.1.0
by Jelle Zijlstra 13 Feb '22

13 Feb '22

I just released typing_extensions 4.1.0, providing many of the new typing features that will be in Python 3.11: - Runtime support for PEP 646, adding `typing_extensions.TypeVarTuple` and `typing_extensions.Unpack`. - Add interaction of `Required` and `NotRequired` with `__required_keys__`, `__optional_keys__` and `get_type_hints()`. Patch by David Cabot (@d-k-bo). - Runtime support for PEP 675 and `typing_extensions.LiteralString`. - Add `Never` and `assert_never`. Backport from bpo-46475. - `ParamSpec` args and kwargs are now equal to themselves. Backport from bpo-46676. Patch by Gregory Beauregard (@GBeauregard). - Add `reveal_type`. Backport from bpo-46414. - Runtime support for PEP 681 and `typing_extensions.dataclass_transform`. - `Annotated` can now wrap `ClassVar` and `Final`. Backport from bpo-46491. Patch by Gregory Beauregard (@GBeauregard). - Add missed `Required` and `NotRequired` to `__all__`. Patch by Yuri Karabas (@uriyyo). - The `@final` decorator now sets the `__final__` attribute on the decorated object to allow runtime introspection. Backport from bpo-46342. - Add `is_typeddict`. Patch by Chris Moradi (@chrismoradi) and James Hilton-Balfe (@Gobot1234). Thank you to all the contributors to the PEPs and to the people who contributed to the typing_extensions implementation!

2 1

Tensor typing meeting: hiatus?
by Matthew Rahtz 10 Feb '22

10 Feb '22

Hey all, Given that we haven't had anything to talk about in the tensor typing meetings for a while, and given that Pradeep's excellent Typing Meetup is providing a place for other kinds of typing discussion, I'm wondering whether it might be a good time to take a hiatus for a while - resuming once we've made a bit more progress on the implementation side/have more conceptual issues to discuss again. Any objections? If we don't hear anything by the end of the week, we'll assume this is ok, and take a break for a while. Thanks! Matthew

2 1

PEP 677 callable syntax rejected
by Gregory P. Smith 10 Feb '22

10 Feb '22

apologies for not cc'ing typing-sig on that email, anyways see https://mail.python.org/archives/list/python-dev@python.org/thread/NHCLHCU2… for the reasoning. -gps

1 0

Proposal: typing.assert_never()
by Jelle Zijlstra 09 Feb '22

09 Feb '22

Another idea that came up in a few recent threads is a `typing.assert_never()` function, which indicates that some code is supposed to be unreachable. I will submit an implementation to CPython, and we can use this thread to work out any details (or to let me know that we shouldn't do this after all). # Specification We will add a new function `typing.assert_never()` with the following signature: def assert_never(arg: NoReturn, /) -> NoReturn: ... At runtime, this will unconditionally raise `RuntimeError(f"Unexpectedly reached {arg!r}")` (wording suggestions welcome). Type checkers should emit an error if the function is called with an argument that has a non-bottom type. Examples: match bool(): case True: pass case False: pass case _ as x: assert_never(x) # ok match int(): case 0: pass case 1: pass case _ as x: assert_never(x) # error: x is an int, not NoReturn # Motivation PEP 484 only allows NoReturn as a return type, but in practice all type checkers allow it in argument positions as a bottom type. Thus, a manually defined `assert_never()` function already has the behavior specified above. Adding the function to the standard library will make this behavior more discoverable for users. Relatedly, I opened https://bugs.python.org/issue46475 to document NoReturn being a general bottom type. # Open questions Suggestions welcome for better wording in the runtime error. There is a related proposal to rename NoReturn to Never. I'm lukewarm about that because renaming things always causes churn, but in any case let's discuss that separately.

6 8

Re: Proposal: typing.assert_never()
by Anton Agestam 08 Feb '22

08 Feb '22

@Jelle I really like this proposal, both the Never and the assert_never() parts. Some bike-shedding: there's an existing issue on the typing repository for this util, I added a comment there linking back to this thread and your PR. On that issue, it's proposed to call the helper assert_exhaustive(). I'd be happy with the tool in the stdlib either way, but I do feel like that name is better as it would be more likely to lead someone looking it up towards the concept of exhaustiveness checking.

1 0

Opened and changed typing issues week 2022-W05
by Typing Bot 07 Feb '22

07 Feb '22

The following is an overview of all issues and pull requests in the typing repository on GitHub with the label 'topic: feature' that were opened or updated last week, excluding closed issues. --------------------------------------------------- The following issues and pull requests were updated last week: #920 Consider loosening restrictions of `Final` opened by @DetachHead https://github.com/python/typing/issues/920 #1043 TypingError: type checker will emit error with a message specified in typing stubs opened by @Akuli https://github.com/python/typing/issues/1043 --------------------------------------------------- All issues and pull requests with the label 'topic: feature' can be viewed under the following URL: https://github.com/python/typing/issues?q=label%3A%22topic%3A+feature%22

1 0

Proposal: typing.reveal_locals()
by Jelle Zijlstra 06 Feb '22

06 Feb '22

In the previous thread about reveal_type(), several people proposed also adding `typing.reveal_locals()`. In this thread I'd like to iron out any details and make sure we're all in agreement. # Specification We will add a new function `typing.reveal_locals()` with the following signature: def reveal_locals() -> None: ... At runtime, this function will do nothing. When a type checker encounters a call to reveal_locals(), it will emit the types of all variables in the local scope. For example: def f() -> None: a = 1 b = "x" reveal_locals() When typechecked, this may produce: Revealed local types are: a: builtins.int b: builtins.str # Motivation `reveal_locals()` is already implemented in mypy ( https://mypy.readthedocs.io/en/stable/common_issues.html#displaying-the-typ…) and pyright (undocumented). It is not as commonly used as reveal_type(), but can be useful for debugging type checker behavior. # Open questions What should the runtime behavior be? We could do something like sys._getframe(1).f_locals to get the calling function's locals and print them, but that isn't code I'd like to write.

7 17

StrictTypeGuard and TypeAssert
by Eric Traut 05 Feb '22

05 Feb '22

In a previous typing-sig thread (https://mail.python.org/archives/list/typing-sig@python.org/thread/EMUD2D42…) there was a discussion about the existing TypeGuard (PEP 647) and the desire to support a form that allows for type narrowing in the negative case. I previously proposed extending the existing TypeGuard to support an optional second type argument that represents the negative type. This proposal received lukewarm reception because it didn't address some desired use cases. Based on this feedback, I proposed a new idea in the python/typing discussion forum (https://github.com/python/typing/discussions/1013#discussioncomment-1966238). This new proposal introduces a form called "StrictTypeGuard". It also describes variants called "TypeAssert" and "StrictTypeAssert". I haven't received much feedback in that forum thread, so I figured I'd repost here. -- Taking in all of the feedback, here's an alternative proposal that involves the introduction of another form of TypeGuard which has "strict" type narrowing semantics. It would be less flexible than the existing TypeGuard, but the added constraints would allow it to be used for these alternate use cases. *StrictTypeGuard* This proposal would introduce a new flavor of TypeGuard that would have strict narrowing semantics. We'd need to come up with a new name for this, perhaps StrictTypeGuard, ExactTypeGuard, or PreciseTypeGuard. Other suggestions are welcome. For now, I'll refer to it as StrictTypeGuard. This new flavor of type guard would be similar to the more flexible version defined in PEP 647 with the following three differences: The type checker would enforce the requirement that the type guard type (specified as a return type of the type guard function) is a subtype of the input type (the declared type of the first parameter to the type guard function). In other words, the type guard type must be strictly narrower than the input type. This precludes some of the use cases anticipated in the original PEP 647. ```python def is_marsupial(val: Animal) -> StrictTypeGuard[Kangaroo | Koala]: # This is allowed return isinstance(val, Kangaroo | Koala) def has_no_nones(val: list[T | None]) -> StrictTypeGuard[list[T]]: # Error: "list[T]" cannot be assigned to "list[T | None]" return None not in val ``` Type narrowing would be applied in the negative ("else") case. This may still lead to incorrect assumptions, but it's less likely to be incorrect with restriction 1 in place. Consider, for example, ```python def is_black_cat(val: Animal) -> StrictTypeGuard[Cat]: return isinstance(val, Cat) and val.color == Color.Black def func(val: Cat | Dog): if is_black_cat(val): reveal_type(val) # Cat else: reveal_type(val) # Dog - which is potentially wrong here ``` If the return type of the type guard function includes a union, the type checker would apply additional type narrowing based on the type of the argument passed to the type guard call, eliminating union elements that are impossible given the argument type. For example: ```python def is_cardinal_direction(val: str) -> StrictTypeGuard[Literal["N", "S", "E", "W"]]: return val in ("N", "S", "E", "W") def func(direction: Literal["NW", "E"]): if is_cardinal_direction(direction): reveal_type(direction) # Literal["E"] # The type cannot be "N", "S" or "W" here because of argument type else: reveal_type(direction) # Literal["NW"] ``` * TypeAssert and StrictTypeAssert* As mentioned above, there has also been a desire to support a "type assert" function. This is similar to a "type guard" function except that it raises an exception (similar to an assert statement, except that its behavior is not dependent on 'debug' mode) if the input's type doesn't match the declared return type. This is analogous to "type predicate functions" supported in TypeScript, like the following: ```typescript function assertAuthenticated(user: User | null): asserts user is User { if (user === null) { throw new Error('Unauthenticated'); } } ``` We propose to add two new forms TypeAssert and StrictTypeAssert, which would be analogous to TypeGuard and StrictTypeGuard. While type guard functions are expected to return a bool value, "type assert" forms would return None (if the input type matches) or raise an exception (if the input type doesn't match). Type narrowing in the negative ("else") case doesn't apply for type assertions because it is assumed that an exception is raised in this event. Here are some examples: ```python def verify_no_nones(val: list[None | T]) -> TypeAssert[list[T]]: if None in val: raise ValueError() def func(x: list[int | None]): verify_no_nones(x) reveal_type(x) # list[int] and def assert_is_one_dimensional(val: tuple[T, ...] | T) -> StrictTypeAssert[tuple[T] | T]: if isinstance(val, tuple) and len(val) != 1: raise ValueError() def func(x: float, y: tuple[float, ...]): assert_is_one_dimensional(x) reveal_type(x) # float assert_is_one_dimensional(y) reveal_type(y) # tuple[float] ``` Thoughts? Suggestions? If we move forward with the above proposal (or some subset thereof), it will probably require a new PEP, as opposed to a modification to PEP 647. - Eric -- Eric Traut Contributor to Pyright & Pylance Microsoft

4 5

RFC on Arbitrary Literal Strings PEP
by S Pradeep Kumar 03 Feb '22

03 Feb '22

Hello all, We've drafted a PEP to support arbitrary literal string types [1]. Feedback appreciated! Scala has a very similar concept. See this talk on using Scala's literal string types for "Preventing SQL injection at compile time" [2] Bikeshedding: We're open to any name that seems easy to understand :) -- Best, S Pradeep Kumar Graham Bleaney [1]: https://docs.google.com/document/d/1fbAbA2MCoAcSO1c8gecmUJYZ-Q0utT3CmaXlQ3I… [2]: https://slideslive.com/38907881/literal-types-what-they-are-good-for (slides 52 to 68) [3]: Previous typing-sig thread: https://mail.python.org/archives/list/typing-sig@python.org/thread/JJWAA5AH…

14 24

Adding `typing.reveal_type`
by Jelle Zijlstra 02 Feb '22

02 Feb '22

I just opened https://bugs.python.org/issue46414 to propose adding `typing.reveal_type` to Python 3.11. Copying my message there: --- The `reveal_type()` primitive is injected by type checkers into the builtins. When the type checker sees a call, it prints the inferred type of the argument. This has been implemented across all type checkers, but adding an implementation to `typing` would help document the behavior and make it more discoverable for users. Also, it means code with `reveal_type()` calls can run without runtime errors, useful if you want to run your tests at the same time as you're debugging a typing issue. The runtime implementation can be very simple: def reveal_type(obj: _T, /) -> _T: print("Runtime type is", type(obj)) return obj reveal_type() is supported by all type checkers that I'm aware of (docs include https://google.github.io/pytype/faq.html#can-i-find-out-what-pytype-thinks-… for pytype and https://mypy.readthedocs.io/en/stable/common_issues.html#reveal-type for mypy). One area of divergence is the return value. Pyright returns the inferred type of the expression as a string (and uses that in its test suite for testing type inference). Mypy returns the argument, which has the advantage that you can insert `reveal_type()` in the middle of an expression without having to put it on its own line. Also, the Pyright behavior cannot sensibly be implemented at runtime. Therefore, I suggest using Mypy's behavior for `typing.reveal_type`. ---- I'm hoping this will be simple and noncontroversial enough that we can add it quickly, but if there are concerns from any type checker authors, I'd love to hear them.

9 23