Mailman 3 October 2021 - Typing-sig

Re: Typing Meetup
by S Pradeep Kumar Sept. 19, 2023

Sept. 19, 2023

Updating the Typing Meetup date to Wednesday as there were some conflicts. Let me know if there are any major time conflicts. Date: *Wednesday, November 03, 2021.*Time: 10 am San Francisco time (UTC-7) / 5 pm London time (UTC) Duration: 1 hour Potential discussion topics: + Typing **kwargs using a TypedDict (Franek Magiera?, Pradeep) [1] + Remaining questions about Callable syntax (Steven) + Non-required Protocol fields (Sebastian Rittau?) [2] + Type support for sealed classes (Adrian Freund?) [3] + Safe, typed JSON validation + Type system documentation (Shannon?) If you're working on these or any other topics and would like to start a discussion at the meetup, reach out to me. If you have interesting topics you'd like to see discussed, share them too. Once we have topics, we can always start a discussion even if there's no proposer yet. We will probably have time for 3-4 topics in this session. On Fri, Oct 15, 2021 at 12:55 PM S Pradeep Kumar <gohanpra(a)gmail.com> wrote: >>> >>>> Hello all, >>>> >>>> We plan to hold a virtual Typing Meetup to discuss key typing-related >>>> issues in real-time. The idea is that we can make progress on important or >>>> controversial discussions, get early feedback on in-progress PEPs, and >>>> gauge interest in new proposals. >>>> >>>> A virtual meetup will be more accessible to people in other timezones >>>> than Bay Area in-person meetups. It should also help consolidate >>>> discussions that are often spread out across mailing lists or Github repos. >>>> The tentative plan is to have a monthly meetup, but we can change that as >>>> needed. >>>> >>>> Date: Monday, November 01, 2021. >>>> Time: 10 am San Francisco time (UTC-7) / 5 pm London time (UTC) >>>> Duration: 1 hour >>>> >>>> Time and date are flexible. I'll share the zoom link and agenda the >>>> week before. I'll also record the meetup and share afterwards. >>>> >>>> If you're interested in presenting a new proposal or your work on some >>>> existing proposal, please respond here. We can shoot for 15 minutes per >>>> presentation. I know there are quite a few interesting threads on >>>> typing-sig that haven't seen much activity in a while :) >>>> -- >>>> S Pradeep Kumar >>>> _______________________________________________ >>>> Typing-sig mailing list -- typing-sig(a)python.org >>>> To unsubscribe send an email to typing-sig-leave(a)python.org >>>> https://mail.python.org/mailman3/lists/typing-sig.python.org/ >>>> Member address: guido(a)python.org >>> >>> Links: [1] Typing **kwargs using TypedDict: https://mail.python.org/archives/list/typing-sig@python.org/thread/IUCXT2IQ… [2] Non-required Protocol fields: https://mail.python.org/archives/list/typing-sig@python.org/thread/P6FCGRJU… [3] Sealed classes: https://mail.python.org/archives/list/typing-sig@python.org/thread/QSCT2N4R… -- S Pradeep Kumar

10 60

Any way to express that a dict is expected to have a key(s) but don't care about other keys?
by Brett Cannon June 19, 2023

June 19, 2023

Looking at TypedDict it seems that it does not like the idea of superfluous keys existing in a dict. In my case I have a REST call returning JSON for an error and all I'm guaranteed is to get an "error" key. But there may be extra keys that provide more details about the specific error. Unfortunately I don't know the exhaustive list of keys so I can't use a TypedDict with `totality == False` to try to be complete. I can't think of any way to somehow say "require/expect this one key, but other keys are OK". Am I missing anything that would let me do this?

17 35

Runtime Access to Type Parameters
by Saul Shanabrook June 7, 2023

June 7, 2023

I would like to propose being able to access, at runtime, the type parameters in classmethods and in constructors. I wrote up a draft PEP on this to capture the thinking here: https://github.com/saulshanabrook/peps/blob/master/pep-9999.md It seems easier to add them to classmethods, so even that addition alone would be very welcome! It could be added to Guido's existing draft PR for pep 585, by making a small change in the getattr function (https://github.com/python/cpython/pull/18239/files#diff-a34ae826f869897e56e…), to first do a `__getattribute__` and check if the result is a descriptor (has a `__get__` attribute), and if so passing the generic class itself in as the class, instead of the origin class. I wrote up a little example of that in the PEP. To make it available in the constructor, I proposed adding a global function `get_parameters` to `typing` which returns the parameters in the constructor. I also provide a little mock implementation of that. Note that work is only for custom classes which inherit from `typing.Generic` not for any built in collections.

5 6

Explicit Type Aliases
by Shannon Zhu Aug. 17, 2022

Aug. 17, 2022

Hi everyone, I’d like to continue the discussion from our last typing summit on explicit type aliases. A quick summary of the current state and the proposal -- Current state: ``` from typing import List x = List[int] # considered a type alias y : Type[List[int]] = List[int] # considered an expression z : Any # considered a type alias z : x = [] # fine z : y = [] # invalid type error ``` Proposal (with explicit aliases): ``` from typing import List, TypeAlias x = TypeAlias[List[int]] # considered a type alias y = List[int] # considered an expression z : Any # considered an expression z = TypeAlias[Any] # considered a type alias reveal_type(x) # Type[List[int]] reveal_type(y) # Any (return type of __getitem__) z : x = [] # fine z : y = [] # invalid type error ``` Some of the benefits we’re hoping to gain from explicit type aliases: * Clearly distinguish between an unannotated global assignment and a type alias, especially when parsing forward-referencing string annotations. * Avoid the confusing case of type aliases in which some part of the type is invalid or undefined. This would facilitate warnings on invalid types at the alias definition rather than later on when the alias is used. * Make valid and invalid types more intuitive to Python programmers by shifting from delineation of non-aliases with a meta annotation toward delineation of aliases with `TypeAlias`. * Remove special treatment of values annotated as `Any`, which currently break all type aliasing rules. This will also allow us to clean up some of the distinctions we maintain between unannotated values and explicit Anys. Note: We also considered denoting the alias as an annotation (e.g. `x: TypeAlias = int`) as an alternative to the syntax above. I’m interested to hear your thoughts and suggestions on making type aliasing explicit going forward – we’d like to start implementing a version of this in Pyre soon! Shannon

7 10

PEP 655: Marking individual TypedDict items as required or potentially-missing
by David Foster Feb. 23, 2022

Feb. 23, 2022

The new syntax Required[...] and NotRequired[...] for marking individual keys of a TypedDict has now been formally proposed as a draft PEP. Please see the latest text at the bottom of this email, or online at: https://www.python.org/dev/peps/pep-0655/ Comments welcome. :) Meanwhile I'll proceed to implement this syntax in mypy. -- David Foster | Seattle, WA, USA Contributor to TypedDict support for mypy >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> PEP: 655 Title: Marking individual TypedDict items as required or potentially-missing Author: David Foster <david at dafoster.net> Sponsor: Guido van Rossum <guido at python.org> Discussions-To: typing-sig at python.org Status: Draft Type: Standards Track Content-Type: text/x-rst Requires: 604 Created: 30-Jan-2021 Python-Version: 3.10 Post-History: 31-Jan-2021, 11-Feb-2021, 20-Feb-2021, 26-Feb-2021 Abstract ======== `PEP 589 <https://www.python.org/dev/peps/pep-0589/>`__ defines syntax for declaring a TypedDict with all required keys and syntax for defining a TypedDict with `all potentially-missing keys <https://www.python.org/dev/peps/pep-0589/#totality>`__ however it does not provide any syntax to declare some keys as required and others as potentially-missing. This PEP introduces two new syntaxes: ``Required[...]`` which can be used on individual items of a TypedDict to mark them as required, and ``NotRequired[...]`` which can be used on individual items to mark them as potentially-missing. Motivation ========== It is not uncommon to want to define a TypedDict with some keys that are required and others that are potentially-missing. Currently the only way to define such a TypedDict is to declare one TypedDict with one value for ``total`` and then inherit it from another TypedDict with a different value for ``total``: :: class _MovieBase(TypedDict): # implicitly total=True title: str class Movie(_MovieBase, total=False): year: int Having to declare two different TypedDict types for this purpose is cumbersome. Rationale ========= One might think it unusual to propose syntax that prioritizes marking *required* keys rather than syntax for *potentially-missing* keys, as is customary in other languages like TypeScript: :: interface Movie { title: string; year?: number; // ? marks potentially-missing keys } The difficulty is that the best word for marking a potentially-missing key, ``Optional[...]``, is already used in Python for a completely different purpose: marking values that could be either of a particular type or ``None``. In particular the following does not work: :: class Movie(TypedDict): ... year: Optional[int] # means int|None, not potentially-missing! Attempting to use any synonym of “optional” to mark potentially-missing keys (like ``Missing[...]``) would be too similar to ``Optional[...]`` and be easy to confuse with it. Thus it was decided to focus on positive-form phrasing for required keys instead, which is straightforward to spell as ``Required[...]``. Nevertheless it is common for folks wanting to extend a regular (``total=True``) TypedDict to only want to add a small number of potentially-missing keys, which necessitates a way to mark keys that are *not* required and potentially-missing, and so we also allow the ``NotRequired[...]`` form for that case. Specification ============= The ``typing.Required`` type qualifier is used to indicate that a variable declared in a TypedDict definition is a required key: :: class Movie(TypedDict, total=False): title: Required[str] year: int Additionally the ``typing.NotRequired`` type qualifier is used to indicate that a variable declared in a TypedDict definition is a potentially-missing key: :: class Movie(TypedDict): # implicitly total=True title: str year: NotRequired[int] It is an error to use ``Required[...]`` or ``NotRequired[...]`` in any location that is not an item of a TypedDict. It is valid to use ``Required[...]`` and ``NotRequired[...]`` even for items where it is redundant, to enable additional explicitness if desired: :: class Movie(TypedDict): title: Required[str] # redundant year: NotRequired[int] Backwards Compatibility ======================= No backward incompatible changes are made by this PEP. How to Teach This ================= To define a TypedDict where most keys are required and some are potentially-missing, define a single TypedDict as normal and mark those few keys that are potentially-missing with ``NotRequired[...]``. To define a TypedDict where most keys are potentially-missing and a few are required, define a ``total=False`` TypedDict and mark those few keys that are required with ``Required[...]``. If some items accept ``None`` in addition to a regular value, it is recommended that the ``TYPE|None`` syntax be preferred over ``Optional[TYPE]`` for marking such item values, to avoid using ``Required[...]`` or ``NotRequired[...]`` alongside ``Optional[...]`` within the same TypedDict definition: Yes: :: from __future__ import annotations # for Python 3.7-3.9 class Dog(TypedDict): name: str owner: NotRequired[str|None] Avoid (unless Python 3.5-3.6): :: class Dog(TypedDict): name: str # ick; avoid using both Optional and NotRequired owner: NotRequired[Optional[str]] Reference Implementation ======================== The goal is to be able to make the following statement: The `mypy <http://www.mypy-lang.org/>`__ type checker supports ``Required`` and ``NotRequired``. A reference implementation of the runtime component is provided in the `typing_extensions <https://github.com/python/typing/tree/master/typing_extensions>`__ module. The mypy implementation is currently still being worked on. Rejected Ideas ============== Special syntax around the *key* of a TypedDict item --------------------------------------------------- :: class MyThing(TypedDict): opt1?: str # may not exist, but if exists, value is string opt2: Optional[str] # always exists, but may have null value or: :: class MyThing(TypedDict): Optional[opt1]: str # may not exist, but if exists, value is string opt2: Optional[str] # always exists, but may have null value These syntaxes would require Python grammar changes and it is not believed that marking TypedDict items as required or potentially-missing would meet the high bar required to make such grammar changes. Also, “let’s just not put funny syntax before the colon.” [1]_ Marking required or potentially-missing keys with an operator ------------------------------------------------------------- We could use unary ``+`` as shorthand to mark a required key, unary ``-`` to mark a potentially-missing key, or unary ``~`` to mark a key with opposite-of-normal totality: :: class MyThing(TypedDict, total=False): req1: +int # + means a required key, or Required[...] opt1: str req2: +float class MyThing(TypedDict): req1: int opt1: -str # - means a potentially-missing key, or NotRequired[...] req2: float class MyThing(TypedDict): req1: int opt1: ~str # ~ means a opposite-of-normal-totality key req2: float Such operators could be implemented on ``type`` via the ``__pos__``, ``__neg__`` and ``__invert__`` special methods without modifying the grammar. It was decided that it would be prudent to introduce longform syntax (i.e. ``Required[...]`` and ``NotRequired[...]``) before introducing any shortform syntax. Future PEPs may reconsider introducing this or other shortform syntax options. Marking absence of a value with a special constant -------------------------------------------------- We could introduce a new type-level constant which signals the absence of a value when used as a union member, similar to JavaScript’s ``undefined`` type, perhaps called ``Missing``: :: class MyThing(TypedDict): req1: int opt1: str|Missing req2: float Such a ``Missing`` constant could also be used for other scenarios such as the type of a variable which is only conditionally defined: :: class MyClass: attr: int|Missing def __init__(self, set_attr: bool) -> None: if set_attr: self.attr = 10 :: def foo(set_attr: bool) -> None: if set_attr: attr = 10 reveal_type(attr) # int|Missing Misalignment with how unions apply to values '''''''''''''''''''''''''''''''''''''''''''' However this use of ``...|Missing``, equivalent to ``Union[..., Missing]``, doesn’t align well with what a union normally means: ``Union[...]`` always describes the type of a *value* that is present. By contrast missingness or non-totality is a property of a *variable* instead. Current precedent for marking properties of a variable include ``Final[...]`` and ``ClassVar[...]``, which the proposal for ``Required[...]`` is aligned with. Misalignment with how unions are subdivided ''''''''''''''''''''''''''''''''''''''''''' Furthermore the use of ``Union[..., Missing]`` doesn’t align with the usual ways that union values are broken down: Normally you can eliminate components of a union type using ``isinstance`` checks: :: class Packet: data: Union[str, bytes] def send_data(packet: Packet) -> None: if isinstance(packet.data, str): reveal_type(packet.data) # str packet_bytes = packet.data.encode('utf-8') else: reveal_type(packet.data) # bytes packet_bytes = packet.data socket.send(packet_bytes) However if we were to allow ``Union[..., Missing]`` you’d either have to eliminate the ``Missing`` case with ``hasattr`` for object attributes: :: class Packet: data: Union[str, Missing] def send_data(packet: Packet) -> None: if hasattr(packet, 'data'): reveal_type(packet.data) # str packet_bytes = packet.data.encode('utf-8') else: reveal_type(packet.data) # Missing? error? packet_bytes = b'' socket.send(packet_bytes) or a check against ``locals()`` for local variables: :: def send_data(packet_data: Optional[str]) -> None: packet_bytes: Union[str, Missing] if packet_data is not None: packet_bytes = packet.data.encode('utf-8') if 'packet_bytes' in locals(): reveal_type(packet_bytes) # bytes socket.send(packet_bytes) else: reveal_type(packet_bytes) # Missing? error? or a check via other means, such as against ``globals()`` for global variables: :: warning: Union[str, Missing] import sys if sys.version_info < (3, 6): warning = 'Your version of Python is unsupported!' if 'warning' in globals(): reveal_type(warning) # str print(warning) else: reveal_type(warning) # Missing? error? Weird and inconsistent. ``Missing`` is not really a value at all; it’s an absence of definition and such an absence should be treated specially. Difficult to implement '''''''''''''''''''''' Eric Traut from the Pyright type checker team has stated that implementing a ``Union[..., Missing]``-style syntax would be difficult. [2]_ Introduces a second null-like value into Python ''''''''''''''''''''''''''''''''''''''''''''''' Defining a new ``Missing`` type-level constant would be very close to introducing a new ``Missing`` value-level constant at runtime, creating a second null-like runtime value in addition to ``None``. Having two different null-like constants in Python (``None`` and ``Missing``) would be confusing. Many newcomers to JavaScript already have difficulty distinguishing between its analogous constants ``null`` and ``undefined``. Replace Optional with Nullable. Repurpose Optional to mean “optional item”. --------------------------------------------------------------------------- ``Optional[...]`` is too ubiquitous to deprecate. Although use of it *may* fade over time in favor of the ``T|None`` syntax specified by `PEP 604 <https://www.python.org/dev/peps/pep-0604/>`__. Change Optional to mean “optional item” in certain contexts instead of “nullable” --------------------------------------------------------------------------------- Consider the use of a special flag on a TypedDict definition to alter the interpretation of ``Optional`` inside the TypedDict to mean “optional item” rather than its usual meaning of “nullable”: :: class MyThing(TypedDict, optional_as_missing=True): req1: int opt1: Optional[str] or: :: class MyThing(TypedDict, optional_as_nullable=False): req1: int opt1: Optional[str] This would add more confusion for users because it would mean that in *some* contexts the meaning of ``Optional[...]`` is different than in other contexts, and it would be easy to overlook the flag. Various synonyms for “potentially-missing item” ----------------------------------------------- - Omittable – too easy to confuse with optional - OptionalItem, OptionalKey – two words; too easy to confuse with optional - MayExist, MissingOk – two words - Droppable – too similar to Rust’s ``Drop``, which means something different - Potential – too vague - Open – sounds like applies to an entire structure rather then to an item - Excludable - Checked References ========== .. [1] https://mail.python.org/archives/list/typing-sig@python.org/message/4I3GPIW… .. [2] https://mail.python.org/archives/list/typing-sig@python.org/message/S2VJSVG… Copyright ========= This document is placed in the public domain or under the CC0-1.0-Universal license, whichever is more permissive. .. Local Variables: mode: indented-text indent-tabs-mode: nil sentence-end-double-space: t fill-column: 70 coding: utf-8 End: <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<

10 31

PEP 613 and type aliases in nested scope
by Shantanu Jain Nov. 20, 2021

Nov. 20, 2021

Hello everybody :-) Linking https://www.python.org/dev/peps/pep-0613/#scope-restrictions PEP 613 appears to disallow explicit type aliases in nested scopes: > With explicit aliases, [...] the inner assignment can either be a valid local variable or a clear error [...] It also maybe seem to not allow for the possibility of type aliases in nested scope at all: > [...] because type aliases cannot be defined inside a nested scope If this was the intention of the PEP, this restriction feels unnecessary to me. Type aliases in nested scope are a valid use case. mypy previously never considered assignments in nested scopes to be type aliases, and we've had multiple issues concerning this. Eric mentioned that as a result of this wording, Pyright errors for explicit type aliases in nested scope, but allows implicit type aliases. I propose we loosen the wording in PEP 613. If type checkers don't wish to add support for type aliases in nested scope that seems fine, but I see no reason to prohibit this via PEP. The following code seems very clear to me in its intention (as a result of PEP 613!) and type checkers should be free to follow that intention: ``` import typing T = typing.NewType('T', int) class C: V: typing.TypeAlias = T x: C.V = C.V(5) ``` Conversely, if this proves controversial and we decide we do actually want to prohibit type aliases in nested scopes, we should probably amend PEP 484 to mention this. Thanks to Nipunn for bringing this up!

5 12

Thoughts on a typing.Self?
by gobot1234yt＠gmail.com Nov. 2, 2021

Nov. 2, 2021

I would like to draft a PEP for a typing.Self class which would perform similarly to rust's Self keyword in function annotations. Examples ```python from typing import TypeVar F = TypeVar("F", bound="Foo") class Foo: def parse(self: F, data: bytes) -> F: ... return self ``` vs ```python from typing import Self class Foo: def parse(self, data: bytes) -> Self: ... return self ``` Another example of where this could be useful is classmethods: ```python class Foo: @classmethod def bar(cls: type[F], *args: Any, **kwargs: Any) -> F: return cls() ``` vs ```python class Foo: @classmethod def bar(cls, *args: Any, **kwargs: Any) -> Self: return cls() ``` Implementation wise it would be a copy of NoReturn, i.e. not subscriptable, a _SpecialForm and should only be valid as a return type. I haven't seen any discussion of the idea on the mailing list, so I wanted to run it by everyone. Thanks, James.

12 27

Syntax for TypeVars with bounds
by Jelle Zijlstra Nov. 1, 2021

Nov. 1, 2021

I was thinking about Joren Hammudoglu's proposed pep adding `+T`/`-T` syntax for TypeVar variance and had a crazy idea: what if we had syntactic support for other kinds of TypeVar customization too? For bounds we could overload the <= operator: from typing import TypeVar, SupportsAbs T = TypeVar("T") def largest_in_absolute_value(*xs: T <= SupportsAbs[float]) -> T: return max(xs, key=abs) And for value restrictions we could use `in`: def concat(x: T in (str, bytes), y: T) -> T: return x + y (Examples taken from https://mypy.readthedocs.io/en/stable/generics.html) The first use of the TypeVar in the function definition would have the bound or constraints, and other uses would then follow the constraint set in the first use. Using in or <= on the same TypeVar more than once in a function definition is an error. The nice thing about this syntax is that it puts all the information about the function definition in one place. You no longer have to create a named (and usually awkwardly named) TypeVar for each possible bound. The syntax is reminiscent of that used in languages like Scala and TypeScript, although in those you would write something like `def largest_in_absolute_value<T <= SupportsAbs[float]>(*xs: T) -> T:`, which would require new syntax in Python. I'm curious if other people think this would be a useful enhancement to the language.

9 13

Self type PEP
by S Pradeep Kumar Nov. 1, 2021

Nov. 1, 2021

Hello everyone, We've drafted a PEP for a `Self` type. This provides a painless, readable way to annotate methods and classmethods that return `self` or `cls(...)` respectively. We found that this pattern comes up a lot in real-world code; for example, typeshed uses this around 40% as often as popular types such as Dict or Callable. Please share your feedback in the Google Doc: https://docs.google.com/document/d/1ujuSMXDmSIOJpiZyV7mvBEC8P-y55AgSzXcvhrZ… Best, S Pradeep Kumar James Hilton-Balfe Original thread: https://mail.python.org/archives/list/typing-sig@python.org/thread/SJAANGA2…

2 1

repr() of callable type
by Guido van Rossum Oct. 29, 2021

Oct. 29, 2021

Regardless of whether we pick shorthand, longhand or hybrid form, we need to decide on what the repr() of such an object would be. E.g. in an interactive session, if I print a callable type object, what will it print? >>> a = (int, int) -> int >>> a (int, int) - int >>> or perhaps >>> a (__0: int, __1: int, /) -> int >>> or maybe >>> a callable[[int, int], int] >>> or >>> a <callable: (int, int) -> int> >>> ??? -- --Guido van Rossum (python.org/~guido) *Pronouns: he/him **(why is my pronoun here?)* <http://feministing.com/2015/02/03/how-using-they-as-a-singular-pronoun-can-…>

4 7

PEP 646: Type suffixes of *args
by Matthew Rahtz Oct. 25, 2021

Oct. 25, 2021

In the last tensor typing meeting we realised there's something missing from PEP 646: the ability to split off suffixes from *args. (Thanks for Pradeep for pointing this out, based on discussions with Eric Traut and this question on python-dev https://mail.python.org/archives/list/python-dev@python.org/thread/K76H6C5J… ) *Intro* To recap, PEP 646 allows us to bind the types of *args into a TypeVarTuple: Ts = TypeVarTuple('Ts') def foo(*args: *Ts): ... foo(1, '2') # Ts is bound to Tuple[int, str] But what if we wanted to do something like the following? def foo( *args: [types of all positional args but the last bound to Ts; type of last positional arg should be int] ) -> Tuple[*Ts]: ... x = foo(True, 1.0, 2) # x should have type Tuple[bool, float] *The problem* You might think the answer is: def foo( *args: *Ts, final_arg: int, ) -> Tuple[*Ts]: ... But note that this means something different; all arguments after *args must be specified as keywords: > foo(True, 1.0, 2) TypeError: foo() missing 1 required keyword-only argument: 'final_arg' Note that this is an only an issue for suffixes of variadic argument lists; prefixes are fine: def foo( first_arg: int, *args: *Ts, ) -> Tuple[*Ts]: ... *Proposed solution* We think the way of encoding what we want most consistent with the rest of PEP 646 would be something like: def foo( *args: *Tuple[*Ts, int], ) -> Tuple[*Ts]: ... Why? Well, when we say `*args: *Ts`, we're saying "Put all the types of `args` into the thing that comes after the starred part of the annotation". So here, we're saying "Put all the types of `args` into Tuple[*Ts, int]" - that is, doing some pattern matching, "Put all types but the last into `Ts`". *Use cases?* This is (for me) an unintuitive enough construction that I think our first question should be, does this have plausible enough use cases to support the complexity? The only one I can think off of the top of my head is something like: def log_with_logger(*args: *Tuple[*Ts, Logger]): ... log_with_logger('foo', 'bar', logger_inst) But here the issue could be worked around by just moving the position of the logger argument: def log_with_logger(logger: Logger, *args: *Ts): ... So I'm not sure I really believe this example. Can anyone else think of any use cases for this? --- Thanks! Matthew

3 5

Steering Council message
by Guido van Rossum Oct. 20, 2021

Oct. 20, 2021

This message from the SC to python-dev seems important to this community: https://mail.python.org/archives/list/python-dev@python.org/message/SZLWVYV… -- --Guido van Rossum (python.org/~guido) *Pronouns: he/him **(why is my pronoun here?)* <http://feministing.com/2015/02/03/how-using-they-as-a-singular-pronoun-can-…>

1 0

Static type checking for sentinel values (PEP 661)
by Tal Einat Oct. 19, 2021

Oct. 19, 2021

Hi typing folks, I'm continuing to pursue adding a tool for defining sentinel values to the standard library. See PEP 616 [1], previous discussion [2] and previous thread on this mailing list [3], for more background and details. The gist of the proposal is to add a function or class, e.g. Sentinel(), for defining sentinels in a standard way, providing nice properties such as clear reprs, support for strict type signatures and surviving copying/unpickling. Old: MISSING = object() New: MISSING = Sentinel("MISSING") Besides allowing use in the stdlib, having this be defined in a PEP and included in the stdlib means that we can agree how static type analysis would handle such sentinels. So here are some questions, where I especially would like input from people implementing static type analyzers: 1. The PEP currently suggests using Literal[MISSING] in type signatures. Is this reasonable? What would this require of static type analyzer authors and maintainers? 2. What would the type signature used during definition look like? I couldn't be MISSING: Literal[MISSING] = Sentinel["MISSING"], right? Would just using MISSING = Sentinel("MISSING") be fine, relying on type inference? I assume this would require special-casing in type checkers? 3. How does Final play into this? Note that the currently proposed implementation does NOT require that the variable name be identical to the string passed to Sentinel, nor that the Sentinel object can be found using the given name in the module's namespace in order for copying/unpickling to work. Is this reasonable or should it be changed? I'd like to add 1-2 paragraphs about type checking to the PEP, addressing the above, and anything else you think is important to have well defined. - Tal Einat [1] https://www.python.org/dev/peps/pep-0661/ [2] https://discuss.python.org/t/pep-661-sentinel-values/9126 [3] https://mail.python.org/archives/list/typing-sig@python.org/thread/NDEJ7UCD…

6 10

Typing Meetup
by S Pradeep Kumar Oct. 15, 2021

Oct. 15, 2021

Hello all, We plan to hold a virtual Typing Meetup to discuss key typing-related issues in real-time. The idea is that we can make progress on important or controversial discussions, get early feedback on in-progress PEPs, and gauge interest in new proposals. A virtual meetup will be more accessible to people in other timezones than Bay Area in-person meetups. It should also help consolidate discussions that are often spread out across mailing lists or Github repos. The tentative plan is to have a monthly meetup, but we can change that as needed. Date: Monday, November 01, 2021. Time: 10 am San Francisco time (UTC-7) / 5 pm London time (UTC) Duration: 1 hour Time and date are flexible. I'll share the zoom link and agenda the week before. I'll also record the meetup and share afterwards. If you're interested in presenting a new proposal or your work on some existing proposal, please respond here. We can shoot for 15 minutes per presentation. I know there are quite a few interesting threads on typing-sig that haven't seen much activity in a while :) -- S Pradeep Kumar

1 0

Next tensor typing meeting
by Matthew Rahtz Oct. 12, 2021

Oct. 12, 2021

Hi all, Our next tensor typing meeting is supposed to be next Monday (the 11th). There are no updates from my end this month (we're just waiting for PEP 646 to be re-reviewed by the steering council); does anyone else have something they want to present? Thanks! Matthew

2 3

Accept Final as indicating ClassVar for dataclasses
by Gregory Beauregard Oct. 10, 2021

Oct. 10, 2021

PEP 591 for Final states "Type checkers should infer a final attribute that is initialized in a class body as being a class variable. Variables should not be annotated with both ClassVar and Final." ClassVar is used to indicate library behavior in dataclasses. Therefore, I propose accepting the Final attribute as an indicator of a ClassVar in dataclass class bodies in order to be better compatible with the Final PEP. One edge case that would need to be handled is when someone might explicitly mark a dataclass field Final: a: Final[int] = dataclasses.field(init=False, default=10) Because it would change a library, I've opened an issue as well, but I'm copying here since it affects type checkers: https://bugs.python.org/issue45384 I appreciate your time, Gregory Beauregard

1 1

Meetup to discuss Callable Syntax
by S Pradeep Kumar Oct. 8, 2021

Oct. 8, 2021

Hello all, User-friendly Callable syntax is something we all want to see. However, there has been no real movement on the discussion in the last one month. People have had differing opinions based on design principles, stats, and personal experience. This has led to half a dozen proposals with some subtle and not-so-subtle differences [1]. And Guido pointed out that we want to settle on a full proposal instead of going for a partial PEP now and settling on the remaining details in a later PEP. How about we meet up over a zoom call and discuss some of the key concerns? Things proceed noticeably faster when people from the typing community are able to get in the same (virtual) room (I saw that with PEP 646 and the typing summit). We don't have to reach a consensus in the first meeting itself. It's more a way to break the deadlock and make progress. Date: 10 am San Francisco time (6 pm UK time), Monday, September 13, 2021. Date is flexible. Duration: 1 hour. If there is enough interest, I can set up the video call and share the link and agenda. If you're interested in presenting pros and cons for your favorite proposal, reply to this mail. [1]: Document containing the various proposals and use cases: https://docs.google.com/document/d/11VkNk9N8whxS1SrBv2BNN1Csl5yeTlgG7WyLrCB… -- S Pradeep Kumar

9 33

Making lengthy types legible in editors
by rsoklaski＠gmail.com Oct. 7, 2021

Oct. 7, 2021

I have encountered the following legibility problem in two libraries that I work on. I will: - describe the issue I am facing - ask for guidance - describe a hack that I have played with & my ideal-solution THE PROBLEM ----------- One of the parameters in my most used, user-facing functions has a "lengthy" annotation; e.g. _T = TypeVar("_T", bound=Callable) Wrapper = Callable[[_T], _T] ValidWrapper = A[Wrapper] | B[Wrapper] | C[Wrapper] | Wrapper | None | str def my_func(x: ValidWrapper | Sequence[ValidWrapper]): ... Viewing the signature of `my_func` in, via tooltip, in IPython, Jupyter, VSCode, or PyCharm reveals an absolute mess. Consider that A, B, and C are generics that are defined in a deeply-nested module of my library and their paths are included in the signature... You can see screenshots from these editors here: https://github.com/mit-ll-responsible-ai/hydra-zen/pull/125 This legibility issue is a major concern to me. Some of my users (including high school students) are rightfully bewildered when they bring up the signature in their editor via a tooltip (or Shift+Tab in Jupyter). However, accurate annotations are obviously important to help to guide them away from making mistakes. GUIDANCE -------- Does anyone have any advice for dealing with this legibility issue? Is there a mechanism in typing that I have overlooked? It occurs to me (as I write this) that perhaps moving to type-stubs could circumvent this. MY ATTEMPT AT A SOLUTION ------------------------ The ideal solution that I can think of is something with the form-factor MyType = Alias("MyType", true_type=ActualType) Where MyType is exactly the same as ActualType, but it provides a string field (that returns "MyType" here), that editors/tools can hook into to inform how they print the type - a purely aesthetic mechanism. I have attempted to experiment with NewType, Alias, and TypeVar to see if I could manage this legibility issue with the current typing capabilities. I have one hack that cleans up the experience for Jupyter users: if TYPE_CHECKING: ValidWrapper = # Actual, correct annotation else: ValidWrapper = TypeVar("ValidWrapper") This coerces Juptyer to render the annotation as `Union[~ValidWrapper , Sequence[~ValidWrapper ]]`. Clearly, this is an abuse of TypeVar, and I am generally not happy with this approach. It also has no improvement for VSCode and PyCharm. Thank you for your time! Ryan Soklaski

3 5

seeking feedback on a proposed deprecation library to be used at type checking time (in pytype)
by Yilei Yang Oct. 5, 2021

Oct. 5, 2021

Hi everyone, I'm working on a proposal to introduce a deprecation library. It is intended to be used at type checking time, and will be supported by pytype. The primary benefit (over DeprecationWarning at runtime) is to provide deprecation signals at static analysis time (think type checking, linting, IDE support). **The gist of the deprecation library** ============================ The gist of the deprecation library is to use a data class Deprecated wrapped in a typing.Annotated to mark variables/methods/functions as deprecated. Here are a couple of examples: # API: @dataclasses.dataclass class Deprecated: reason: str # Usages: my_constant: Annotated[int, Deprecated("my_constant is deprecated.")] = 123 def do_something(name: str, arg: Annotated[Any, Deprecated("`arg` is deprecated.")] = None) -> None: # The parameter `arg` is deprecated def my_function(name: str) -> Annotated[str, Deprecated("my_function is deprecated.")]: # Putting Deprecated in the return value type marks the method/function as deprecated. class MyClass: def __init__(self) -> Annotated[None, Deprecated("MyClass is deprecated.")]: # Marking __init__ as deprecated makes the class deprecated It is a no-op at runtime, and per PEP 593 it should be ignored by type checkers that don't support it. We also intend to include a higher-level decorator that will be turned into the lower-level annotations at type checking time. Here is an example: # API: def deprecated(message: str, stacklevel: int = 2) -> Callable[[_T], _T]: def decorator(wrapped): @functools.wraps(wrapped) def wrapper(*args, **kwargs): warnings.warn(message, DeprecationWarning, stacklevel=stacklevel) return wrapped(*args, **kwargs) return wrapper return decorator # Usage: @deprecated("my_func is deprecated") def my_func(name): print(f"Called my_func with {name}") At type checking time, it turns the function to def my_func(name) -> Annotated[?, Deprecated("my_func is deprecated")]. ? is either a concrete type if the type checker can infer its return type, or Any if not. At runtime, it raises a DeprecationWarning whenever my_func is called. **Feedback I'm looking for** ====================== Above isn't very detailed nor a finalized design. But I hope the idea is just clear enough to seek some initial feedback. The things I'm specifically looking for are: * Would the use of these APIs cause issues for other type checkers? Per my interpretation of PEP 593, it shouldn't. I tested with pytype, mypy, pyright, and pyre, one hiccup I found is pyre thinks annotating __init__ with Annotated[None, Deprecated("")] is an error (Incompatible constructor annotation [17]: __init__ is annotated as returning typing.Annotated[None], but it should return None.) * Is there interest from other type checkers to support such a deprecation library, so that static analysis tooling can provide better support detecting the use of deprecated APIs? Thanks, Yilei

5 6

"Asserting" type casts
by Stephen Rosen Oct. 1, 2021

Oct. 1, 2021

Hi all, I've been reading about the TypeGuards coming in 3.10 (and I'm excited!). I have a question which is related to guards and casts. One use-case I have is parsing data from some external source (e.g. JSON from a web API), but then presenting the result as a well-typed-value. For example: def get_error_code(error_data: Dict[str, Any]) -> str: code = error_data["code"] if isinstance(code, str): return code raise ValueError("bah! bad code!") The type of a value from an untrustworthy source should be checked at runtime, and we raise an error if the type is not correct. There are a few other ways of writing this, but with pitfalls of varying levels of obviousness. e.g. return cast(str, error_data["code"]) or code = error_data["code"] assert isinstance(code, str) return code The cast has no runtime check, and any `assert` gets removed if optimization is used. The `cast` "says what I mean" most succinctly, but by design doesn't include a runtime check. This seems like something which could be made the responsibility of `typing` in 3.11 or later. For example, an 'asserting_cast' variant which takes a (TypeGuard | Type) and lets the above examples be written as code = error_data["code"] return asserting_cast(code, str) or, to clarify the TypeGuard case, def is_str(x: Any) -> TypeGuard[str]: return isinstance(x, str) ... code = error_data["code"] return asserting_cast(code, is_str) I think the implementation would be quite simple, but IMO there's value in stdlib `typing` providing a pre-built solution which has thought about odd cases like non-runtime-checkable protocols. I'd be more than happy to work on adding something like this to `typing` if it seems reasonable. It would be a "safer but slower" alternative to cast and can raise a custom error, e.g. FailedTypeCastError(ValueError) . My biggest two questions are... Have I missed something obvious in `typing` or maybe `typing-extensions` which already covers this? Is this a bad idea, and if so can someone help me understand why? Thanks in advance, -Stephen

1 0