Mailman 3 June 2021 - Typing-sig

Any way to express that a dict is expected to have a key(s) but don't care about other keys?
by Brett Cannon 23 Dec '22

23 Dec '22

Looking at TypedDict it seems that it does not like the idea of superfluous keys existing in a dict. In my case I have a REST call returning JSON for an error and all I'm guaranteed is to get an "error" key. But there may be extra keys that provide more details about the specific error. Unfortunately I don't know the exhaustive list of keys so I can't use a TypedDict with `totality == False` to try to be complete. I can't think of any way to somehow say "require/expect this one key, but other keys are OK". Am I missing anything that would let me do this?

16 34

Explicit Type Aliases
by Shannon Zhu 17 Aug '22

17 Aug '22

Hi everyone, I’d like to continue the discussion from our last typing summit on explicit type aliases. A quick summary of the current state and the proposal -- Current state: ``` from typing import List x = List[int] # considered a type alias y : Type[List[int]] = List[int] # considered an expression z : Any # considered a type alias z : x = [] # fine z : y = [] # invalid type error ``` Proposal (with explicit aliases): ``` from typing import List, TypeAlias x = TypeAlias[List[int]] # considered a type alias y = List[int] # considered an expression z : Any # considered an expression z = TypeAlias[Any] # considered a type alias reveal_type(x) # Type[List[int]] reveal_type(y) # Any (return type of __getitem__) z : x = [] # fine z : y = [] # invalid type error ``` Some of the benefits we’re hoping to gain from explicit type aliases: * Clearly distinguish between an unannotated global assignment and a type alias, especially when parsing forward-referencing string annotations. * Avoid the confusing case of type aliases in which some part of the type is invalid or undefined. This would facilitate warnings on invalid types at the alias definition rather than later on when the alias is used. * Make valid and invalid types more intuitive to Python programmers by shifting from delineation of non-aliases with a meta annotation toward delineation of aliases with `TypeAlias`. * Remove special treatment of values annotated as `Any`, which currently break all type aliasing rules. This will also allow us to clean up some of the distinctions we maintain between unannotated values and explicit Anys. Note: We also considered denoting the alias as an annotation (e.g. `x: TypeAlias = int`) as an alternative to the syntax above. I’m interested to hear your thoughts and suggestions on making type aliasing explicit going forward – we’d like to start implementing a version of this in Pyre soon! Shannon

7 10

PEP 655: Marking individual TypedDict items as required or potentially-missing
by David Foster 23 Feb '22

23 Feb '22

The new syntax Required[...] and NotRequired[...] for marking individual keys of a TypedDict has now been formally proposed as a draft PEP. Please see the latest text at the bottom of this email, or online at: https://www.python.org/dev/peps/pep-0655/ Comments welcome. :) Meanwhile I'll proceed to implement this syntax in mypy. -- David Foster | Seattle, WA, USA Contributor to TypedDict support for mypy >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> PEP: 655 Title: Marking individual TypedDict items as required or potentially-missing Author: David Foster <david at dafoster.net> Sponsor: Guido van Rossum <guido at python.org> Discussions-To: typing-sig at python.org Status: Draft Type: Standards Track Content-Type: text/x-rst Requires: 604 Created: 30-Jan-2021 Python-Version: 3.10 Post-History: 31-Jan-2021, 11-Feb-2021, 20-Feb-2021, 26-Feb-2021 Abstract ======== `PEP 589 <https://www.python.org/dev/peps/pep-0589/>`__ defines syntax for declaring a TypedDict with all required keys and syntax for defining a TypedDict with `all potentially-missing keys <https://www.python.org/dev/peps/pep-0589/#totality>`__ however it does not provide any syntax to declare some keys as required and others as potentially-missing. This PEP introduces two new syntaxes: ``Required[...]`` which can be used on individual items of a TypedDict to mark them as required, and ``NotRequired[...]`` which can be used on individual items to mark them as potentially-missing. Motivation ========== It is not uncommon to want to define a TypedDict with some keys that are required and others that are potentially-missing. Currently the only way to define such a TypedDict is to declare one TypedDict with one value for ``total`` and then inherit it from another TypedDict with a different value for ``total``: :: class _MovieBase(TypedDict): # implicitly total=True title: str class Movie(_MovieBase, total=False): year: int Having to declare two different TypedDict types for this purpose is cumbersome. Rationale ========= One might think it unusual to propose syntax that prioritizes marking *required* keys rather than syntax for *potentially-missing* keys, as is customary in other languages like TypeScript: :: interface Movie { title: string; year?: number; // ? marks potentially-missing keys } The difficulty is that the best word for marking a potentially-missing key, ``Optional[...]``, is already used in Python for a completely different purpose: marking values that could be either of a particular type or ``None``. In particular the following does not work: :: class Movie(TypedDict): ... year: Optional[int] # means int|None, not potentially-missing! Attempting to use any synonym of “optional” to mark potentially-missing keys (like ``Missing[...]``) would be too similar to ``Optional[...]`` and be easy to confuse with it. Thus it was decided to focus on positive-form phrasing for required keys instead, which is straightforward to spell as ``Required[...]``. Nevertheless it is common for folks wanting to extend a regular (``total=True``) TypedDict to only want to add a small number of potentially-missing keys, which necessitates a way to mark keys that are *not* required and potentially-missing, and so we also allow the ``NotRequired[...]`` form for that case. Specification ============= The ``typing.Required`` type qualifier is used to indicate that a variable declared in a TypedDict definition is a required key: :: class Movie(TypedDict, total=False): title: Required[str] year: int Additionally the ``typing.NotRequired`` type qualifier is used to indicate that a variable declared in a TypedDict definition is a potentially-missing key: :: class Movie(TypedDict): # implicitly total=True title: str year: NotRequired[int] It is an error to use ``Required[...]`` or ``NotRequired[...]`` in any location that is not an item of a TypedDict. It is valid to use ``Required[...]`` and ``NotRequired[...]`` even for items where it is redundant, to enable additional explicitness if desired: :: class Movie(TypedDict): title: Required[str] # redundant year: NotRequired[int] Backwards Compatibility ======================= No backward incompatible changes are made by this PEP. How to Teach This ================= To define a TypedDict where most keys are required and some are potentially-missing, define a single TypedDict as normal and mark those few keys that are potentially-missing with ``NotRequired[...]``. To define a TypedDict where most keys are potentially-missing and a few are required, define a ``total=False`` TypedDict and mark those few keys that are required with ``Required[...]``. If some items accept ``None`` in addition to a regular value, it is recommended that the ``TYPE|None`` syntax be preferred over ``Optional[TYPE]`` for marking such item values, to avoid using ``Required[...]`` or ``NotRequired[...]`` alongside ``Optional[...]`` within the same TypedDict definition: Yes: :: from __future__ import annotations # for Python 3.7-3.9 class Dog(TypedDict): name: str owner: NotRequired[str|None] Avoid (unless Python 3.5-3.6): :: class Dog(TypedDict): name: str # ick; avoid using both Optional and NotRequired owner: NotRequired[Optional[str]] Reference Implementation ======================== The goal is to be able to make the following statement: The `mypy <http://www.mypy-lang.org/>`__ type checker supports ``Required`` and ``NotRequired``. A reference implementation of the runtime component is provided in the `typing_extensions <https://github.com/python/typing/tree/master/typing_extensions>`__ module. The mypy implementation is currently still being worked on. Rejected Ideas ============== Special syntax around the *key* of a TypedDict item --------------------------------------------------- :: class MyThing(TypedDict): opt1?: str # may not exist, but if exists, value is string opt2: Optional[str] # always exists, but may have null value or: :: class MyThing(TypedDict): Optional[opt1]: str # may not exist, but if exists, value is string opt2: Optional[str] # always exists, but may have null value These syntaxes would require Python grammar changes and it is not believed that marking TypedDict items as required or potentially-missing would meet the high bar required to make such grammar changes. Also, “let’s just not put funny syntax before the colon.” [1]_ Marking required or potentially-missing keys with an operator ------------------------------------------------------------- We could use unary ``+`` as shorthand to mark a required key, unary ``-`` to mark a potentially-missing key, or unary ``~`` to mark a key with opposite-of-normal totality: :: class MyThing(TypedDict, total=False): req1: +int # + means a required key, or Required[...] opt1: str req2: +float class MyThing(TypedDict): req1: int opt1: -str # - means a potentially-missing key, or NotRequired[...] req2: float class MyThing(TypedDict): req1: int opt1: ~str # ~ means a opposite-of-normal-totality key req2: float Such operators could be implemented on ``type`` via the ``__pos__``, ``__neg__`` and ``__invert__`` special methods without modifying the grammar. It was decided that it would be prudent to introduce longform syntax (i.e. ``Required[...]`` and ``NotRequired[...]``) before introducing any shortform syntax. Future PEPs may reconsider introducing this or other shortform syntax options. Marking absence of a value with a special constant -------------------------------------------------- We could introduce a new type-level constant which signals the absence of a value when used as a union member, similar to JavaScript’s ``undefined`` type, perhaps called ``Missing``: :: class MyThing(TypedDict): req1: int opt1: str|Missing req2: float Such a ``Missing`` constant could also be used for other scenarios such as the type of a variable which is only conditionally defined: :: class MyClass: attr: int|Missing def __init__(self, set_attr: bool) -> None: if set_attr: self.attr = 10 :: def foo(set_attr: bool) -> None: if set_attr: attr = 10 reveal_type(attr) # int|Missing Misalignment with how unions apply to values '''''''''''''''''''''''''''''''''''''''''''' However this use of ``...|Missing``, equivalent to ``Union[..., Missing]``, doesn’t align well with what a union normally means: ``Union[...]`` always describes the type of a *value* that is present. By contrast missingness or non-totality is a property of a *variable* instead. Current precedent for marking properties of a variable include ``Final[...]`` and ``ClassVar[...]``, which the proposal for ``Required[...]`` is aligned with. Misalignment with how unions are subdivided ''''''''''''''''''''''''''''''''''''''''''' Furthermore the use of ``Union[..., Missing]`` doesn’t align with the usual ways that union values are broken down: Normally you can eliminate components of a union type using ``isinstance`` checks: :: class Packet: data: Union[str, bytes] def send_data(packet: Packet) -> None: if isinstance(packet.data, str): reveal_type(packet.data) # str packet_bytes = packet.data.encode('utf-8') else: reveal_type(packet.data) # bytes packet_bytes = packet.data socket.send(packet_bytes) However if we were to allow ``Union[..., Missing]`` you’d either have to eliminate the ``Missing`` case with ``hasattr`` for object attributes: :: class Packet: data: Union[str, Missing] def send_data(packet: Packet) -> None: if hasattr(packet, 'data'): reveal_type(packet.data) # str packet_bytes = packet.data.encode('utf-8') else: reveal_type(packet.data) # Missing? error? packet_bytes = b'' socket.send(packet_bytes) or a check against ``locals()`` for local variables: :: def send_data(packet_data: Optional[str]) -> None: packet_bytes: Union[str, Missing] if packet_data is not None: packet_bytes = packet.data.encode('utf-8') if 'packet_bytes' in locals(): reveal_type(packet_bytes) # bytes socket.send(packet_bytes) else: reveal_type(packet_bytes) # Missing? error? or a check via other means, such as against ``globals()`` for global variables: :: warning: Union[str, Missing] import sys if sys.version_info < (3, 6): warning = 'Your version of Python is unsupported!' if 'warning' in globals(): reveal_type(warning) # str print(warning) else: reveal_type(warning) # Missing? error? Weird and inconsistent. ``Missing`` is not really a value at all; it’s an absence of definition and such an absence should be treated specially. Difficult to implement '''''''''''''''''''''' Eric Traut from the Pyright type checker team has stated that implementing a ``Union[..., Missing]``-style syntax would be difficult. [2]_ Introduces a second null-like value into Python ''''''''''''''''''''''''''''''''''''''''''''''' Defining a new ``Missing`` type-level constant would be very close to introducing a new ``Missing`` value-level constant at runtime, creating a second null-like runtime value in addition to ``None``. Having two different null-like constants in Python (``None`` and ``Missing``) would be confusing. Many newcomers to JavaScript already have difficulty distinguishing between its analogous constants ``null`` and ``undefined``. Replace Optional with Nullable. Repurpose Optional to mean “optional item”. --------------------------------------------------------------------------- ``Optional[...]`` is too ubiquitous to deprecate. Although use of it *may* fade over time in favor of the ``T|None`` syntax specified by `PEP 604 <https://www.python.org/dev/peps/pep-0604/>`__. Change Optional to mean “optional item” in certain contexts instead of “nullable” --------------------------------------------------------------------------------- Consider the use of a special flag on a TypedDict definition to alter the interpretation of ``Optional`` inside the TypedDict to mean “optional item” rather than its usual meaning of “nullable”: :: class MyThing(TypedDict, optional_as_missing=True): req1: int opt1: Optional[str] or: :: class MyThing(TypedDict, optional_as_nullable=False): req1: int opt1: Optional[str] This would add more confusion for users because it would mean that in *some* contexts the meaning of ``Optional[...]`` is different than in other contexts, and it would be easy to overlook the flag. Various synonyms for “potentially-missing item” ----------------------------------------------- - Omittable – too easy to confuse with optional - OptionalItem, OptionalKey – two words; too easy to confuse with optional - MayExist, MissingOk – two words - Droppable – too similar to Rust’s ``Drop``, which means something different - Potential – too vague - Open – sounds like applies to an entire structure rather then to an item - Excludable - Checked References ========== .. [1] https://mail.python.org/archives/list/typing-sig@python.org/message/4I3GPIW… .. [2] https://mail.python.org/archives/list/typing-sig@python.org/message/S2VJSVG… Copyright ========= This document is placed in the public domain or under the CC0-1.0-Universal license, whichever is more permissive. .. Local Variables: mode: indented-text indent-tabs-mode: nil sentence-end-double-space: t fill-column: 70 coding: utf-8 End: <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<

10 31

Thoughts on a typing.Self?
by gobot1234yt＠gmail.com 02 Nov '21

02 Nov '21

I would like to draft a PEP for a typing.Self class which would perform similarly to rust's Self keyword in function annotations. Examples ```python from typing import TypeVar F = TypeVar("F", bound="Foo") class Foo: def parse(self: F, data: bytes) -> F: ... return self ``` vs ```python from typing import Self class Foo: def parse(self, data: bytes) -> Self: ... return self ``` Another example of where this could be useful is classmethods: ```python class Foo: @classmethod def bar(cls: type[F], *args: Any, **kwargs: Any) -> F: return cls() ``` vs ```python class Foo: @classmethod def bar(cls, *args: Any, **kwargs: Any) -> Self: return cls() ``` Implementation wise it would be a copy of NoReturn, i.e. not subscriptable, a _SpecialForm and should only be valid as a return type. I haven't seen any discussion of the idea on the mailing list, so I wanted to run it by everyone. Thanks, James.

12 27

Support for sealed classes
by Adrian Freund 19 Aug '21

19 Aug '21

Hello typing-sig, I wanted to suggest adding support for sealed classes to the python type system. What are sealed classes? Sealed classes are classes of which all subclasses are known statically. This is usually accomplished by requiring all subclasses to be created in the same file/module as the parent class. What languages support sealed classes? Sealed classes are supported by C#, Kotlin and starting with Java 15 even by Java. What can sealed classes be used for? Sealed classes are useful for checking the exhaustiveness of if/else towers and (python 3.10) match statements for example when working with abstract syntax trees. Take a look at this example: ``` class Node(ABC): ... class Value(Node): ... class BinOp(Node): ... class UnOp(Node): ... n: Node match n: case Value(v): ... case BinOp(n1, op, n2): ... case UnOp(op, n1): ... # Can we statically check if all cases are handled? ``` Sealed classes in python: My suggestion would be to add a new class "Sealed" to typing and typing_extensions that can be inherited from: class Node(Sealed): ... Type checkers should make sure that no subclasses of Node are created in modules other than the module Node was created in and should treat sealed classes as abstract classes. Additionally they can keep track of all the subclasses of Node do implement exhaustiveness checking. The runtime implementation of sealed could also prevent non-typechecked code from creating additional subclasses: ``` class Sealed(ABC): def __init_subclass__(cls): for c in cls.__mro__: if Sealed in c.__bases__ and c.__module__ != cls.__module__: raise RuntimeError('Sealed class can\'t be subclassed from another module. Sealed in module "{}".'.format(c.__module__)) ``` Final example: a.py: ``` from typing import Sealed class Base(Sealed): pass class A(Base): pass class B(A): pass ``` b.py: ``` from a import Base, A class C(Base): # raises an exception pass class D(A): # also raises an exception pass ``` Adrian Freund

5 7

Compact syntax for Callable: `(Arg1, Arg2) -> Ret`
by Steven Troxler 16 Aug '21

16 Aug '21

Hi typing-sig, I’d like to start a discussion about moving forward with improved syntax for typing.Callable based on discussion in the Typing Summit: * Allow `(ArgType0, ArgType1) -> ReturnType` * In place of `Callable[[ArgType0, ArgType1], ReturnType]` A lot of folks also want a way to express things like named, optional, and variadic args, but this is more complicated and has harder tradeoffs. I'll start a separate thread next week to discuss that, let's aim to keep this thread focused on the syntactic sugar described above. If we think this is a good idea, I'm happy to work on drafting a PEP. ## Background The existing Callable type is useful for simple functions with required arguments passed by position, but there’s consensus that it can be hard to keep track of brackets, especially as the types get complex. For example, even in very simple functional-style code with higher-order functions the brackets can be cumbersome: ``` from typing import Callable def accepts_callback(callback: Callable[[str, int], str]) -> None: print(callback("answer", 42) def f(s: str, x: int) -> str: return s + ": " + str(x) accepts_callback(f) accepts_callback(lambda s, _: s) ``` In agreement with a poll from the Typing Summit, I’d like to propose allowing * `(Argtype1, Argtype2) -> ReturnType` to be used in place of * `Callable[[Argtype1, Argtype2], ReturnType]` This allows us to rewrite the example above as: ``` def accepts_callback(callback: (str, int) -> str) -> None: print(callback("answer", 42) def f(s: str, x: int) -> str: return s + ": " + str(x) accepts_callback(f) accepts_callback(lambda s, _: s) ``` Advantages of arrow syntax include: * eliminate the need to import Callable * easier-to-skim types * they are shorter, and one of the problems with Callables is just verbosity * the parentheses and arrow have more visual structure than existing bracket-based Callable syntax * a syntax that should feel familiar: * it mimics the syntax for ordinary functions and stubs * it is similar to several other languages e.g. * [typescript](https://basarat.gitbook.io/typescript/type-system/callable#arrow-syntax) * [flow](https://flow.org/en/docs/types/functions/#toc-function-types) * [kotlin](https://kotlinlang.org/docs/lambdas.html#function-types) ## Open questions about syntax rules Would we require parentheses around a callable return type as above, or allow it to be bare (which causes two ->s to be visually at the same level of the code, even though one is nested in the AST)? Consider this function (annotated using a conservative syntax where we wrap the return type in paranetheses): ``` def identity( f: (str, int, int) -> int, ) -> ((str, int, int) -> int): return f ``` In this case, would we want to * require parentheses around a callable return type as above, or * allow it to be bare (which causes two ->s to look a little bit as if they are at the same code level) Here’s what the example above would look like if we allow bare arrow syntax in the return type: ``` def identity( f: (str, int, int) -> int, ) -> (str, int, int) -> int: return f ``` Another option, if we feel like the outer parentheses are necessary would be to make the parameter grouping optional in this case, for example: ``` def identity( f: (str, int, int) -> int, ) -> (str, int, int -> int): return f ``` ## Alternatives considered We ran a poll at the typing summit comparing three options for nicer Callable syntax: * This proposal `(ArgType1, ArgType2, ArgType3) -> ReturnType` * 72% of participants upvoted this * A typescript-like syntax `(a: ArgType1, b: ArgType2) => ReturnType` * 62% of participants upvoted this * Disadvantages are * a new symbol `=>` * argument names are nuisance parameters for positional args (which is all the existing Callable supports) * `[ArgType1, ArgType2, ArgType3] -> ReturnType` * 8% of participants upvoted this * The main disadvantage is that brackets don’t seem as intuitive - Steven

10 55

Support for type checking fixed width integers
by Arun Sharma 07 Aug '21

07 Aug '21

I'm aware that the python3 type system primarily supports arbitrary precision integers and there is no support for type checking fixed width integers. However, when python is used as an approachable first language for data scientists, scientists interested in numerical analysis and possibly other generic use cases, one common technique is to map python's type system to that of a lower level underlying system. I'm primarily coming at it from the third use case in the form of py2many, a transpiler for converting python to many statically typed C-like languages. I suspect pytorch, tensorflow or anything that maps python to vectorized hardware efficiently has to deal with the precision of integers. Canonical use case: ``` #!/usr/bin/env python3 from ctypes import c_int8 as i8 def test(): a: i8 = 10 b: i8 = 20 c = a + b reveal_type(c) ``` Also would love to have type checkers infer ranges and throw errors for cases like: c: i8 = 300 Doing so would significantly benefit the use case of generating secure C++/Rust code from python source: https://github.com/adsharma/py2many/issues/233 Eric Traut suggested I post to the mailing list. More context here: https://github.com/microsoft/pyright/issues/1872 -Arun

2 4

PEP-563 and nested classes
by Sergei Lebedev 23 Jun '21

23 Jun '21

Hi typing-sig, PEP-563 has a "Backward compatibility" <https://www.python.org/dev/peps/pep-0563/#backwards-compatibility> section which contains examples of how type annotations work for nested classes. In particular, the PEP says that class C: class D: def method(self) -> D: ... should be accepted by the type checker. This does not contradict > Annotations using nested classes and their respective state are still valid. They can use local names or the fully qualified name. but is still surprising, because if we apply the usual scoping rules, the name D is not visible from the body of D. The name C is visible, so is C.D, but D is not. Does this look right to you? If yes, could you help me understand the motivation for having this special-case in the PEP? As an aside, I have tried this example in mypy (0.800), PyCharm (2020.3), pyright (in pylance 2021.6.2) and pytype (2021.05.25), and none of them seem to accept it. The same goes for typing.get_type_hints, which fails with a NameError Python 3.9.2 (default, Feb 28 2021, 17:03:44) >>> from __future__ import annotations >>> import typing >>> class C: ... class D: ... def method(self) -> D: ... ... >>> typing.get_type_hints(C.D.method) Traceback (most recent call last): ... NameError: name 'D' is not defined Cheers, Sergei

2 4

Result[T]
by Arun Sharma 22 Jun '21

22 Jun '21

Potentially off topic for this list. But posting here to get suggestions on where I might find the right audience for the discussion. Problem: how to transpile python code that might throw exceptions to languages that don't have exceptions, but do have Result<T, Err>. Does something like this make sense? ``` from typing import Optional, Generic, TypeVar from dataclasses import dataclass T = TypeVar("T") @dataclass class Result(Generic[T]): data: T error: Optional[Exception] ``` Details: https://github.com/adsharma/py2many/issues/322 -Arun

3 2

Re: Compact syntax for Callable: `(Arg1, Arg2) -> Ret`
by Daniel Mouritzen 20 Jun '21

20 Jun '21

Thanks for the feedback Jelle, Rebecca and Eric. It's true as you all point out that the same callback function could be invoked in multiple different ways, but I don't remember coming across this much in practice. A real-life example would be helpful for this discussion. Anyway I don't think the shorthand syntax needs to cover all possible uses if that means it becomes less useful for the common cases as a result. Wouldn't it make sense to use callback protocols as Eric suggested for these complex cases and use the shorthand for the more common cases? @Jelle > For example, a callback could be called sometimes with named and sometimes with positional arguments; an argument with a default is sometimes omitted and sometimes is not; et cetera. I don't think it's common to call the same callback sometimes with named and sometimes with positional arguments. When adding type annotations to code that does this, the callback invocations can just be updated to be consistent with each other by adding or removing argument names. This causes no loss of generality. As to the arguments with default values, yes I suppose allowing the previously proposed syntax for specifying that an argument must be optional (i.e. (foo: int = ...)) in the hybrid syntax would be needed to cover those cases. @Rebecca > Either way, it feels very inadequate to have large classes of functions, Python's main callable objects, that are inexpressible as Callable types. The proposed syntax can describe any function *invocation*, but no not all valid invocations of a specific function. > This is especially relevant because any sort of function can be used as a value, for example as a default argument, and many untyped or partially-typed programs take a more duck-typed approach; "here's the default function but if you pass something that takes the same arguments that will be used instead." This is exactly why the callable type should describe how the function is *used* not how it might be *defined*. With the hybrid syntax, the simplest valid type annotation will also automatically be the most general one as it defines only the minimal requirements the passed callback needs to fulfill. > Python programs do all sorts of funny things with functions and their arguments, and we should be wary of limiting the types of functions and callables that can be expressed in the Python type system — a limited type system makes many legacy libraries untypable (for example, if it can't provide a more accurate annotation than "Callable") and restricts which "Pythonic" APIs (which tend to be extremely dynamic) can be added to new, typed Python programs. This is why I brought up PEP 612. It tries to address exactly these dynamically defined functions and such. Using the hybrid syntax or stub-like syntax wouldn't change anything about this, neither of them support what you're describing (as they are defined currently). @Eric > I disagree with the premise that *function type annotations* and *callable type annotations* are different or have different requirements. Treating them differently would lead to unnecessary and undesirable inconsistencies and limitations. A type annotated function describes all the valid ways in which that specific function can be invoked. A callable type describes the requirements a function would need to fulfill in order to be used in a given context. In other words, a function type describes all usages of a given function, a callable type describes all functions that match a given usage (or set of usages, point taken). If a callable type was simply a copy of the definition of some specific usable callback function, then the callable type would be overly specific and might exclude other functions that would've worked fine. Unless we decide (as for the stub-like syntax) that e.g. a positional-only argument in a callable can match either a positional-only or positional-or-keyword argument in a function type. And that the names of positional arguments don't matter. And that the function can have additional optional arguments that are not in the callable type. But these decisions change the meaning of the syntax and thus might harm more than the familiar-looking syntax helps. > Yes, a callback annotation specifies how the callback must be called, but that doesn't mean it must limit the caller to only one way of invoking the callback. A callback can be invoked in multiple ways just like a regular function. When a callback is invocable in multiple ways, it is up the a type checker to verify that any supplied callback is compatible with all possible invocation techniques described in the annotation. This is already implemented in type checkers today with [callback protocols]( https://www.python.org/dev/peps/pep-0544/#callback-protocols). Note that none of the proposed syntaxes would be sufficient to support all possible callback usages. The callback could be used in a way that would require overloads. In this case you would need to use a callback protocol, which I think is fine. - Daniel PS: Sorry for starting a new thread with my previous email, I'm new to this mailing list stuff

1 0

Typing-sig June 2021