Hello all, Typing-sig has been discussing user-friendly syntax for the type used to represent callables. [1] Since this affects the Python language syntax, we wanted to get some high-level feedback from you before putting up a detailed PEP. TL;DR: We want to propose syntax for Callable types, e.g., `(int, str) -> bool` instead of `typing.Callable[[int, str], bool]`. Questions: 1. Are there concerns we need to keep in mind about such a syntax change? 2. Should we propose syntax for additional features in the same PEP or in a future PEP? # Motivation Guido has pointed out that `Callable` is one of the most frequently used type forms but has the least readable syntax. For example, say we have a callback `formatter` that accepts a record and a list of permissions: ```python def print_purchases( user, formatter, # <-- callback ): <...> output = formatter(record, permissions) print(output) ``` To give it a type, we currently have to write: ```python from typing import Callable def print_purchases( user: User, formatter: Callable[[PurchaseRecord, List[AuthPermission]], FormattedItem] # Hard to read. ) -> None: <...> output = formatter(record, permissions) print(output) ``` `Callable` can be hard to understand for new users since it doesn't resemble existing function signatures and there can be multiple square brackets. It also requires an import from `typing`, which is inconvenient. Around 40% of the time [2], users just give up on precisely typing the parameters and return type of their callbacks and just leave it as a blank Callable. In other cases, they don't add a type for their callback at all. Both mean that they lose the safety guarantees from typing and leave room for bugs. We believe that adding friendly syntax for Callable types will improve readability and encourage users to type their callbacks more precisely. Other modern, gradually-typed languages like TypeScript (JS), Hack (PHP), etc. have special syntax for Callable types. (As a precedent, PEP 604 recently added clean, user-friendly syntax for the widely-used `Union` type. Instead of importing `Union` and writing `expr: Union[AddExpr, SubtractExpr], we can just write `expr: AddExpr | SubtractExpr`.) # Proposal and Questions We have a two-part proposal and questions for you: 1. Syntax to replace Callable After a lot of discussion, there is strong consensus in typing-sig about adding syntax to replace Callable. So, the above example would be written as: ```python def print_purchases( user: User, formatter: (PurchaseRecord, List[AuthPermission]) -> FormattedItem, ) -> None: <...> output = formatter(record, permissions) print(output) ``` This feels more natural and succinct. Async callbacks currently need to be written as ``` from typing import Callable async_callback: Callable[[HttpRequest], Awaitable[HttpResponse]] ``` With the new syntax, they would be written as ``` async_callback: async (HttpRequest) -> HttpResponse ``` which again seems more natural. There is similar syntax for the type of decorators that pass on *args and **kwargs to the decorated function. Note that we considered and rejected using a full def-signature syntax like ```` (record: PurchaseRecord, permissions: List[AuthPermission], /) -> FormattedItem ```` because it would be more verbose for common cases and could lead to subtle bugs; more details in [3]. The Callable type is also usable as an expression, like in type aliases `IntOperator = (int, int) -> int` and `cast((int) -> int, f)` calls. **Question 1**: Are there concerns we should keep in mind about such a syntax proposal? 2. Syntax for callback types beyond Callable `Callable` can't express the type of all possible callbacks. For example, it doesn't support callbacks where some parameters have default values: `formatter(record)` (the user didn't pass in `permissions`). It *is* possible to express these advanced cases using Callback Protocols (PEP 544) [4] but it gets verbose. There are two schools of thought on typing-sig on adding more syntax on top of (1): (a) Some, including Guido, feel that it would be a shame to not have syntax for core Python features like default values, keyword arguments, etc. One way to represent default values or optionally name parameters would be: ``` # permissions is optional formatter: (PurchaseRecord, List[AuthPermission]=...) -> FormattedItem # permissions can be called using a keyword argument. formatter: (PurchaseRecord, permissions: List[AuthPermission]) -> FormattedItem ``` There are also alternative syntax proposals. (b) Others want to wait till we have more real-world experience with the syntax in (1). The above cases occur < 5% of the time in typed or untyped code [5]. And the syntax in (1) is forward-compatible with the additional proposals. So we could add them later if needed or leave them out, since we can always use callback protocols. **Question 2**: Do you have preferences either way? Do we propose (1) alone or (1) + (2)? Once we get some high-level feedback here, we will draft a PEP going into the details for various use cases. Best, Pradeep Kumar Srinivasan Steven Troxler Eric Traut PS: We've linked to more details below. Happy to provide more details as needed. [1]: typing-sig thread about the proposal: https://mail.python.org/archives/list/typing-sig@python.org/message/JZLYRAXJ... [2]: Stats about loosely-typed Callables: https://github.com/pradeep90/annotation_collector#typed-projects---callable-... [3]: Comparison and rejection of proposals: https://www.dropbox.com/s/sshgtr4p30cs0vc/Python%20Callable%20Syntax%20Propo... [4]: Callback protocols: https://www.python.org/dev/peps/pep-0544/#callback-protocols [5]: Stats on callbacks not expressible with Callable: https://drive.google.com/file/d/1k_TqrNKcbWihRZdhMGf6K_GcLmn9ny3m/view?usp=s...