Re: Compact syntax for Callable: `(Arg1, Arg2) -> Ret`
I think it's important to be aware of the subtle differences between *function type annotations* and *callable types*. The type annotations on a function definition (or stub) define *all* the possible ways the function can be called. Callable types on the other hand define *one* particular way a function can be called. When you're writing type annotations for a callback argument, you base it on how the callback will be *called*, not how any particular function is *defined*. These differences also mean we have slightly different requirements from the syntax. A function definition can have three types of arguments: positional-or-keyword arguments (the default), positional-only arguments and keyword-only arguments. For a callable type on the other hand there are two types (which correspond to how the function will actually be called): positional arguments and keyword arguments. Due to these differences I don't think the "stub syntax" makes sense. In order to make it unambiguous we would have to disallow positional-or-keyword arguments, so you would always need to use / or * or both. And even then it wouldn't be quite intuitive to newcomers that a callable type like (a: int, /) -> bool would also match a function defined as def foo(bar: int, baz: int = 0) -> bool. It's better to use new intuitive syntax than to use existing syntax in unintuitive ways IMO. The hybrid syntax makes more sense, but I would not allow * and / in this syntax since they have no useful meaning in this context. There is also no point being able to specify that a named argument should be optional - it will either be given or not when calling the function, so it's irrelevant whether the implementation provides a default value. Variadic arguments are still needed, but giving them names would have no meaning. Here are a few options I can think of for how they could be specified: - (*: int, **: Any) - (*int, **Any) - (*int, *: Any) - (*int, **: Any) I think the second option would be preferable, because it matches the syntax for unpacking. On the other hand, the last two keep the convention of using ":" for keyword arguments. To be clear about the meaning of this syntax: Any matching function *has to have* corresponding *args and **kwargs in its definition - variadic arguments in the callable type would never match non-variadic required arguments in the function definition. To give an example of how the hybrid syntax could be used in practice, consider the following function taking a callback argument: ``` def foo(x: int, callback: ??) -> bool: y = "bar" return callback(x, y, asdf=6.5) ``` From this, writing the type annotation for the callback is straightforward: ``` callback: (int, str, asdf: float) -> bool ``` And one with variadic arguments: ``` def foo(x: Sequence[int], callback: (*int, **Any) -> bool, **kwargs: Any) -> bool: return callback(*x, **kwargs) ``` I'm also in favor of the proposed syntax of (...) -> T to cover the case where it is not statically known how the function might be called, e.g. for most decorators. We should also consider how the new syntax will interact with PEP 612 (ParamSpec). Should the following modified example from the PEP be allowed? ``` P = ParamSpec("P") R = TypeVar("R") def add_logging(f: P -> R) -> (P -> Awaitable[R]): ... ``` Or perhaps even something like this? ``` def with_request(f: (Request, *P.args, **P.kwargs) -> R) -> (P -> R): ... ``` - Daniel
El sáb, 19 jun 2021 a las 3:28, Daniel Mouritzen (<dmrtzn@gmail.com>) escribió: places. For example, a callback could be called sometimes with named and sometimes with positional arguments; an argument with a default is sometimes omitted and sometimes is not; et cetera.
There is also no point being able to specify that a named argument should be optional - it will either be given or not when calling the function, so it's irrelevant whether the implementation provides a default value.
It's not hard to imagine a program that calls a function with different arguments in different circumstances. Either way, it feels very inadequate to have large classes of functions, Python's main callable objects, that are inexpressible as Callable types. This is especially relevant because any sort of function can be used as a value, for example as a default argument, and many untyped or partially-typed programs take a more duck-typed approach; "here's the default function but if you pass something that takes the same arguments that will be used instead." Python programs do all sorts of funny things with functions and their arguments, and we should be wary of limiting the types of functions and callables that can be expressed in the Python type system — a limited type system makes many legacy libraries untypable (for example, if it can't provide a more accurate annotation than "Callable") and restricts which "Pythonic" APIs (which tend to be extremely dynamic) can be added to new, typed Python programs. On Sat, Jun 19, 2021, at 6:28 AM, Daniel Mouritzen wrote:
I disagree with the premise that *function type annotations* and *callable type annotations* are different or have different requirements. Treating them differently would lead to unnecessary and undesirable inconsistencies and limitations. That is to say, I agree with Jelle and Rebecca. Yes, a callback annotation specifies how the callback must be called, but that doesn't mean it must limit the caller to only one way of invoking the callback. A callback can be invoked in multiple ways just like a regular function. When a callback is invocable in multiple ways, it is up the a type checker to verify that any supplied callback is compatible with all possible invocation techniques described in the annotation. This is already implemented in type checkers today with [callback protocols](https://www.python.org/dev/peps/pep-0544/#callback-protocols). -Eric -- Eric Traut Contributor to Pyright & Pylance Microsoft Corp.
El sáb, 19 jun 2021 a las 3:28, Daniel Mouritzen (<dmrtzn@gmail.com>) escribió: places. For example, a callback could be called sometimes with named and sometimes with positional arguments; an argument with a default is sometimes omitted and sometimes is not; et cetera.
There is also no point being able to specify that a named argument should be optional - it will either be given or not when calling the function, so it's irrelevant whether the implementation provides a default value.
It's not hard to imagine a program that calls a function with different arguments in different circumstances. Either way, it feels very inadequate to have large classes of functions, Python's main callable objects, that are inexpressible as Callable types. This is especially relevant because any sort of function can be used as a value, for example as a default argument, and many untyped or partially-typed programs take a more duck-typed approach; "here's the default function but if you pass something that takes the same arguments that will be used instead." Python programs do all sorts of funny things with functions and their arguments, and we should be wary of limiting the types of functions and callables that can be expressed in the Python type system — a limited type system makes many legacy libraries untypable (for example, if it can't provide a more accurate annotation than "Callable") and restricts which "Pythonic" APIs (which tend to be extremely dynamic) can be added to new, typed Python programs. On Sat, Jun 19, 2021, at 6:28 AM, Daniel Mouritzen wrote:
I disagree with the premise that *function type annotations* and *callable type annotations* are different or have different requirements. Treating them differently would lead to unnecessary and undesirable inconsistencies and limitations. That is to say, I agree with Jelle and Rebecca. Yes, a callback annotation specifies how the callback must be called, but that doesn't mean it must limit the caller to only one way of invoking the callback. A callback can be invoked in multiple ways just like a regular function. When a callback is invocable in multiple ways, it is up the a type checker to verify that any supplied callback is compatible with all possible invocation techniques described in the annotation. This is already implemented in type checkers today with [callback protocols](https://www.python.org/dev/peps/pep-0544/#callback-protocols). -Eric -- Eric Traut Contributor to Pyright & Pylance Microsoft Corp.
participants (4)
-
Daniel Mouritzen -
Eric Traut -
Jelle Zijlstra -
Rebecca Turner