Thanks for the feedback Jelle, Rebecca and Eric.
It's true as you all point out that the same callback function could be invoked in multiple different ways, but I don't remember coming across this much in practice. A real-life example would be helpful for this discussion. Anyway I don't think the shorthand syntax needs to cover all possible uses if that means it becomes less useful for the common cases as a result. Wouldn't it make sense to use callback protocols as Eric suggested for these complex cases and use the shorthand for the more common cases?
For example, a callback could be called sometimes with named and
sometimes with positional arguments; an argument with a default is sometimes omitted and sometimes is not; et cetera.
I don't think it's common to call the same callback sometimes with named and sometimes with positional arguments. When adding type annotations to code that does this, the callback invocations can just be updated to be consistent with each other by adding or removing argument names. This causes no loss of generality.
As to the arguments with default values, yes I suppose allowing the previously proposed syntax for specifying that an argument must be optional (i.e. (foo: int = ...)) in the hybrid syntax would be needed to cover those cases.
Either way, it feels very inadequate to have large classes of functions,
Python's main callable objects, that are inexpressible as Callable types.
The proposed syntax can describe any function *invocation*, but no not all valid invocations of a specific function.
This is especially relevant because any sort of function can be used as a
value, for example as a default argument, and many untyped or partially-typed programs take a more duck-typed approach; "here's the default function but if you pass something that takes the same arguments that will be used instead."
This is exactly why the callable type should describe how the function is *used* not how it might be *defined*. With the hybrid syntax, the simplest valid type annotation will also automatically be the most general one as it defines only the minimal requirements the passed callback needs to fulfill.
Python programs do all sorts of funny things with functions and their
arguments, and we should be wary of limiting the types of functions and callables that can be expressed in the Python type system — a limited type system makes many legacy libraries untypable (for example, if it can't provide a more accurate annotation than "Callable") and restricts which "Pythonic" APIs (which tend to be extremely dynamic) can be added to new, typed Python programs.
This is why I brought up PEP 612. It tries to address exactly these dynamically defined functions and such. Using the hybrid syntax or stub-like syntax wouldn't change anything about this, neither of them support what you're describing (as they are defined currently).
I disagree with the premise that *function type annotations* and
*callable type annotations* are different or have different requirements. Treating them differently would lead to unnecessary and undesirable inconsistencies and limitations.
A type annotated function describes all the valid ways in which that specific function can be invoked. A callable type describes the requirements a function would need to fulfill in order to be used in a given context. In other words, a function type describes all usages of a given function, a callable type describes all functions that match a given usage (or set of usages, point taken).
If a callable type was simply a copy of the definition of some specific usable callback function, then the callable type would be overly specific and might exclude other functions that would've worked fine. Unless we decide (as for the stub-like syntax) that e.g. a positional-only argument in a callable can match either a positional-only or positional-or-keyword argument in a function type. And that the names of positional arguments don't matter. And that the function can have additional optional arguments that are not in the callable type. But these decisions change the meaning of the syntax and thus might harm more than the familiar-looking syntax helps.
Yes, a callback annotation specifies how the callback must be called, but
that doesn't mean it must limit the caller to only one way of invoking the callback. A callback can be invoked in multiple ways just like a regular function. When a callback is invocable in multiple ways, it is up the a type checker to verify that any supplied callback is compatible with all possible invocation techniques described in the annotation. This is already implemented in type checkers today with [callback protocols]( https://www.python.org/dev/peps/pep-0544/#callback-protocols).
Note that none of the proposed syntaxes would be sufficient to support all possible callback usages. The callback could be used in a way that would require overloads. In this case you would need to use a callback protocol, which I think is fine.
PS: Sorry for starting a new thread with my previous email, I'm new to this mailing list stuff