[I subscribed to the list now... unfortunately the mailman 3 web interface turns out to be pretty much unusable for me.] Mark Mendoza wrote:
Hi Nathaniel! Thank you for the interest and feedback on the PEP!
Thanks for working on it! :-)
def call_in_special_context(f, *args, **kwargs, special_kwarg): This example doesn't actually pass the python parser. The rule is that nothing can come after a **kwargs parameter. However, if you consider the corrected spelling: def call_in_special_context(__f: Callable[Ps, R], *args: Ps.args, special_kwarg: SomeType, **kwargs, Ps.kwargs) -> R:
This is explicitly banned by the PEP: "we require that the additional arguments be anonymous arguments that precede the args and kwargs".
Note: I think the PEP text is currently pretty unclear about what it means by "anonymous arguments". In 3.8+ we have syntax for those (via PEP 570), but you're not using that syntax here, and it would be unfortunate if this feature were restricted to projects that have adopted 3.8+ only syntax...
The rationale for this is that otherwise we could end up substituting our way into impossible callable types: in this example, if the callable passed in for __f also had a special_kwarg keyword only parameter." Consider the following example: def wrapper(f: Callable[TParams, int]) -> Callable[TParams, int]: def inner(*args: TParams.args, **kwargs: TParams.args) -> int: # fails at runtime with TypeError: call_in_special_context() got multiple values for keyword argument 'special_kwarg' return call_in_special_context(f, *args, special_kwarg=42, **kwargs) return inner
@wrapper def ohno(*, special_kwarg: str) -> int: return -1
ohno("A")
We can avoid this situation with positional-only parameters, so those are the only ones we accept.
Hmmmmmm. I see how that code raises an error, but I don't really understand the connection between that and disallowing additional kwargs. It seems to me that if we follow the rules of the PEP and define: def call_in_special_context(__f: Callable[Ps, R], *args: Ps.args, **kwargs: Ps.kwargs) -> R: ... ...then the PEP does allow your example, but you still get the same error at runtime. Also, even if we fail to catch this error, is that a problem? Python type annotations don't try to guarantee that all errors will be caught statically. The reason I'm asking about this case is that in Trio, we have a lot of 'call_in_special_context' type functions, and are trying to figure out the best convention for how to pass arguments to 'call_in_special_context' vs 'f'. Currently we follow the asyncio convention where positional args go to 'f' while kwargs go to 'call_in_special_context', but this turns out to be unsustainable – it's a constant irritant for users, we get complaints all the time, and it has bad effects like encouraging API designers to avoid using kwargs in their APIs. So we need something else, and most options are pretty awkward. The frontrunner so far is to declare that __dunder__ kwargs modify *how* the function is called, and regular kwargs are passed through to the function, e.g.: def run_in_thread(f, *args, __thread_limit__=None, **kwargs): raise_if_has_unrecognized_dunders(kwargs) ... [This would be clearer if we could write it 'def run_in_thread(f, *args, **kwargs, __thread_limit__=None)', but oh well.] It would be very convenient if we could write: def run_in_thread(f: Callable[TP, TR], *args: TP.args, __thread_limit__: Optional[int]=None, **kwargs: TP.kwargs) -> TR: ... def some_func(x: int): ... run_in_thread(f, x="hello") and get a nice error from the type checker complaining that we passed 'x' as a str but expected int. I feel like 99% of the value of parameter types here is that they let you do *any* type-checking on 'x'. With PEP 570 as currently written, 'run_in_thread' has to be typed with 'Any's everywhere, so you don't catch *any* errors, which seems clearly worse than catching *most* errors but still missing a few exotic cases with kwarg collisions.