PEP 677 (Callable Type Syntax): Final Call for Comments
Hello all, We’d like to make a last call for discussion on PEP 677: Callable Type Syntax [1] before sending this to the Steering Council. For context, PEP 677 proposes an arrow-like callable syntax for python: ``` (int, str) -> bool # equivalent to Callable[[int, str], bool] ``` Thanks everyone for comments on our earlier thread [2]. We made some changes to the PEP, but not the proposal itself, in response to that discussion: - More discussion about readability of return types and parentheses - Noting that complex code might be improved by using TypeAliases - Discussion of additional ideas in the Rejected Alternatives section Many of the concerns in the previous thread are about whether new callable syntax is worth the implementation complexity, which is an important question. We’ve seen demand for this from regular typed Python users. And given that Callable is one of the most used and complicated type annotations, we at typing-sig favor new syntax. We have considered the feedback here and in typing-sig and feel that the syntax in the PEP is a reasonable tradeoff between usability and simplicity. One concern raised was that the `async` portion of the proposed syntax may not be worthwhile, even if the rest of the proposal is. We considered removing it, but several heavy users of async pointed out [3] that: - For authors of large, typed async codebases, this syntax would be very helpful - For everyone else there’s minimal readability cost, because async callables will almost never come up. As a result, we decided to leave it in for now, but would be happy to remove it if the SC would prefer that. If there are no other concerns, we'd like to move forward and submit it to the SC. Cheers, Steven ------- [1] PEP 677: https://www.python.org/dev/peps/pep-0677/ [2] Previous RFC thread in python-dev: https://mail.python.org/archives/list/python-dev@python.org/thread/OGACYN2X7... [3] Discussion in typing-sig about removing `async` keyword: https://mail.python.org/archives/list/typing-sig@python.org/thread/RFZOSCJHD...
Like others expressed, I don't like the idea of the typing and non-typing parts of Python separating. Has anyone considered adding a new special method like `__arrow__` or something, that would be user-definable, but also defined for tuples and types as returning a Callable? For example `int -> str` could mean Callable[[int], str], and (int, str) -> bool could mean Callable[[int, str], bool]. I would find that sort of semantics more agreeable since Python already has operators that dispatch to dunder methods, and anyone who knows how that bit of Python works would automatically mostly know how the new operator works. If I understand right, this is a sort of combination of two things for which there is more precedent: first, adding a new operator based on the needs of a subset of users (the @ operator and __matmul__), and second, adding new operators to existing objects for the sake of typing (like the list[int] syntax in which type.__getitem__ was implemented to dispatch to the_type.__class_getitem__). If people don't want to add a new operator and dunder, I assume using the right shift operator like `(int, bool) >> str` would be too cheesy?
Has anyone considered adding a new special method like `__arrow__` or something, that would be user-definable, but also defined for tuples and types as returning a Callable? For example `int -> str` could mean Callable[[int], str], and (int, str) -> bool could mean Callable[[int, str], bool]. I would find that sort of semantics more agreeable since Python already has operators that dispatch to dunder methods, and anyone who knows how that bit of Python works would automatically mostly know how the new operator works.
I would personally argue that this would make the situation worse, considering the complexity would be increased with no possible use case on the route beside type annotations. On Thu, Jan 13, 2022 at 7:30 AM Dennis Sweeney <sweeney.dennis650@gmail.com> wrote:
Like others expressed, I don't like the idea of the typing and non-typing parts of Python separating.
Has anyone considered adding a new special method like `__arrow__` or something, that would be user-definable, but also defined for tuples and types as returning a Callable? For example `int -> str` could mean Callable[[int], str], and (int, str) -> bool could mean Callable[[int, str], bool]. I would find that sort of semantics more agreeable since Python already has operators that dispatch to dunder methods, and anyone who knows how that bit of Python works would automatically mostly know how the new operator works.
If I understand right, this is a sort of combination of two things for which there is more precedent: first, adding a new operator based on the needs of a subset of users (the @ operator and __matmul__), and second, adding new operators to existing objects for the sake of typing (like the list[int] syntax in which type.__getitem__ was implemented to dispatch to the_type.__class_getitem__).
If people don't want to add a new operator and dunder, I assume using the right shift operator like `(int, bool) >> str` would be too cheesy? _______________________________________________ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-leave@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/GTYLK4QA... Code of Conduct: http://python.org/psf/codeofconduct/
On Thu, 13 Jan 2022, 2:24 pm Dennis Sweeney, <sweeney.dennis650@gmail.com> wrote:
Like others expressed, I don't like the idea of the typing and non-typing parts of Python separating.
Has anyone considered adding a new special method like `__arrow__` or something, that would be user-definable, but also defined for tuples and types as returning a Callable? For example `int -> str` could mean Callable[[int], str], and (int, str) -> bool could mean Callable[[int, str], bool]. I would find that sort of semantics more agreeable since Python already has operators that dispatch to dunder methods, and anyone who knows how that bit of Python works would automatically mostly know how the new operator works.
If such a protocol were to be proposed, then int.__arrow__ could be defined such that "1 -> 100" was equivalent to "range(1, 100)" (in addition to the typing use cases). The existing section on runtime behaviour in the PEP doesn't look ready for SC consideration, since it essentially says "this hasn't been defined yet". Cheers, Nick.
On Thu, Jan 13, 2022 at 06:41:14PM +1000, Nick Coghlan wrote:
If such a protocol were to be proposed, then int.__arrow__ could be defined such that "1 -> 100" was equivalent to "range(1, 100)"
Why? What's the benefit? It is slightly shorter, but no more clear than range. I would never guess that `1 -> 100` would mean a range object. I would think it was a transformation of 1 -> 100. In e.g. Ruby the equivalent of range has syntax 1...100, which matches how we write ranges of values in English. If we had an arrow operator beyond the typing proposal, I think it would be a terrible wasted opportunity to use it for something as unclear and trivial as a range object :-( -- Steve
On Thu, 13 Jan 2022 at 08:48, Nick Coghlan <ncoghlan@gmail.com> wrote:
The existing section on runtime behaviour in the PEP doesn't look ready for SC consideration, since it essentially says "this hasn't been defined yet".
+1. The runtime behaviour needs to be specified. Otherwise this PEP is a spec for how type checkers work, but it fails to say how CPython itself will work :-( Paul
Good catch, thanks Nick! It's been specified in a linked doc [0] but I forgot to move that into the PEP itself. I'll aim to get that done today. --- [0] https://docs.google.com/document/d/15nmTDA_39Lo-EULQQwdwYx_Q1IYX4dD5WPnHbFG7...
On Thu, Jan 13, 2022 at 04:23:09AM -0000, Dennis Sweeney wrote:
Like others expressed, I don't like the idea of the typing and non-typing parts of Python separating.
Its a good thing that this PEP doesn't separate the typing and non-typing world. The arrow syntax will be a plain old Python expression that is usable anywhere, not just in annotations. It will only be *useful* in code that makes use of generic types, whether that is for annotations or runtime introspection, or exploration in the REPL, but that's okay. `is` is a built-in operator despite have exceedingly limited use-cases and sometimes being an attractive nuisance.
Has anyone considered adding a new special method like `__arrow__` or something,
In the absense of any useful functionality for this arrow syntax, I think that is a clear case of YAGNI. As it turns out, I do have a use-case for an arrow operator, but it wouldn't use a dunder either. And since my use-case doesn't have a PEP written, it would be unfair of me to derail the conversation with a half-baked proposal that is nowhere near ready to be a PEP. But if it gets rejected, all bets are off :-) If you do have some non-trivial uses for the arrow operator, it would be good to hear what they are.
that would be user-definable, but also defined for tuples and types as returning a Callable? For example `int -> str` could mean Callable[[int], str], and (int, str) -> bool could mean Callable[[int, str], bool].
That wouldn't work. The PEP explains that they don't want people to be able to write: int -> bool without the parentheses. I agree with them. I think it is important that the syntax be visually similar to an anonymous function signature with the parameter names scrubbed out: def func(a:int) -> str https://www.python.org/dev/peps/pep-0677/#parenthesis-free-syntax Analogy: we require generator comprehensions to be surrounded by parentheses: it = (expr for i in sequence) In the desired syntax: (int) -> bool that is not a tuple. Its a parenthesised comma-separated expression. The same objection applies to using the `>>` operator. There is no way to force `(int) >> str` or prevent `int >> str`.
I would find that sort of semantics more agreeable since Python already has operators that dispatch to dunder methods,
And Python already has operators which don't: - `or` and `and` - `is` and `is not` - name binding (assignment) `=` - walrus assignment operator `:=` - ternary `if else` operator. So there is plenty of precedent for dunder-less operators. -- Steve
Using an operator is an interesting idea, and we should probably call it out as an alternative in the PEP. It's not a substitute for the current PEP from the standpoint of typing-sig for a few reasons: (A) We care that the syntax be forward-compatible with supporting named arguments, as outlined in the "Extended Syntax" section of Rejected Alternatives and I don't think an operator-based syntax would extend to that. (B) We like supporting `(**P) -> bool` and `(int, **P) -> bool` for PEP 612 ParamSpec and Concatenate; this makes it much easier to write types for decorators, which are a common use case. It would be tricky to do this using an arrow operator. (C) I'd hesitate to add a dunder method on the tuple type, for reasons already mentioned here as well as the fact that it would force us to write unary callables - which are the most common kind - as `(int,) >> bool` ------ Separate from the question of arrow types, I wonder if an arrow operator would be worthwhile; I don't think it would have to be just for typing. Discussion probably belongs in a separate thread and I'd be happy to kick one off, but it sounds like Steve has a potential use case and I've always found the use of `>>`, which feels very low-level because it has baggage as a bit-shift, in DSLs to be a little weird. Proposing an operator like `=>` or `|>` (my preference would be not to use `->`) might make sense. The DSLs I'm thinking of (coming from data science) are all, loosely speaking, about some kind of pipelining - dplython uses `>>` to mimic R's pipeline operator, which works well for certain data transformations - At least at one point airflow supported creating DAG links using `>>` to represent control flow - As far as I know Tensorflow doesn't currently use `>>`, but their curried API that would lend itself to this
On Fri, Jan 14, 2022 at 2:12 AM Steven Troxler <steven.troxler@gmail.com> wrote:
Using an operator is an interesting idea, and we should probably call it out as an alternative in the PEP. It's not a substitute for the current PEP from the standpoint of typing-sig for a few reasons:
(A) We care that the syntax be forward-compatible with supporting named arguments, as outlined in the "Extended Syntax" section of Rejected Alternatives and I don't think an operator-based syntax would extend to that.
(B) We like supporting `(**P) -> bool` and `(int, **P) -> bool` for PEP 612 ParamSpec and Concatenate; this makes it much easier to write types for decorators, which are a common use case. It would be tricky to do this using an arrow operator.
Ah, good point(s), so it wouldn't really work unless there could be some dict/tuple hybrid and that would be a much bigger change.
(C) I'd hesitate to add a dunder method on the tuple type, for reasons already mentioned here as well as the fact that it would force us to write unary callables - which are the most common kind - as `(int,) >> bool`
This one's easily solved with a reflected dunder though - "(x,y,z) + q" can be handled by q.__radd__ without any help from the tuple. But yeah, it'd only work for tuples, so that's the end of that theory. Oh well. ChrisA
We just merged changes to the PEP with a detailed runtime API specification. Highlights are that: - The new type should have an `inspect.Signature`- inspired structured API - It should have a backward-compatible interface with the `__args__` and `__params__` - As with the PEP 604 union syntax, `__eq__` should treat `typing.Callable` and equivalent builtin callable types as equal
participants (7)
-
Batuhan Taskaya -
Chris Angelico -
Dennis Sweeney -
Nick Coghlan -
Paul Moore -
Steven D'Aprano -
Steven Troxler