Hello all,

I know many of us have been privately discussing the PEP 677 (Callable syntax) rejection and what it might mean for future typing efforts. I wanted to start a thread so that we can pool our thoughts and converge on some lessons for the future.

("We" below means various people from typing-sig - Steven, Eric, Guido, me, and everybody else who participated.)

Timeline:

+ May 2021: We started discussing callable syntax, among other syntax proposals, at the Typing Summit

+ June 2021: Steven kicked off a typing-sig thread about callable syntax [2]

+ September 2021: We all had a Zoom meetup about Callable syntax [3]. The main outcome was that, as a group, we dropped the def-style syntax proposal because it was too verbose. We didn't reach consensus about shorthand vs extended style.

+ early October 2021: We mailed python-dev to get preliminary feedback about new callable syntax [4]. There was no strong pushback at that point, as far as I can see. People pointed out edge cases such as trailing commas and proposed alternatives such as function-name-as-a-type. The Steering Council asked us to explicitly discuss the syntax changes.

+ November 2021: We had another heated Zoom discussion about shorthand vs extended syntax. [5] We still didn't reach consensus. Given that shorthand syntax was forward-compatible with extended syntax, Steven and I drafted a PEP with shorthand syntax, hoping to get it in before the deadline for 3.11.

+ December 2021: We put up the draft PEP in python-dev [6]. There was some strong pushback, especially in this thread [7] - "tl;dr: I find it very troubling that we are going on a path where we need to increase the language complexity (syntax) only in the cause 'easier' typing. So I am opposed to this change".

+ December 2021: We discovered some wrinkles in the proposal: readers could find it hard to distinguish the multiple `->`s in a function signature (this was point (4) in the SC rejection notice); readers could get confusing errors because of the relative binding of `->` and `|` [8].

+ February 2022: The SC rejected PEP 677 [1]. The reasons were: (1) "A feature for use only in a fraction of type annotations ... did not feel like a strong enough reason to justify the complexity needed to parse this new syntax and provide meaningful error messages" (2) "While the current Callable[x, y] syntax is not loved, it does work." (3) "Each time we add syntax solely for typing it shifts us further in the direction of typing being its own mini-language so we aim to tread lightly in what gets added here." (4) "We did not like the visual and cognitive consequence of multiple `->` tokens in a def."

+ February 2022: SC clarification in that thread based on Shantanu's question: "If the 677 proposed syntax had been expanded to include more def features instead of being conservative I think it would've been an even easier rejection for many of us".

My takeaways:
1. We could have tested the syntax in real-world code a lot earlier. That would have caught the readability issue with multiple arrows and the relative binding of `->` and `|`. This can be a lesson for future syntax changes.
2. It would have been good to hear the December python-dev pushback earlier. If I'd heard that in the early-October python-dev thread, I would have been much more pessimistic about the PEP's chances. Maybe instead of the informal mail in October, we could have put up a draft PEP right away. That might have gotten more eyeballs in python-dev.
3. The def vs shorthand discussion was useful and helped us decisively eliminate a proposal.
4. In retrospect, the shorthand vs extended discussion seems less useful. From the SC clarification above, it looks like it was always going to favor a simpler proposal over a more complicated proposal. Once we realized that we weren't going to reach consensus anytime soon, maybe we could have avoided the extended discussions (but maybe not).
5. It was a good call to defer the full CPython implementation until the SC verdict. That would have been a month or two of work that went nowhere.
6. Finally, on a positive note, this syntax rejection doesn't mean we can't have any typing-related Python language changes. For example, I'd be interested in seeing builtins for `mapping[str, int]` and `sequence[int]`, similar to `dict` and `list`. We could even consider `callable[[int, str], bool]`. If there's enough interest in such ideas, we could have a discussion session for this at the PyCon Typing Summit.

Eric Traut had mentioned that he'd provisionally implemented callable syntax in Pyright's parser. With this rejection, I'm guessing he'd need to roll that back. Let me know if anyone has any objection.

Overall, I think this PEP was still worth submitting. It got us valuable feedback from python-dev and the SC about future syntax changes. Special thanks to Steven Troxler, who put in a *lot* of work writing the PEP, understanding the CPython code, and implementing the syntax changes. Thanks also to everyone who participated in the vigorous debates, particularly Eric Traut, who presented alternative proposals, and Guido van Rossum, who provided much wisdom and guidance behind the scenes.

What are your takeaways? What could we (typing-sig as a whole or individual people) have done differently? Also feel free to vent your disappointment, surprise, or worry :)
--
S Pradeep Kumar

References:
[1]: https://mail.python.org/archives/list/python-dev@python.org/thread/NHCLHCU2XCWTBGF732WESMN42YYVKOXB/
[2]: https://mail.python.org/archives/list/typing-sig@python.org/thread/3JNXLYH5VFPBNIVKT6FFBVVFCZO4GFR2/
[3]: https://docs.google.com/document/d/17iqV7WWvB0IwA43EPlIqlUS6Xuvk08X3sEudAA-gQIo/edit#heading=h.12brgj3pltmn
[4]: https://mail.python.org/archives/list/python-dev@python.org/thread/VBHJOS3LOXGVU6I4FABM6DKHH65GGCUB/
[5]: https://docs.google.com/document/d/17iqV7WWvB0IwA43EPlIqlUS6Xuvk08X3sEudAA-gQIo/edit#heading=h.br0lkxyf6yj9
[6]: https://mail.python.org/archives/list/python-dev@python.org/thread/OGACYN2X7RX2GHAUP2AKRPT6DP432VCN/#2OPXHGWUDCFTDIYEARIRJFIGWLM6JTLR
[7]: https://mail.python.org/archives/list/python-dev@python.org/thread/FI4AFU3I25PECARIH2EVKAD5C5RJRE2N/#LS7DT7QVJ5RO5VX4IVCBMVQILGD2UQMQ
[8]: https://mail.python.org/archives/list/typing-sig@python.org/thread/DWIKV5XSICAXUQPNTIWMMM4YWKF6WRC6/#GAVRQL4QQWQ6L3H7V5XGHMXREBSQGBLW