Mailman 3 February 2022 - Typing-sig

Debrief for Callable syntax (PEP 677)
by S Pradeep Kumar 16 Feb '22

16 Feb '22

Hello all, I know many of us have been privately discussing the PEP 677 (Callable syntax) rejection and what it might mean for future typing efforts. I wanted to start a thread so that we can pool our thoughts and converge on some lessons for the future. ("We" below means various people from typing-sig - Steven, Eric, Guido, me, and everybody else who participated.) Timeline: + May 2021: We started discussing callable syntax, among other syntax proposals, at the Typing Summit + June 2021: Steven kicked off a typing-sig thread about callable syntax [2] + September 2021: We all had a Zoom meetup about Callable syntax [3]. The main outcome was that, as a group, we dropped the def-style syntax proposal because it was too verbose. We didn't reach consensus about shorthand vs extended style. + early October 2021: We mailed python-dev to get preliminary feedback about new callable syntax [4]. There was no strong pushback at that point, as far as I can see. People pointed out edge cases such as trailing commas and proposed alternatives such as function-name-as-a-type. The Steering Council asked us to explicitly discuss the syntax changes. + November 2021: We had another heated Zoom discussion about shorthand vs extended syntax. [5] We still didn't reach consensus. Given that shorthand syntax was forward-compatible with extended syntax, Steven and I drafted a PEP with shorthand syntax, hoping to get it in before the deadline for 3.11. + December 2021: We put up the draft PEP in python-dev [6]. There was some strong pushback, especially in this thread [7] - "tl;dr: I find it very troubling that we are going on a path where we need to increase the language complexity (syntax) only in the cause 'easier' typing. So I am opposed to this change". + December 2021: We discovered some wrinkles in the proposal: readers could find it hard to distinguish the multiple `->`s in a function signature (this was point (4) in the SC rejection notice); readers could get confusing errors because of the relative binding of `->` and `|` [8]. + February 2022: The SC rejected PEP 677 [1]. The reasons were: (1) "A feature for use only in a fraction of type annotations ... did not feel like a strong enough reason to justify the complexity needed to parse this new syntax and provide meaningful error messages" (2) "While the current Callable[x, y] syntax is not loved, it does work." (3) "Each time we add syntax solely for typing it shifts us further in the direction of typing being its own mini-language so we aim to tread lightly in what gets added here." (4) "We did not like the visual and cognitive consequence of multiple `->` tokens in a def." + February 2022: SC clarification in that thread based on Shantanu's question: "If the 677 proposed syntax had been expanded to include more def features instead of being conservative I think it would've been an even easier rejection for many of us". My takeaways: 1. We could have tested the syntax in real-world code a lot earlier. That would have caught the readability issue with multiple arrows and the relative binding of `->` and `|`. This can be a lesson for future syntax changes. 2. It would have been good to hear the December python-dev pushback earlier. If I'd heard that in the early-October python-dev thread, I would have been much more pessimistic about the PEP's chances. Maybe instead of the informal mail in October, we could have put up a draft PEP right away. That might have gotten more eyeballs in python-dev. 3. The def vs shorthand discussion was useful and helped us decisively eliminate a proposal. 4. In retrospect, the shorthand vs extended discussion seems less useful. From the SC clarification above, it looks like it was always going to favor a simpler proposal over a more complicated proposal. Once we realized that we weren't going to reach consensus anytime soon, maybe we could have avoided the extended discussions (but maybe not). 5. It was a good call to defer the full CPython implementation until the SC verdict. That would have been a month or two of work that went nowhere. 6. Finally, on a positive note, this syntax rejection doesn't mean we can't have any typing-related Python language changes. For example, I'd be interested in seeing builtins for `mapping[str, int]` and `sequence[int]`, similar to `dict` and `list`. We could even consider `callable[[int, str], bool]`. If there's enough interest in such ideas, we could have a discussion session for this at the PyCon Typing Summit. Eric Traut had mentioned that he'd provisionally implemented callable syntax in Pyright's parser. With this rejection, I'm guessing he'd need to roll that back. Let me know if anyone has any objection. Overall, I think this PEP was still worth submitting. It got us valuable feedback from python-dev and the SC about future syntax changes. Special thanks to Steven Troxler, who put in a *lot* of work writing the PEP, understanding the CPython code, and implementing the syntax changes. Thanks also to everyone who participated in the vigorous debates, particularly Eric Traut, who presented alternative proposals, and Guido van Rossum, who provided much wisdom and guidance behind the scenes. What are your takeaways? What could we (typing-sig as a whole or individual people) have done differently? Also feel free to vent your disappointment, surprise, or worry :) -- S Pradeep Kumar References: [1]: https://mail.python.org/archives/list/python-dev@python.org/thread/NHCLHCU2… [2]: https://mail.python.org/archives/list/typing-sig@python.org/thread/3JNXLYH5… [3]: https://docs.google.com/document/d/17iqV7WWvB0IwA43EPlIqlUS6Xuvk08X3sEudAA-… [4]: https://mail.python.org/archives/list/python-dev@python.org/thread/VBHJOS3L… [5]: https://docs.google.com/document/d/17iqV7WWvB0IwA43EPlIqlUS6Xuvk08X3sEudAA-… [6]: https://mail.python.org/archives/list/python-dev@python.org/thread/OGACYN2X… [7]: https://mail.python.org/archives/list/python-dev@python.org/thread/FI4AFU3I… [8]: https://mail.python.org/archives/list/typing-sig@python.org/thread/DWIKV5XS…

5 4

Allow subclassing from Any at runtime
by Shantanu Jain 15 Feb '22

15 Feb '22

Currently subclassing from Any gives you `TypeError: Cannot subclass typing.Any` at runtime. You can currently work around this in .py with something like: ``` if TYPE_CHECKING: AnyBaseClass = Any else: AnyBaseClass = object class X(AnyBaseClass): ... ``` As far as I'm aware, type checkers already support this. We inherit from Any in typeshed in a couple places. The need to support Any inheritance also arises naturally as a result of gradual typing, e.g. if your base class comes from an untyped library or the import cannot be resolved. I've used this trick myself a couple times and have seen use cases for this a couple times on gitter.im. It's a neat way to silence many errors from classes that do super dynamic things or to allow for classes that can duck type anywhere (e.g. mocks).

4 5

PyCon 2022 Typing Summit Interest Form
by Shannon Zhu 14 Feb '22

14 Feb '22

Hi everyone, PyCon 2022 is just a few months out, and Alfonso and I are looking forward to organizing another Typing Summit this year! We plan to be in-person in Salt Lake City during PyCon US 2022, on Thursday April 28th from 1-5pm. For those who haven’t been before, the summit is a 3-4 hour event organized around a series of presentation and discussion topics related to the type system or type checkers. Some topics are presentation heavy, some have just a few slides and then focus most of the time on group discussions of details or related questions. You can find a recording with time stamps of last year’s (virtual) typing summit here<https://pyvideo.org/pycon-us-2021/summit-typing.html>. Please fill out the PyCon 2022 Typing Summit Interest Form<https://docs.google.com/forms/d/e/1FAIpQLSdsgrGXrKZUXv9KMBwyYJkJ6L3CL_Os4Yg…> if you are interested in attending, so we can get a sense of expected size and potential presentation topics! Let me or Alfonso know if you have any questions! :) Shannon

2 1

Phantom types via NewType?
by David Foster 14 Feb '22

14 Feb '22

Hi Anton, I was just reviewing your slides on "phantom types" from the Jan 13, 2022 Typing Meetup [1], and eyeballed your related phantom-types library [2]. It seems to me that a lightweight way to implement a phantom type would be to just use a NewType. For example: ``` IntStr = NewType('IntStr', str) def parse_int_str(s: str) -> Optional[IntStr]: try: int(s) except: return None else: return cast(IntStr, s) ``` Does this seem correct? Am I understanding the idea of a "phantom type" correctly? [1]: https://docs.google.com/document/d/17iqV7WWvB0IwA43EPlIqlUS6Xuvk08X3sEudAA-… [2]: https://phantom-types.readthedocs.io/en/stable/pages/getting-started.html -- David Foster | Seattle, WA, USA Contributor to TypedDict support for mypy

2 1

Opened and changed typing issues week 2022-W06
by Typing Bot 14 Feb '22

14 Feb '22

The following is an overview of all issues and pull requests in the typing repository on GitHub with the label 'topic: feature' that were opened or updated last week, excluding closed issues. --------------------------------------------------- The following issues and pull requests were updated last week: #593 Buffer protocol types opened by @srittau https://github.com/python/typing/issues/593 #685 Record Types opened by @saulshanabrook https://github.com/python/typing/issues/685 #735 Exhaustiveness checking util opened by @Phlogistique https://github.com/python/typing/issues/735 #920 Consider loosening restrictions of `Final` opened by @DetachHead https://github.com/python/typing/issues/920 --------------------------------------------------- All issues and pull requests with the label 'topic: feature' can be viewed under the following URL: https://github.com/python/typing/issues?q=label%3A%22topic%3A+feature%22

1 0

Release: typing_extensions 4.1.0
by Jelle Zijlstra 13 Feb '22

13 Feb '22

I just released typing_extensions 4.1.0, providing many of the new typing features that will be in Python 3.11: - Runtime support for PEP 646, adding `typing_extensions.TypeVarTuple` and `typing_extensions.Unpack`. - Add interaction of `Required` and `NotRequired` with `__required_keys__`, `__optional_keys__` and `get_type_hints()`. Patch by David Cabot (@d-k-bo). - Runtime support for PEP 675 and `typing_extensions.LiteralString`. - Add `Never` and `assert_never`. Backport from bpo-46475. - `ParamSpec` args and kwargs are now equal to themselves. Backport from bpo-46676. Patch by Gregory Beauregard (@GBeauregard). - Add `reveal_type`. Backport from bpo-46414. - Runtime support for PEP 681 and `typing_extensions.dataclass_transform`. - `Annotated` can now wrap `ClassVar` and `Final`. Backport from bpo-46491. Patch by Gregory Beauregard (@GBeauregard). - Add missed `Required` and `NotRequired` to `__all__`. Patch by Yuri Karabas (@uriyyo). - The `@final` decorator now sets the `__final__` attribute on the decorated object to allow runtime introspection. Backport from bpo-46342. - Add `is_typeddict`. Patch by Chris Moradi (@chrismoradi) and James Hilton-Balfe (@Gobot1234). Thank you to all the contributors to the PEPs and to the people who contributed to the typing_extensions implementation!

2 1

Tensor typing meeting: hiatus?
by Matthew Rahtz 10 Feb '22

10 Feb '22

Hey all, Given that we haven't had anything to talk about in the tensor typing meetings for a while, and given that Pradeep's excellent Typing Meetup is providing a place for other kinds of typing discussion, I'm wondering whether it might be a good time to take a hiatus for a while - resuming once we've made a bit more progress on the implementation side/have more conceptual issues to discuss again. Any objections? If we don't hear anything by the end of the week, we'll assume this is ok, and take a break for a while. Thanks! Matthew

2 1

PEP 677 callable syntax rejected
by Gregory P. Smith 10 Feb '22

10 Feb '22

apologies for not cc'ing typing-sig on that email, anyways see https://mail.python.org/archives/list/python-dev@python.org/thread/NHCLHCU2… for the reasoning. -gps

1 0

Proposal: typing.assert_never()
by Jelle Zijlstra 09 Feb '22

09 Feb '22

Another idea that came up in a few recent threads is a `typing.assert_never()` function, which indicates that some code is supposed to be unreachable. I will submit an implementation to CPython, and we can use this thread to work out any details (or to let me know that we shouldn't do this after all). # Specification We will add a new function `typing.assert_never()` with the following signature: def assert_never(arg: NoReturn, /) -> NoReturn: ... At runtime, this will unconditionally raise `RuntimeError(f"Unexpectedly reached {arg!r}")` (wording suggestions welcome). Type checkers should emit an error if the function is called with an argument that has a non-bottom type. Examples: match bool(): case True: pass case False: pass case _ as x: assert_never(x) # ok match int(): case 0: pass case 1: pass case _ as x: assert_never(x) # error: x is an int, not NoReturn # Motivation PEP 484 only allows NoReturn as a return type, but in practice all type checkers allow it in argument positions as a bottom type. Thus, a manually defined `assert_never()` function already has the behavior specified above. Adding the function to the standard library will make this behavior more discoverable for users. Relatedly, I opened https://bugs.python.org/issue46475 to document NoReturn being a general bottom type. # Open questions Suggestions welcome for better wording in the runtime error. There is a related proposal to rename NoReturn to Never. I'm lukewarm about that because renaming things always causes churn, but in any case let's discuss that separately.

6 8

Re: Proposal: typing.assert_never()
by Anton Agestam 08 Feb '22

08 Feb '22

@Jelle I really like this proposal, both the Never and the assert_never() parts. Some bike-shedding: there's an existing issue on the typing repository for this util, I added a comment there linking back to this thread and your PR. On that issue, it's proposed to call the helper assert_exhaustive(). I'd be happy with the tool in the stdlib either way, but I do feel like that name is better as it would be more likely to lead someone looking it up towards the concept of exhaustiveness checking.

1 0