Mailman 3 April 2022 - Typing-sig

Support Algebraic Data Types with @sealed
by John Hagen 20 Mar '23

20 Mar '23

The original (superseded) PEP 622 proposed adding a @sealed decorator so that users could define algebraic data types (ADTs) that static type checkers could verify are `match`ed exhaustively: - https://peps.python.org/pep-0622/#sealed-classes-as-algebraic-data-types There is a somewhat fairly-upvoted question about ADTs in Python, indicating interest in this area: - https://stackoverflow.com/questions/16258553 @sealed would be very valuable towards making ADT usage within Python more type-safe. Is there any historical discussion about why @sealed was dropped from the final pattern matching PEP? What would be the next steps needed to push this forward? Would writing a new PEP be the next step?

15 32

Any way to express that a dict is expected to have a key(s) but don't care about other keys?
by Brett Cannon 23 Dec '22

23 Dec '22

Looking at TypedDict it seems that it does not like the idea of superfluous keys existing in a dict. In my case I have a REST call returning JSON for an error and all I'm guaranteed is to get an "error" key. But there may be extra keys that provide more details about the specific error. Unfortunately I don't know the exhaustive list of keys so I can't use a TypedDict with `totality == False` to try to be complete. I can't think of any way to somehow say "require/expect this one key, but other keys are OK". Am I missing anything that would let me do this?

16 34

Re: Typing Meetup
by S Pradeep Kumar 15 Nov '22

15 Nov '22

Updating the Typing Meetup date to Wednesday as there were some conflicts. Let me know if there are any major time conflicts. Date: *Wednesday, November 03, 2021.*Time: 10 am San Francisco time (UTC-7) / 5 pm London time (UTC) Duration: 1 hour Potential discussion topics: + Typing **kwargs using a TypedDict (Franek Magiera?, Pradeep) [1] + Remaining questions about Callable syntax (Steven) + Non-required Protocol fields (Sebastian Rittau?) [2] + Type support for sealed classes (Adrian Freund?) [3] + Safe, typed JSON validation + Type system documentation (Shannon?) If you're working on these or any other topics and would like to start a discussion at the meetup, reach out to me. If you have interesting topics you'd like to see discussed, share them too. Once we have topics, we can always start a discussion even if there's no proposer yet. We will probably have time for 3-4 topics in this session. On Fri, Oct 15, 2021 at 12:55 PM S Pradeep Kumar <gohanpra(a)gmail.com> wrote: >>> >>>> Hello all, >>>> >>>> We plan to hold a virtual Typing Meetup to discuss key typing-related >>>> issues in real-time. The idea is that we can make progress on important or >>>> controversial discussions, get early feedback on in-progress PEPs, and >>>> gauge interest in new proposals. >>>> >>>> A virtual meetup will be more accessible to people in other timezones >>>> than Bay Area in-person meetups. It should also help consolidate >>>> discussions that are often spread out across mailing lists or Github repos. >>>> The tentative plan is to have a monthly meetup, but we can change that as >>>> needed. >>>> >>>> Date: Monday, November 01, 2021. >>>> Time: 10 am San Francisco time (UTC-7) / 5 pm London time (UTC) >>>> Duration: 1 hour >>>> >>>> Time and date are flexible. I'll share the zoom link and agenda the >>>> week before. I'll also record the meetup and share afterwards. >>>> >>>> If you're interested in presenting a new proposal or your work on some >>>> existing proposal, please respond here. We can shoot for 15 minutes per >>>> presentation. I know there are quite a few interesting threads on >>>> typing-sig that haven't seen much activity in a while :) >>>> -- >>>> S Pradeep Kumar >>>> _______________________________________________ >>>> Typing-sig mailing list -- typing-sig(a)python.org >>>> To unsubscribe send an email to typing-sig-leave(a)python.org >>>> https://mail.python.org/mailman3/lists/typing-sig.python.org/ >>>> Member address: guido(a)python.org >>> >>> Links: [1] Typing **kwargs using TypedDict: https://mail.python.org/archives/list/typing-sig@python.org/thread/IUCXT2IQ… [2] Non-required Protocol fields: https://mail.python.org/archives/list/typing-sig@python.org/thread/P6FCGRJU… [3] Sealed classes: https://mail.python.org/archives/list/typing-sig@python.org/thread/QSCT2N4R… -- S Pradeep Kumar

8 51

Defaults for TypeVar-likes
by James H-B 03 Nov '22

03 Nov '22

I've been toying with the idea of writting a PEP for this long requested feature (https://github.com/python/typing/issues/307). I've got a preliminary draft of it with the motivation and a specification for the feature. In short features like ```py T = TypeVar("T", default=int) # This means that if no type is specified T = int @dataclass class Box(Generic[T]): value: T | None = None reveal_type(Box()) # type is Box[int] reveal_type(Box(value="Hello World!")) # type is Box[str] ``` would be supported as would ```py YieldT = TypeVar("YieldT") SendT = TypeVar("SendT", default=None) ReturnT = TypeVar("ReturnT", default=None) class Generator(Generic[YieldT, SendT, ReturnT]): ... Generator[int] == Generator[int, None] == Generator[int, None, None] ``` Feedback and any questions on this would be very appreciated. https://gist.github.com/Gobot1234/8c9bfe8eb88f5ad42bf69b6f118033a7 Thanks, James H-B.

8 20

Type narrowing for TypeGuard in the negative case
by Eric Traut 29 Sep '22

29 Sep '22

Since the introduction of PEP 647 (User-defined Type Guards), we've received a steady stream of input from users saying that they don't like the limitation that type narrowing is applied only in the positive case and is not applied in the negative case. In general, type narrowing is not safe to perform if a user-defined type guard returns False, so I've stood by the original decision not to provide type narrowing in the negative case, but there are cases where such narrowing is safe and desirable. Here's a recent thread where this was discussed in some detail: https://github.com/python/typing/issues/996#issuecomment-1002716830 In this discussion, Ilya Kamen proposed a solution. The proposal is to extend the existing `TypeGuard` to support an optional second type argument. If present, the second argument indicates the narrowed type in the negative type narrowing situation. Here's a simple (admittedly contrived) example: ```python def is_str(val: str | int) -> TypeGuard[str, int]: return isinstance(val, str) def func(val: str | int): if is_str(val): reveal_type(val) # str else: reveal_type(val) # int ``` I've implemented this proposal in the latest published version of pyright (1.1.202) so folks can experiment with it and see if they like it. If there's general consensus that it's the right approach, I can file an amendment for the existing PEP 647 to include this functionality. It was trivial to add to pyright, so I'm optimistic that it would likewise be easy to add to the other type checkers. Another common request for PEP 647 is the desire for an "assert" form of TypeGuard — a way to indicate that a function performs runtime validation of a type, raising an exception if the type is incorrect. It occurred to me that this two-argument form of `TypeGuard` could also be used to handle this use case. The second argument would be specified as a `NoReturn`. I've provisionally implemented this in pyright as well (although it's not in the currently-published version — you'll need to wait until 1.1.203 is published). Here's what this would look like: ```python def validate_sequence(val: Sequence[_T | None]) -> TypeGuard[Sequence[_T], NoReturn]: """Verifies that the sequence contains no None values""" if len([x for x in val if x is not None]) > 0: raise Exception() return True def func(val: Sequence[int | None]): validate_sequence(val) reveal_type(val) # Sequence[int] ``` I'm interested in input on these proposals. -Eric -- Eric Traut Contributor to Pyright & Pylance Microsoft

6 7

Explicit Type Aliases
by Shannon Zhu 17 Aug '22

17 Aug '22

Hi everyone, I’d like to continue the discussion from our last typing summit on explicit type aliases. A quick summary of the current state and the proposal -- Current state: ``` from typing import List x = List[int] # considered a type alias y : Type[List[int]] = List[int] # considered an expression z : Any # considered a type alias z : x = [] # fine z : y = [] # invalid type error ``` Proposal (with explicit aliases): ``` from typing import List, TypeAlias x = TypeAlias[List[int]] # considered a type alias y = List[int] # considered an expression z : Any # considered an expression z = TypeAlias[Any] # considered a type alias reveal_type(x) # Type[List[int]] reveal_type(y) # Any (return type of __getitem__) z : x = [] # fine z : y = [] # invalid type error ``` Some of the benefits we’re hoping to gain from explicit type aliases: * Clearly distinguish between an unannotated global assignment and a type alias, especially when parsing forward-referencing string annotations. * Avoid the confusing case of type aliases in which some part of the type is invalid or undefined. This would facilitate warnings on invalid types at the alias definition rather than later on when the alias is used. * Make valid and invalid types more intuitive to Python programmers by shifting from delineation of non-aliases with a meta annotation toward delineation of aliases with `TypeAlias`. * Remove special treatment of values annotated as `Any`, which currently break all type aliasing rules. This will also allow us to clean up some of the distinctions we maintain between unannotated values and explicit Anys. Note: We also considered denoting the alias as an annotation (e.g. `x: TypeAlias = int`) as an alternative to the syntax above. I’m interested to hear your thoughts and suggestions on making type aliasing explicit going forward – we’d like to start implementing a version of this in Pyre soon! Shannon

7 10

Support for import hooks for static, type checkers and IDEs
by Bernat Gabor 16 Aug '22

16 Aug '22

As some of you might be aware the python packaging ecosystem has adopted a standardized solution of editable installations in form of https://peps.python.org/pep-0660. Historically editable installations worked via `pth` files that inject the projects working directory onto the sys.path. This solution, while it mostly works, is fairly inflexible when it comes to more advanced functionality (such as excluding files from some modules, loading the code from in-memory or the database). A more powerful solution from the runtimes point of view is to use import hooks to dynamically allow/exclude module imports. The problem with this solution today is that static (and type) checkers are not capable of knowing/indexing the project layout, and as such most IDEs e.g. underline the imports with a red line as cannot be resolved from their POV. A possible suggestion was to allow the py.typed (or another file) to contain a rough layout of the project, e.g. a list of {module x is importable from file at path y}. This file then could be loaded and indexed by both type and static checkers, and periodically be refreshed by tooling for discovering new files/modules. See discussion at https://discuss.python.org/t/pep-660-support-and-ides/13878/13. Opinions or better options to tackle this problem? Thanks, Bernat

8 22

Almost accepting PEP 681 – Data Class Transforms
by Petr Viktorin 06 Jun '22

06 Jun '22

Hello, As an initial implementation that will be improved in the future, the specification in PEP 681 is fine. Feel free to add the decorator to Python 3.11 at your convenience. However, the PEP includes several worrying recommendations like: - we recommend that the maintainers of attrs move away from the legacy semantics and adopt auto_attribs behaviors by default. - We chose not to support this feature and recommend that attrs users avoid converters. - Attrs users should use the dataclass-standard eq and order parameter names instead. These are probably meant as recommendations from typing-sig, but an accepted PEP represents consensus of the entire Python community. A typing PEP is not an appropriate place to make recommendations like this, especially without reaching out to the maintainer of attrs. As far as I know,the attrs and pydantic libraries are using the reference implementation, but their authors weren't consulted on the PEP itself. Could you either change the wording (e.g. say that the unsupported features need bespoke type-checker functionality for proper type checking), or work with attrs to make the same recommendations in its documentation? Happy typing, — Petr, on behalf of the Steering Council

9 11

Generic NamedTuples
by Jelle Zijlstra 07 May '22

07 May '22

At the moment, this will throw a runtime error: from typing import Generic, NamedTuple, TypeVar T = TypeVar("T") class NT(NamedTuple, Generic[T]): foo: int bar: T There is an open CPython PR to add runtime support for this pattern: https://github.com/python/cpython/pull/92027. I'm inclined to accept this change, because it seems like a natural extension and would work well with the rest of the type system. Mypy doesn't currently support this for internal reasons, but I'm told Pyre already supports it. Let me know if there's some problem I missed with this idea. We just discussed this at the typing summit at PyCon, and one concern was brought up about variance. I think variance should work the same as with non-NamedTuple generic classes and no special treatment is required. Relatedly, there is another open PR that allows generalized multiple inheritance on NamedTuple (https://github.com/python/cpython/pull/31781). This would allow code like this: class A: @property def prop(self) -> int: ... class B(NamedTuple, A): foo: int assert_type(B(1).prop, int) I think this is too much to ask static type checkers to support: regular classes are too different from namedtuples. But I'd be happy to hear other opinions, especially from type checker maintainers.

6 11

PEP 688: Making the buffer protocol accessible in Python
by Jelle Zijlstra 25 Apr '22

25 Apr '22

It's been a longstanding problem in typeshed that we cannot fully represent the C buffer protocol in stubs, and we have to make do with annotations of `bytes` or unions of common buffer types. I just submitted PEP 688 to add a new `types.Buffer` type that will fix this problem. A preview of the rendered PEP is available at https://pep-previews--2549.org.readthedocs.build/pep-0688/. The PR is at https://github.com/python/peps/pull/2549. Summary: * types.Buffer is added to CPython, using an isinstance hook to check for the presence of the buffer protocol. * In stubs, types.Buffer can be used as a base class. * Static type checkers should not need to special case the new class in any way. Please let me know if you have any feedback.

2 3

Typing-sig April 2022