Mailman 3 August 2022 - Typing-sig

Creating types with functions/classes
by Patrick Arminio Aug. 23, 2022

Aug. 23, 2022

Hello everyone! I want to see what's your opinion on allowing functions to return types and allow them to be used by a mypy and other type checkers. For context, I have a library that allows to annotate function arguments with some metadata, and it looks similar to this: ```python def argument(name: str, description: str) -> dict: return {"name": name, "description": description} def get_user_name(id_: Annotated[int, argument(name="id", description="this should be an uuid")]) -> str | None: return None ``` this works fine, but I was wondering if we could do something (or allow) like this: ```python def argument(type_: T, *, name: str, description: str) -> T: return Annotated[type_, {"name": name, "description": description}] def get_user_name(id_: argument(int, name="id", description="this should be an uuid")) -> str | None: return None ``` it might seem like a minor change, but to me it feels like it could improve the dx of my library 😊 Just to clarify, the goal is not to allow dynamic types, it is more to have a convenient way of making an alias to lazy type Let me know what you think

2 1

Opened and changed typing issues week 2022-W33
by Typing Bot Aug. 22, 2022

Aug. 22, 2022

The following is an overview of all issues and pull requests in the typing repository on GitHub with the label 'topic: feature' that were opened or updated last week, excluding closed issues. --------------------------------------------------- The following issues and pull requests were updated last week: #1043 TypingError: type checker will emit error with a message specified in typing stubs opened by @Akuli https://github.com/python/typing/issues/1043 --------------------------------------------------- All issues and pull requests with the label 'topic: feature' can be viewed under the following URL: https://github.com/python/typing/issues?q=label%3A%22topic%3A+feature%22

1 0

Explicit Type Aliases
by Shannon Zhu Aug. 17, 2022

Aug. 17, 2022

Hi everyone, I’d like to continue the discussion from our last typing summit on explicit type aliases. A quick summary of the current state and the proposal -- Current state: ``` from typing import List x = List[int] # considered a type alias y : Type[List[int]] = List[int] # considered an expression z : Any # considered a type alias z : x = [] # fine z : y = [] # invalid type error ``` Proposal (with explicit aliases): ``` from typing import List, TypeAlias x = TypeAlias[List[int]] # considered a type alias y = List[int] # considered an expression z : Any # considered an expression z = TypeAlias[Any] # considered a type alias reveal_type(x) # Type[List[int]] reveal_type(y) # Any (return type of __getitem__) z : x = [] # fine z : y = [] # invalid type error ``` Some of the benefits we’re hoping to gain from explicit type aliases: * Clearly distinguish between an unannotated global assignment and a type alias, especially when parsing forward-referencing string annotations. * Avoid the confusing case of type aliases in which some part of the type is invalid or undefined. This would facilitate warnings on invalid types at the alias definition rather than later on when the alias is used. * Make valid and invalid types more intuitive to Python programmers by shifting from delineation of non-aliases with a meta annotation toward delineation of aliases with `TypeAlias`. * Remove special treatment of values annotated as `Any`, which currently break all type aliasing rules. This will also allow us to clean up some of the distinctions we maintain between unannotated values and explicit Anys. Note: We also considered denoting the alias as an annotation (e.g. `x: TypeAlias = int`) as an alternative to the syntax above. I’m interested to hear your thoughts and suggestions on making type aliasing explicit going forward – we’d like to start implementing a version of this in Pyre soon! Shannon

7 10

PEP 692: Using TypedDict for more precise **kwargs typing
by Franek Magiera Aug. 16, 2022

Aug. 16, 2022

Adding support for more precise **kwargs (consisting of different types) typing using TypedDict has been a very desired feature in python community [1]. The feature proposal has been discussed both on the mailing list and at a typing meetup and received positive feedback. I've submitted PEP 692 that aims to add support for more precise **kwargs typing. The PR is available at https://github.com/python/peps/pull/2620 And the preview at https://pep-previews--2620.org.readthedocs.build/pep-0692/ - **kwargs can consist of different types - A new syntax `def foo(**kwargs: **Movie): ...` that enables the feature is introduced Please let me know what you think about it. Franek [1] https://github.com/python/mypy/issues/4441

10 16

Support for import hooks for static, type checkers and IDEs
by Bernat Gabor Aug. 16, 2022

Aug. 16, 2022

As some of you might be aware the python packaging ecosystem has adopted a standardized solution of editable installations in form of https://peps.python.org/pep-0660. Historically editable installations worked via `pth` files that inject the projects working directory onto the sys.path. This solution, while it mostly works, is fairly inflexible when it comes to more advanced functionality (such as excluding files from some modules, loading the code from in-memory or the database). A more powerful solution from the runtimes point of view is to use import hooks to dynamically allow/exclude module imports. The problem with this solution today is that static (and type) checkers are not capable of knowing/indexing the project layout, and as such most IDEs e.g. underline the imports with a red line as cannot be resolved from their POV. A possible suggestion was to allow the py.typed (or another file) to contain a rough layout of the project, e.g. a list of {module x is importable from file at path y}. This file then could be loaded and indexed by both type and static checkers, and periodically be refreshed by tooling for discovering new files/modules. See discussion at https://discuss.python.org/t/pep-660-support-and-ides/13878/13. Opinions or better options to tackle this problem? Thanks, Bernat

8 22

Opened and changed typing issues week 2022-W32
by Typing Bot Aug. 15, 2022

Aug. 15, 2022

The following is an overview of all issues and pull requests in the typing repository on GitHub with the label 'topic: feature' that were opened or updated last week, excluding closed issues. --------------------------------------------------- The following issues and pull requests were updated last week: #1231 TypeVarTuple with Callable and default arguments opened by @cdce8p https://github.com/python/typing/issues/1231 --------------------------------------------------- All issues and pull requests with the label 'topic: feature' can be viewed under the following URL: https://github.com/python/typing/issues?q=label%3A%22topic%3A+feature%22

1 0

Support for sealed classes
by AbyxDev Aug. 9, 2022

Aug. 9, 2022

Hi Typing-SIG, Just wanting to revive this thread <https://mail.python.org/archives/list/typing-sig@python.org/thread/QSCT2N4R…>. I recently came across a use case for sealed classes in the process of making my mahjong game engine library <https://github.com/Kenny2github/mahjong> (still WIP), and was disappointed that no such concept was available. Specifically, mahjong/qna.py <https://github.com/Kenny2github/mahjong/blob/master/mahjong/qna.py> defines the Q&A API, where the idea is to check against each question subclass (see example in mahjong/__main__.py <https://github.com/Kenny2github/mahjong/blob/master/mahjong/__main__.py#L59…>) and answer the question with the correct type. Currently I can mimic sealed classes (in the typechecking sense) by defining an alias to a Union of the subclasses and annotating with that instead of the base class. By doing this, I can make assert_never(question) typecheck if I apply this patch: diff --git a/mahjong/__main__.py b/mahjong/__main__.py index b8da955..dfa760e 100644 --- a/mahjong/__main__.py +++ b/mahjong/__main__.py @@ -27,6 +27,7 @@ for p in game.players: #''' #''' # uncomment this opening triple quote when commenting out one elsewhere import sys +from typing_extensions import assert_never from mahjong.game import Game, Hand from mahjong import qna @@ -115,11 +116,13 @@ while question is not None: elif isinstance(question, qna.HandEnding): if isinstance(question, qna.Goulash): print('Goulash! Nobody wins. Starting next game...') - else: + elif isinstance(question, (qna.DealerWon, qna.NormalHandEnding)): print('Player #%s won with %s (%s faan; %s points; %s)! Starting next game...' % ( question.winner.seat.value, ','.join(map(str, question.choice)), question.faan()[0], *question.points(1) )) + else: + assert_never(question) question = question.answer() print('Game Over!') #''' On the other hand, if I remove the Union alias and change all references to HandEndingType to refer to HandEnding, Pylance reports the following: > Argument of type "HandEnding" cannot be assigned to parameter "__arg" of > type "NoReturn" in function "assert_never" > Type "HandEnding" cannot be assigned to type "NoReturn" > This problem is not solved by making HandEnding an ABC, because the fundamental issue is that other subclasses could be defined outside of the file that would pass an isinstance(question, qna.HandEnding) check. Being able to mark HandEnding and UserIO as sealed would allow the assert_never(question) call to typecheck without having to re-enumerate the subclasses in a Union alias. In addition, what would further help my use case is if subclasses of sealed classes were themselves sealable. Marking the PlayeredIO and ArrivedIO subclasses of UserIO as sealed would allow an assert_never(question) in the UserIO if-elif tower to typecheck as well. (The latter is supported by all languages that support sealed classes, but I feel like it's worth explicitly mentioning.) Additional properties I believe sealed classes should have: - Though I only have contrived examples for this, I feel like sealed classes should *not* be automatically marked abstract. A class that is marked sealed alone should be treated like a Union of the subclasses *and the base class*, to allow for the possibility of directly instantiating the base class. If the base class is not to be included in the Union, it should be explicitly and additionally marked abstract, which would then preclude it from being instantiated and thus exclude it from the sealed list. (Since Unions fold subclasses into their parent classes, this means that in practice the difference between an abstract sealed class and a non-abstract sealed class is that the former is equivalent to a Union of its subclasses, while the latter acts like a regular type and merely prevents further direct subclassing from typechecking.) - Following on, sealed abstract subclasses should not be included in the sealed list, as they are not directly instantiable. However, non-sealed abstract subclasses should still be included, because they can be subclassed outside of the file, so while no instance of the class itself will ever exist, an instance of a second-level subclass might. - Subclasses of sealed classes that are not sealed should not be automatically final. That is, if A is sealed, and B, C, and D inherit from it in the same file, it should be typecheckable to inherit from B, C, or D from another file unless they are explicitly marked final. As far as I can tell, this is consistent with all three languages mentioned above. Note: All final classes are sealed classes, since final is just sealed with no subclasses. However, sealed classes are not necessarily final. Prior art: - PEP 622 <https://peps.python.org/pep-0622/#sealed-classes-as-algebraic-data-types>, before it was split, would have introduced sealed classes. However, it implied that sealed classes would also be automatically abstract, because the examples did not include the base class in the equivalent Union. As mentioned above, I believe abstractness and sealing should be distinct. - John T. Hagen (CC'd, hope you don't mind) made a draft PEP <https://github.com/johnthagen/sealed-typing-pep> that includes some nice additional details. However, since its core idea is taken from PEP 622, it also suffers from the same conflation of abstractness and sealing. (It also isn't a proper fork of the python/peps <https://github.com/python/peps> repository, as demanded by the PEP submission process <https://peps.python.org/pep-0001/#submitting-a-pep>.) - Kotlin, Scala, and Java (as of certain versions) support sealed as a language construct. However, Kotlin forces sealing to imply abstractness <https://kotlinlang.org/docs/sealed-classes.html#:~:text=A%20sealed%20class%…>, while Scala <https://scastie.scala-lang.org/4PaP8zaGS5iNwqYQVks2Hg> and Java <https://openjdk.org/jeps/409#:~:text=public%20sealed%20class%20Rectangle> support the distinction between sealed class and sealed abstract class. (Links are to evidence.) This at least shows precedent for distinguishing between them. User-facing implementation and open questions: - Guido has already expressed his preference for a decorator over a base class or metaclass, since decorators can combine while metaclasses can't. I'd tend to agree on the merits. - Currently, @typing.final does not do any runtime prevention of subclassing. Does this mean @typing.sealed shouldn't either? It probably wouldn't be too much of a hassle to add a runtime=True parameter to the decorator (or something) that inserts an __init_subclass__ or something, but it could also just be left as a typechecking-only thing. I'm willing to draft up a proper PEP for this if that's appropriate. Beyond that, though, I just want to get some feedback, discussion, and thoughts going (again). Hopefully I can at least accomplish that! Best regards, AbyxDev; https://abyx.dev

12 32

Opened and changed typing issues week 2022-W31
by Typing Bot Aug. 8, 2022

Aug. 8, 2022

The following is an overview of all issues and pull requests in the typing repository on GitHub with the label 'topic: feature' that were opened or updated last week, excluding closed issues. --------------------------------------------------- The following issues and pull requests were opened last week: #1231 TypeVarTuple with Callable and default arguments opened by @cdce8p https://github.com/python/typing/issues/1231 #1232 Option to type default default arguments in Callable types - `WithDefault` opened by @cdce8p https://github.com/python/typing/issues/1232 --------------------------------------------------- The following issues and pull requests were updated last week: #801 Introduce a Not type opened by @antonagestam https://github.com/python/typing/issues/801 --------------------------------------------------- All issues and pull requests with the label 'topic: feature' can be viewed under the following URL: https://github.com/python/typing/issues?q=label%3A%22topic%3A+feature%22

1 0

A more generic solution to the @dataclass problem?
by Buck Evan Aug. 5, 2022

Aug. 5, 2022

When I first stubbed my toe on dataclass typing the code I tried was this: ``` def mydataclass(...): ... return dataclasses.dataclass(...) mydataclass = typing.cast('dataclasses.dataclass', mydataclass) ``` I see now that this was wrong and naive, but perhaps you can see my intent there. I was trying to hint the type system that it should treat mydataclass exactly as it treats `@dataclass`. Do any of you see value in making such a thing work? It would solve a address a good deal more problems than PEP-681, in that it could help with custom properties, custom Enum, and the list goes on. More concretely, I propose a `typing.cast_by_value` (naming is hard) that will have the semantics of replacing itself with its first argument, as far as typechecking is concerned. This adds missing and needed functionality, because it's currently impossible to create anything other than `__builtins__.property` that is treated exactly like `__builtins__.property` by typechecking, especially in a standards-based way that will reliably work across type checking implementations. If my `property` example is a bad one, please use the strongest of these examples: `enum.Enum`, `dataclasses.dataclass`, `namedtuple`, `NamedTuple`.

4 6

Should plain-old users be enabled to define new types?
by Buck Evan Aug. 5, 2022

Aug. 5, 2022

Python has done a good job of ensuring that stdlib libraries and other code have equivalent power. If you move any stdlib module out of stdlib, it behaves exactly the same and is exactly as useful, either way. I'm worried that typing-SIG is not considering this in their designs. Consider: in an alternate universe python3.11 does not contain `dataclasses` because it hasn't been invented yet. There are many semantics of dataclasses that have no way to be represented in the current type system. But should an enterprising user be enabled to create such a thing, and provide strong typing for it? I hope your answer is yes. Currently, most python typecheckers answer this issue with some kind of programmable extension that users can use to add out-of-type semantics to code: `mypy` has extensive plugins, `pytype` has "overlays". Notably [`pyright` offers no plugin API], and plans to never have any. I believe their reasoning is strong though: such extensions would necessarily not be portable to other typecheckers, and also represent deficiencies in the type system / standards that should be brought to light and worked on. That said, how could an out-of-tree `dataclasses` be given its necessary type semantics, even in principle? I'll leave the question open-ended, as I think your answers will be better than mine. [`pyright` offers no plugin API]: https://github.com/microsoft/pyright/issues/607

2 1