Mailman 3 July 2022 - Typing-sig

Support for import hooks for static, type checkers and IDEs
by Bernat Gabor Aug. 16, 2022

Aug. 16, 2022

As some of you might be aware the python packaging ecosystem has adopted a standardized solution of editable installations in form of https://peps.python.org/pep-0660. Historically editable installations worked via `pth` files that inject the projects working directory onto the sys.path. This solution, while it mostly works, is fairly inflexible when it comes to more advanced functionality (such as excluding files from some modules, loading the code from in-memory or the database). A more powerful solution from the runtimes point of view is to use import hooks to dynamically allow/exclude module imports. The problem with this solution today is that static (and type) checkers are not capable of knowing/indexing the project layout, and as such most IDEs e.g. underline the imports with a red line as cannot be resolved from their POV. A possible suggestion was to allow the py.typed (or another file) to contain a rough layout of the project, e.g. a list of {module x is importable from file at path y}. This file then could be loaded and indexed by both type and static checkers, and periodically be refreshed by tooling for discovering new files/modules. See discussion at https://discuss.python.org/t/pep-660-support-and-ides/13878/13. Opinions or better options to tackle this problem? Thanks, Bernat

8 22

Support for sealed classes
by AbyxDev Aug. 9, 2022

Aug. 9, 2022

Hi Typing-SIG, Just wanting to revive this thread <https://mail.python.org/archives/list/typing-sig@python.org/thread/QSCT2N4R…>. I recently came across a use case for sealed classes in the process of making my mahjong game engine library <https://github.com/Kenny2github/mahjong> (still WIP), and was disappointed that no such concept was available. Specifically, mahjong/qna.py <https://github.com/Kenny2github/mahjong/blob/master/mahjong/qna.py> defines the Q&A API, where the idea is to check against each question subclass (see example in mahjong/__main__.py <https://github.com/Kenny2github/mahjong/blob/master/mahjong/__main__.py#L59…>) and answer the question with the correct type. Currently I can mimic sealed classes (in the typechecking sense) by defining an alias to a Union of the subclasses and annotating with that instead of the base class. By doing this, I can make assert_never(question) typecheck if I apply this patch: diff --git a/mahjong/__main__.py b/mahjong/__main__.py index b8da955..dfa760e 100644 --- a/mahjong/__main__.py +++ b/mahjong/__main__.py @@ -27,6 +27,7 @@ for p in game.players: #''' #''' # uncomment this opening triple quote when commenting out one elsewhere import sys +from typing_extensions import assert_never from mahjong.game import Game, Hand from mahjong import qna @@ -115,11 +116,13 @@ while question is not None: elif isinstance(question, qna.HandEnding): if isinstance(question, qna.Goulash): print('Goulash! Nobody wins. Starting next game...') - else: + elif isinstance(question, (qna.DealerWon, qna.NormalHandEnding)): print('Player #%s won with %s (%s faan; %s points; %s)! Starting next game...' % ( question.winner.seat.value, ','.join(map(str, question.choice)), question.faan()[0], *question.points(1) )) + else: + assert_never(question) question = question.answer() print('Game Over!') #''' On the other hand, if I remove the Union alias and change all references to HandEndingType to refer to HandEnding, Pylance reports the following: > Argument of type "HandEnding" cannot be assigned to parameter "__arg" of > type "NoReturn" in function "assert_never" > Type "HandEnding" cannot be assigned to type "NoReturn" > This problem is not solved by making HandEnding an ABC, because the fundamental issue is that other subclasses could be defined outside of the file that would pass an isinstance(question, qna.HandEnding) check. Being able to mark HandEnding and UserIO as sealed would allow the assert_never(question) call to typecheck without having to re-enumerate the subclasses in a Union alias. In addition, what would further help my use case is if subclasses of sealed classes were themselves sealable. Marking the PlayeredIO and ArrivedIO subclasses of UserIO as sealed would allow an assert_never(question) in the UserIO if-elif tower to typecheck as well. (The latter is supported by all languages that support sealed classes, but I feel like it's worth explicitly mentioning.) Additional properties I believe sealed classes should have: - Though I only have contrived examples for this, I feel like sealed classes should *not* be automatically marked abstract. A class that is marked sealed alone should be treated like a Union of the subclasses *and the base class*, to allow for the possibility of directly instantiating the base class. If the base class is not to be included in the Union, it should be explicitly and additionally marked abstract, which would then preclude it from being instantiated and thus exclude it from the sealed list. (Since Unions fold subclasses into their parent classes, this means that in practice the difference between an abstract sealed class and a non-abstract sealed class is that the former is equivalent to a Union of its subclasses, while the latter acts like a regular type and merely prevents further direct subclassing from typechecking.) - Following on, sealed abstract subclasses should not be included in the sealed list, as they are not directly instantiable. However, non-sealed abstract subclasses should still be included, because they can be subclassed outside of the file, so while no instance of the class itself will ever exist, an instance of a second-level subclass might. - Subclasses of sealed classes that are not sealed should not be automatically final. That is, if A is sealed, and B, C, and D inherit from it in the same file, it should be typecheckable to inherit from B, C, or D from another file unless they are explicitly marked final. As far as I can tell, this is consistent with all three languages mentioned above. Note: All final classes are sealed classes, since final is just sealed with no subclasses. However, sealed classes are not necessarily final. Prior art: - PEP 622 <https://peps.python.org/pep-0622/#sealed-classes-as-algebraic-data-types>, before it was split, would have introduced sealed classes. However, it implied that sealed classes would also be automatically abstract, because the examples did not include the base class in the equivalent Union. As mentioned above, I believe abstractness and sealing should be distinct. - John T. Hagen (CC'd, hope you don't mind) made a draft PEP <https://github.com/johnthagen/sealed-typing-pep> that includes some nice additional details. However, since its core idea is taken from PEP 622, it also suffers from the same conflation of abstractness and sealing. (It also isn't a proper fork of the python/peps <https://github.com/python/peps> repository, as demanded by the PEP submission process <https://peps.python.org/pep-0001/#submitting-a-pep>.) - Kotlin, Scala, and Java (as of certain versions) support sealed as a language construct. However, Kotlin forces sealing to imply abstractness <https://kotlinlang.org/docs/sealed-classes.html#:~:text=A%20sealed%20class%…>, while Scala <https://scastie.scala-lang.org/4PaP8zaGS5iNwqYQVks2Hg> and Java <https://openjdk.org/jeps/409#:~:text=public%20sealed%20class%20Rectangle> support the distinction between sealed class and sealed abstract class. (Links are to evidence.) This at least shows precedent for distinguishing between them. User-facing implementation and open questions: - Guido has already expressed his preference for a decorator over a base class or metaclass, since decorators can combine while metaclasses can't. I'd tend to agree on the merits. - Currently, @typing.final does not do any runtime prevention of subclassing. Does this mean @typing.sealed shouldn't either? It probably wouldn't be too much of a hassle to add a runtime=True parameter to the decorator (or something) that inserts an __init_subclass__ or something, but it could also just be left as a typechecking-only thing. I'm willing to draft up a proper PEP for this if that's appropriate. Beyond that, though, I just want to get some feedback, discussion, and thoughts going (again). Hopefully I can at least accomplish that! Best regards, AbyxDev; https://abyx.dev

12 32

Opened and changed typing issues week 2022-W29
by Typing Bot July 25, 2022

July 25, 2022

The following is an overview of all issues and pull requests in the typing repository on GitHub with the label 'topic: feature' that were opened or updated last week, excluding closed issues. --------------------------------------------------- The following issues and pull requests were updated last week: #548 Higher-Kinded TypeVars opened by @tek https://github.com/python/typing/issues/548 #795 TypedDict with patterns as key names opened by @abersheeran https://github.com/python/typing/issues/795 --------------------------------------------------- All issues and pull requests with the label 'topic: feature' can be viewed under the following URL: https://github.com/python/typing/issues?q=label%3A%22topic%3A+feature%22

1 0

Subtyping work
by luk-f-a＠outlook.com July 24, 2022

July 24, 2022

hi there! I was looking at the notes of the typing summit and saw the call to action to write a spec for subtyping. I've done some work on record and function subtyping for Numba and I'm very interested in the topic. I'd love to join whatever work might be going on for subtyping. Cheers, Luk ps: is Kevin Millikin on this list?

2 1

Re: Draft PEP for Improved Type Parameter Syntax
by James Duley July 23, 2022

July 23, 2022

I really like this PEP, it will really help my team to adopt type hints. The auto variance is great. > Regarding extensibility: Apart from defaults (which we will need in a foreseeable future), what features do other languages have for type variables that we don't yet? Coming from Java there are two features that I miss: 1. Lower bounds - Java has the `super` keyword for this. This doesn't stop any type hinting because you can always use a covariant/contravariant Protocol in place of an upper/lower bound as far as I can tell. It would be nice to avoid creating one-off Protocols, though. 2. Generic type bounds - in Java you can write something like this `class MyOtherGeneric<T, U extends MyGeneric<T>> {}`. My team has some python code like this but there's no way to type hint it as far as I know. It would be great if the scoping rules for these type variables don't preclude something like this being supported in the future `class ClassA[T, U: MyGeneric<T>]:`

1 0

Third-order decorators?
by Yonatan Zunger July 23, 2022

July 23, 2022

Hi everyone, I have a question about the typing system that may make heads explode a bit. It's certainly messing with mine. The mypy documentation gives a good summary of how you describe the type signature of second-order decorators. Let me stick in some names for these things to clarify the question: - A decorable (or decorated) function is TypeVar['F', bound=Callable[..., Any]] - A zero-argument decorator is a Callable[[F], F] - A multi-argument decorator is a Callable[[], ZeroArgumentDecorator] (it usually takes kwargs) - A polydecorator (cf the example in the mypy docs of a decorator that supports both bare calls and ones with arguments) is basically a union of the two; it's a callable which, if called with a single positional DecorableFunction argument, returns a DecorableFunction, and if called without that, returns a ZeroArgumentDecorator. The documentation shows how you can type-annotate polydecorators using @overload, and this is all well and good. But... writing nice polydecorators is a PITA, so as part of a tools library I'm working on, I'm writing a third-order decorator called @flex_decorator. This thing takes a "decorator spec" -- a function which takes a positional decorable function argument plus arbitrary kwargs and returns a decorated function -- and returns a polydecorator. This way, you can say something like @flex_decorator def my_decorator(target: Callable, *, arg1=foo, arg2=bar): .... do something and return a new Callable and then you can either say @my_decorator def some_function(...) which uses the default values for all the kwargs (and raises an error if some kwargs don't have defaults), or @my_decorator(arg1, arg2) def some_function(...) which sets values. This frees the user of my_decorator from having to remember whether it requires a () or not, and frees the author of my_decorator from having to handle all that delicate branching and overload annotation. And so I come to the problem: I need to figure out how to manage type signatures in the definition of flex_decorator. Internally, that function constructs a polydecorator out of the decorator spec and returns it. And inside the definition of flex_decorator itself, it's easy enough to put the @overload instructions on the nested function decoration. But what's the return type of flex_decorator? The problem I'm facing is that I'm not sure how to define a type signature, rather than a concrete function definition, with an overload. But if I don't do that, then when I actually use a flex_decorator as in the example above, a call like the last example above -- @my_decorator(foo, bar) def some_function(...) -- causes mypy to conclude that an "Untyped decorator makes function "some_function" untyped." To give a concrete example, one of the call sites looks like this: @flex_decorator def cachemethod( function: WrappedFunctionType, *, cache: MethodCacheArgument = dict, lock: MethodLockArgument = False, key: Optional[Callable[..., KeyType]] = None, cache_exceptions: bool = False, **kwargs: Any, ) -> '_WrappedDescriptor': The resulting cachemethod object should end up with the type signature @overload def cachemethod(function: WrappedFunctionType) -> _WrappedDescriptor: ... @overload def cachemethod(*, cache: MethodCacheArgument = dict, ....) -> Callable[[WrappedFunctionType], _WrappedDescriptor]: ... But I'm not sure how flex_decorator can best cause that to be its type signature, because stuffing @overload defs into something else's def isn't quite well-specified. Does anyone have any idea of how to handle this? I'd really like to be able to provide this functionality to users without completely breaking their ability to do type-checking!

3 2

Opened and changed typing issues week 2022-W28
by Typing Bot July 18, 2022

July 18, 2022

No issues or pull requests with the label 'topic: feature' were opened or updated last week in the typing repository on GitHub. All issues and pull requests with the label 'topic: feature' can be viewed under the following URL: https://github.com/python/typing/issues?q=label%3A%22topic%3A+feature%22

1 0

are unions ordered?
by Martin DeMello July 11, 2022

July 11, 2022

Ran into an interesting user issue when unifying Union[T, List[T]] against List[str]. The binding of T depends on whether you express the type as Union[T, List[T]] or Union[List[T], T] since str matches List[str]. We've been doing the simplest thing and matching against the types in the order they appear in the union until the first one matches, but now I'm wondering if unions are conceptually unordered and we should use a smarter algorithm. martin

8 8

Opened and changed typing issues week 2022-W27
by Typing Bot July 11, 2022

July 11, 2022

The following is an overview of all issues and pull requests in the typing repository on GitHub with the label 'topic: feature' that were opened or updated last week, excluding closed issues. --------------------------------------------------- The following issues and pull requests were updated last week: #1160 Allow `float("inf")` and `float("-inf")` in literals opened by @antonagestam https://github.com/python/typing/issues/1160 --------------------------------------------------- All issues and pull requests with the label 'topic: feature' can be viewed under the following URL: https://github.com/python/typing/issues?q=label%3A%22topic%3A+feature%22

1 0

PyCon 2022 Typing Summit Recap
by Shannon Zhu July 10, 2022

July 10, 2022

Hello everyone, I’m excited to share some resources from the 2022 Typing Summit! The summit was held last week, on 4/28 at PyCon in Salt Lake City. It was in-person this year without support for live streaming, so we know a lot of you couldn’t fly out but are still interested in catching up with the topics discussed. We took some best-effort video and notes for each of the presentations and discussions, and some of the speakers have also uploaded their slides and speaker notes to share. You can find links to video and slides per topic (where available), as well as paraphrased audience-member notes and action-items here: https://docs.google.com/document/d/1_CBM9SeBrFTFULtwAzGmxlBRhqzpIArQ65cWa2W… Please thank Matthew Rahtz for personally taking and cutting video of every topic. They are linked individually in the doc above, and you can find them all in one place here: https://drive.google.com/drive/folders/1IVV0URv7_CCxiOvY6GC6adB72d5LKcvI<https://drive.google.com/drive/folders/1IVV0URv7_CCxiOvY6GC6adB72d5LKcvI> We will be uploading a full-length video of the typing summit to PyCon’s YouTube channel in the coming days. Note: Unfortunately due to battery life and other technical limitations, the video slides are hard to read, and we missed the last bit of the panel discussion. If anyone else at the summit had taken notes or wanted to share takeaways or further questions, please feel free to respond to this thread. 😊 Thanks again to all of the presenters for the work you’ve put in, and thanks to everyone who made it out in person to listen and discuss! It’s exciting to see so many typing related features already going into 3.11 and all the potential for future changes to improve the typing experience in Python. Best, Shannon & Alfonso

6 5