Mailman 3 March 2023 - Typing-sig

Extending published stubs
by Irv Lustig 27 Apr '23

27 Apr '23

I'm the leader of the pandas-stubs project. In pandas, users can extend the library by creating their own accessor via `(a)pd.api.extensions.register_dataframe_accessor` (see https://pandas.pydata.org/docs/reference/api/pandas.api.extensions.register…). So for a `DataFrame`, if you do `(a)pd.api.extensions.register_dataframe_accessor("xyz")`, you now have `DataFrame.xyz` returning the class you annotated. Now suppose you want to create a stub for your new accessor, that would extend the existing pandas-stubs. AFAIK, there isn't a way to do something like this in a stub: ``` from pandas import DataFrame @typing.extend class DataFrame: def xyz(self) -> MyXyzClass: ... ``` The idea here is that there is a class (in this case, `DataFrame`) that already has stubs, and you want to add a method to it. Has this idea been discussed anywhere?

6 11

Inline syntax for TypedDict
by Никита Соболев 23 Apr '23

23 Apr '23

Hi everyone! ## Problem Dicts are fundamental to Python programming. And today I found myself writing this piece of code (again): ```python def process_user(user_dict: User) -> ModifiedUser: ... class User(TypedDict): ... class ModifiedUser(User): ... ``` While in reality it was as simple as: `{'user_id': int}` -> `{'user_id': int, 'external_id': int}` What really bothers me is that I have to either: 1. Create a new whole class for it 2. Create a new type variable in a form of `User = TypedDict('User', {...})` I think that it hurts the readability quite a lot, because the definition of this type might not be near with a function that uses it. Another problem is that due to the nature of `TypedDict`s there are quite a lot of them in a regular web application, because of json. The biggest problem (for me personally) is that you cannot simply nest TypedDict. So, if you have a nested dict that you want to use, like `{'user': {'email': str}}`? You have to define *two* classes. And the final touch: it looks like many TypedDict are just used in one place. They are just like throw-away definitions. And don't forget that classes cannot have all keys in their body: `return`, `@1`, etc are all not allowed. ## Proposal To solve all the problems above, I propose adding "inline" syntax for typed dicts: ```python def get_user_data(arg: {'user': {'email': str}}) ->{'email': str}: return user['user'] ``` It has great potential: 0. Grammar already allows that with no changes from our side 1. `Required` and `NotRequired` can be used to enforce the presence of keys and values 2. Dicts can be nested with ease, just like in my example. Even multiline nesting is supported: ```python >>> def multiline() -> { ... 'user': { ... 'email': str, ... }, ... }: ... ... ... ``` 3. The intent is pretty clear: any user familiar with dicts would be able to read and understand this code 4. Other languages already have this feature, TS example: https://www.typescriptlang.org/play?#code/GYVwdgxgLglg9mABAcwKZQPogM6oE4YAm… 5. Existing tooling continues to work, see this example: ```python >>> import typing >>> def get_user_data(arg: {'user': {'email': str}}) ->{'email': str}: ... ... >>> get_user_data.__annotations__ {'arg': {'user': {'email': <class 'str'>}}, 'return': {'email': <class 'str'>}} >>> typing.get_type_hints(get_user_data) {'arg': {'user': {'email': <class 'str'>}}, 'return': {'email': <class 'str'>}} ``` And: ```python >>> import typing >>> class Some: ... a: {'x': int} = {'x': 1} ... def method(self, arg: {'x': int}) -> {'x': int, 'y': int}: ... return {'x': 1, 'y': 2} ... >>> Some.__annotations__ {'a': {'x': <class 'int'>}} >>> Some.method.__annotations__ {'arg': {'x': <class 'int'>}, 'return': {'x': <class 'int'>, 'y': <class 'int'>}} >>> typing.get_type_hints(Some) {'a': {'x': <class 'int'>}} >>> typing.get_type_hints(Some.method) {'arg': {'x': <class 'int'>}, 'return': {'x': <class 'int'>, 'y': <class 'int'>}} ``` 6. It is just a regular dict! No new things to create :) ## Extra We can also discuss the variable syntax change from: ```python X = TypedDict('X', {'a': int}) ``` To: ```python X: TypeAlias = {'a': int} ``` Because again, it is simpler: no need to duplicate the name and values can be nested. ## Problems The most important problem I see is that it is just a syntax to solve a problem with an existing solution. But, I think it is a good addition overall. ## Typecheckers From what I know, it would not be too hard to support this in mypy. We would need to add new logic to generate names from context (instead of an explicit name). Nesting needs to be tested, we might have some troubles here. I cannot speak for other typecheckers. ## Feedback This is a very rough idea for now. But, what do you think? Is that a problem you face in your projects? I would be glad to hear the feedback from the Python's typing community :) Best, Nikita Sobolev

13 24

Opened and changed typing issues week 2023-W12
by Typing Bot 27 Mar '23

27 Mar '23

The following is an overview of all issues and pull requests in the typing repository on GitHub with the label 'topic: feature' that were opened or updated last week, excluding closed issues. --------------------------------------------------- The following issues and pull requests were updated last week: #213 Introduce an Intersection opened by @ilevkivskyi https://github.com/python/typing/issues/213 #688 advanced SimpleNamespace typing opened by @itajaja https://github.com/python/typing/issues/688 #1371 Proposal: Programmatically create types opened by @NeilGirdhar https://github.com/python/typing/issues/1371 --------------------------------------------------- All issues and pull requests with the label 'topic: feature' can be viewed under the following URL: https://github.com/python/typing/issues?q=label%3A%22topic%3A+feature%22

1 0

Reminder
by Shalok Shalom 24 Mar '23

24 Mar '23

Hi there. I send an email to this mailing list several days ago, and it seems like the admins of this list are either busy, or it got lost. Can you confirm, its one of those two? Thanks

2 1

How can I annotate class variables and instance variables with different types?
by Chihiro Sakai 23 Mar '23

23 Mar '23

I want means to annotate class variables and instance variables with distinct types. I'm using the well-known ORM library SQLAlchemy. With entity declaration, class variables are representations of table columns, while instance variables are the actual values of each record. Thus, because of this behaviour, I cannot write fully-typed code. For example, with the class declaration shown below: ``` class Entity(Base): __tablename__ = 'entity' column_1 = Column('column_1', BIGINT(unsigned=True), primary_key=True, autoincrement=True) ``` The actual type of the class variable `Entity.column_1` is `Column`, but on the other hand, the instance variable `Entity().column_1` is `int`. As far as I know, today, we cannot annotate variables like `Entity.column_1` with correct types, but I want to achieve those codes fully typed.

4 7

Support Algebraic Data Types with @sealed
by John Hagen 20 Mar '23

20 Mar '23

The original (superseded) PEP 622 proposed adding a @sealed decorator so that users could define algebraic data types (ADTs) that static type checkers could verify are `match`ed exhaustively: - https://peps.python.org/pep-0622/#sealed-classes-as-algebraic-data-types There is a somewhat fairly-upvoted question about ADTs in Python, indicating interest in this area: - https://stackoverflow.com/questions/16258553 @sealed would be very valuable towards making ADT usage within Python more type-safe. Is there any historical discussion about why @sealed was dropped from the final pattern matching PEP? What would be the next steps needed to push this forward? Would writing a new PEP be the next step?

15 32

Opened and changed typing issues week 2023-W11
by Typing Bot 20 Mar '23

20 Mar '23

The following is an overview of all issues and pull requests in the typing repository on GitHub with the label 'topic: feature' that were opened or updated last week, excluding closed issues. --------------------------------------------------- The following issues and pull requests were opened last week: #1371 Proposal: Programmatically create types opened by @NeilGirdhar https://github.com/python/typing/issues/1371 #1372 Add PartialApplication opened by @NeilGirdhar https://github.com/python/typing/issues/1372 --------------------------------------------------- The following issues and pull requests were updated last week: #213 Introduce an Intersection opened by @ilevkivskyi https://github.com/python/typing/issues/213 #256 Possible to distinguish between Sequence[str]/Iterable[str] and str? opened by @jtatum https://github.com/python/typing/issues/256 #802 Wrapper/Proxy Generic Type opened by @hatal175 https://github.com/python/typing/issues/802 #1273 Type mappings for generics opened by @jonathanslenders https://github.com/python/typing/issues/1273 #1362 Union of empty tuple should return `Never` opened by @randolf-scholz https://github.com/python/typing/issues/1362 --------------------------------------------------- All issues and pull requests with the label 'topic: feature' can be viewed under the following URL: https://github.com/python/typing/issues?q=label%3A%22topic%3A+feature%22

1 0

PyCon 2023 Typing Summit: Interest form
by S Pradeep Kumar 17 Mar '23

17 Mar '23

Hello all, PyCon 2023 is just a few months out. Jelle and I are looking forward to organizing the Typing Summit this year. We plan to be in-person in Salt Lake City, as we did last year. We have confirmed the Typing Summit with PyCon and will announce the exact date soon (tentatively a day before the main conference talks). We will try to record the event for those who cannot make it. For those who haven't attended before, the Typing Summit is a 3-4 hour event with a series of presentations and group discussions related to type checking. You can find the 2022 summit here: https://youtu.be/BNTkWQfqP_c If you are interested in attending, please fill out this form: https://forms.gle/yv1WnDoAVscg21vW8. This way, we can get a sense of the expected number of attendees and the topics you want to hear about. In a few weeks, we will send out another mail to typing-sig calling for presenters. Let us know if you have any questions. Best, Jelle Zijlstra Pradeep Kumar Srinivasan

1 2

`__class_getitem__` returning a mix of GenericAlias and concrete classes?
by Sebastian Berg 17 Mar '23

17 Mar '23

Hi all, In NumPy, we have `np.dtype` as a superclass for all DTypes. This implements `__class_getitem__` for typing to allow `np.dtype[Any]` or `np.dtype[np.float64]` returning a `GenericAlias`. Now, `np.dtype[Any]` probably must return a GenericAlias. Maybe `np.dtype[something_that_makes_little_sense]` also. However, `np.dtype[np.float64]` is well defined, there is a concrete runtime subclass of `np.dtype` representing exactly that construct. This would be the same as `array.array['l']` if `array.array` had concrete subclasses implementation for each supported type. My question is: would it be possible or even better to returning the concrete class (probably only when it exists)? Or would that just confuse the typing step, e.g. because we cannot actually type that `np.dtype[np.float64] -> np.types.Float64DType`. (Also, this isn't a fixed set in theory, users can create new subclasses in principle.) Hopefully this is very obvious in one direction, but unfortunately, I have no idea. Thanks and Cheers, Sebastian

5 16

type hint of function exceptions
by david.froger＠mailoo.org 13 Mar '23

13 Mar '23

Hi everyone, I've read https://stackoverflow.com/questions/44282268/python-type-hinting-with-excep… and especially the https://jonathan-scholbach.github.io/2023/02/06/what-does-it-mean-to-be-exc… blog post, but I want to keep coding with exceptions rather than returning `Ok | Err`. Here is a simple pattern I'm thinking to use, to let a function indicate the `Exception`s it may raised, which can be used: - by the developper as documentation, - by the static type checker for exhaustiveness ```python from typing import NoReturn, cast, TypeAlias class SomeError(Exception): ... class AnotherError(Exception): ... class StillAnotherError(Exception): ... FooError: TypeAlias = ValueError | SomeError | AnotherError def assert_never(value: NoReturn) -> NoReturn: # This also works at runtime as well assert False, f"This code should never be reached, got: {value}" def foo(i: int): if i == 0: raise ValueError("Must not be zero") elif i == 1: raise SomeError("Must not be one") elif i == 2: raise AnotherError("Must not be two") print("that's ok") try: foo(0) except Exception as e: e = cast(FooError, e) if isinstance(e, ValueError): print("value error") elif isinstance(e, SomeError): print("some error") else: assert_never(e) raise ``` It's far from being perfect, but it's simple and not too much intrusive nor obscure. In the above example, `mypy` can tell me which exception I forgot to manage: ``` foo.py:43: error: Argument 1 to "assert_never" has incompatible type "AnotherError"; expected "NoReturn" [arg-type] Found 1 error in 1 file (checked 1 source file) ``` What do you thing of this? Here is a proposal where the exception are annotated in the return type, and the try/except would directly check for exhaustiveness: ``` from typing import NoReturn, Annotated class SomeError(Exception): ... class AnotherError(Exception): ... class StillAnotherError(Exception): ... def assert_never(value: NoReturn) -> NoReturn: # This also works at runtime as well assert False, f"This code should never be reached, got: {value}" def Raises(exceptions): ... # TODO class NeverException(Exception): ... # TODO def foo(i: int) -> Annotated[None, Raises(ValueError | SomeError | AnotherError)]: if i == 0: ¦ raise ValueError("Must not be zero") elif i == 1: ¦ raise SomeError("Must not be one") elif i == 2: ¦ raise AnotherError("Must not be two") print("that's ok") try: foo(0) except ValueError: print("value error") except SomeError: print("some error") except NeverException: # static type checker warns here that AnotherError is not managed raise ``` and here is another fictive example with one intermediary function managing one of the 3 exceptions: ```python from typing import NoReturn, Annotated class SomeError(Exception): ... class AnotherError(Exception): ... class StillAnotherError(Exception): ... def assert_never(value: NoReturn) -> NoReturn: # This also works at runtime as well assert False, f"This code should never be reached, got: {value}" def Raises(exceptions): ... # TODO class NeverException(Exception): ... # TODO def foo(i: int) -> Annotated[None, Raises(ValueError | SomeError | AnotherError)]: if i == 0: ¦ raise ValueError("Must not be zero") elif i == 1: ¦ raise SomeError("Must not be one") elif i == 2: ¦ raise AnotherError("Must not be two") print("that's ok") def bar(i: int) -> Annotated[None, Raises(ValueError | SomeError)]: try: ¦ foo(i) except AnotherError: ¦ print("another error") try: foo(0) except ValueError: print("value error") except SomeError: print("some error") except NeverException: # no errer reported by the static type checker raise ``` Thanks!

3 4

Typing-sig March 2023