Mailman 3 April 2022 - Typing-sig

PEP 688: Making the buffer protocol accessible in Python
by Jelle Zijlstra 25 Apr '22

25 Apr '22

It's been a longstanding problem in typeshed that we cannot fully represent the C buffer protocol in stubs, and we have to make do with annotations of `bytes` or unions of common buffer types. I just submitted PEP 688 to add a new `types.Buffer` type that will fix this problem. A preview of the rendered PEP is available at https://pep-previews--2549.org.readthedocs.build/pep-0688/. The PR is at https://github.com/python/peps/pull/2549. Summary: * types.Buffer is added to CPython, using an isinstance hook to check for the presence of the buffer protocol. * In stubs, types.Buffer can be used as a base class. * Static type checkers should not need to special case the new class in any way. Please let me know if you have any feedback.

2 3

Opened and changed typing issues week 2022-W16
by Typing Bot 25 Apr '22

25 Apr '22

The following is an overview of all issues and pull requests in the typing repository on GitHub with the label 'topic: feature' that were opened or updated last week, excluding closed issues. --------------------------------------------------- The following issues and pull requests were opened last week: #1160 Allow `float("inf")` and `float("-inf")` in literals opened by @antonagestam https://github.com/python/typing/issues/1160 --------------------------------------------------- The following issues and pull requests were updated last week: #593 Buffer protocol types opened by @srittau https://github.com/python/typing/issues/593 #788 Adding loosly defined types opened by @cesco-fran https://github.com/python/typing/issues/788 #1129 Typing Feature Flags opened by @hmc-cs-mdrissi https://github.com/python/typing/issues/1129 --------------------------------------------------- All issues and pull requests with the label 'topic: feature' can be viewed under the following URL: https://github.com/python/typing/issues?q=label%3A%22topic%3A+feature%22

1 0

Dataclass Transform draft PEP
by Erik De Bonte 21 Apr '22

21 Apr '22

I've created a draft PEP for the "Dataclass Transforms" concept that Eric Traut presented at the December 1st Typing SIG meetup. Feedback is welcome! https://github.com/debonte/peps/blob/dataclass_transforms/pep-9999.rst Thanks, Erik

12 52

Simple syntax for Literal
by dev＠reggx.eu 19 Apr '22

19 Apr '22

Hi typing-sig, not sure if this is the correct place to post this. If it isn't, I'd be happy to be guided to the correct place. Anyways, here goes: I'm using Literal a lot for constants/magic numbers, both internally and when referring to constants/#defines used in foreign C functions. And I have one major gripe with them: they are unnecessarily verbose because they require writting everything twice. # +++++ CODE BLOCK START +++++ # Example: some constants from winuser.h used in windll.user32.MapVirtualKeyW MAPVK_VK_TO_VSC: Literal[0] = 0 MAPVK_VSC_TO_VK: Literal[1] = 1 MAPVK_VK_TO_CHAR: Literal[2] = 2 MAPVK_VSC_TO_VK_EX: Literal[3] = 3 MAPVK_VK_TO_VSC_EX: Literal[4] = 4 # +++++ CODE BLOCK END +++++ As you can see, every literal needs to be written twice to conform with the type annotation syntax as it currently is. Similar to the type inference for basic Final annotations without [<type>], I'd like to propse allowing the usage of basic Literal annotations without [<value>] if they are annotated and assigned on the same line. So these two lines would both result in MAPVK_VK_TO_VSC having the Literal[0] type hint. # +++++ CODE BLOCK START +++++ MAPVK_VK_TO_VSC: Literal = 0 MAPVK_VK_TO_VSC: Literal[0] = 0 # +++++ CODE BLOCK END +++++ For the implementation detail, I would err on the side of caution and only allow this notation in basic "variable: Literal = value" form. More complex forms like "variable: tuple[Literal, Literal] = [0, 1]" don't strike me as worthwhile enough to consider the additional implementation overhead/complexity. There are already enough things to consider, even with just the basic form. See the following two approaches. Approach A: If a naked Literal annotation is found, type checkers should ideally use the right hand side of the expression and treat it as if it was copied into the Literal[ ] annotation by the user. # +++++ CODE BLOCK START +++++ def get_value() -> Literal[0]: ... var different_literal: Literal[Color.RED] = Color.RED var non_literal: Final[bytes] = b'data' var: Literal = "value" # legal var: Literal["value"] = "value" # equivalent expression var: Literal = get_value() # should be treated as error and flagged by type checkers, even if get_value() returns Literal types. var: Literal[get_value()] = get_value() # equivalent expression, showing why it's an error var: Literal = different_literal # should be treated as error and flagged by type checkers, var: Literal[different_literal] = different_literal # equivalent expression var: Literal = non_literal # should be treated as error and flagged by type checkers, var: Literal[non_literal] = non_literal # equivalent expression # +++++ CODE BLOCK END +++++ This is probably the simpliest to implement, since it requires no inference of the right hand side. A non-literal, even another variable with Literal annotation would simply be treated as illegal. Approach B: Infer the value of the Literal to insert based on the right hand side's annotation. # +++++ CODE BLOCK START +++++ var: Literal = "value" # legal var: Literal["value"] = "value" # equivalent expression var: Literal = get_value() # legal, since the naked Literal would look for the annotation of get_value() and get Literal[0] var: Literal[get_value()] = get_value() # equivalent expression, showing why it's an error var: Literal = different_literal # legal, since the naked Literal will look up different_literal's annotation and get Literal[Color.RED] var: Literal[different_literal] = different_literal # equivalent expression var: Literal = non_literal # not sure about this one, looking at current behavior when assigning a Final to a Literal, mypy would flag it, pyright would treat non_literal as a literal when possible. var: Literal[non_literal] = non_literal # equivalent expression # +++++ CODE BLOCK END +++++ I'd be fine with any approach, since both solve my basic use case of reducing duplicated code. I've scanned through PEP 586 (Literal Types), and I found no mention on why "naked" Literal should be disallowed, indicating to me that it was simply never considered. (Which is not surprising, since most examples in that PEP are using Literal in function argument or return types, where the simple syntax would not apply) Under the point "Adding more concise syntax", it simply mentions skipping on Literal completely is not an option. This proposal still requires the Literal annotation, it simply extends its usage in a minor way that doesn't break grammar. I'd appreciate any feedback on angles that I might have missed while investigating the feasability of this propsal. Best wishes, Jakob

4 3

Syntax for inline-definition of TypeVar
by Sebastian Rittau 19 Apr '22

19 Apr '22

After the discussion on type var defaults, Guido and I had a short discussion about a potential inline-definition syntax for type vars. Such a syntax has a few advantages over the definition using TypeVar: * The type vars would be defined close to where they are actually used. * Type vars would not leak their name into the module-global scope. * Removes the duplication from TypeVar: _T = TypeVar("_T") * This is in line with what other languages are doing and so more familiar for developers coming from other languages. Even if we implement an inline syntax, TypeVar should not be deprecated as it's still quite useful for defining re-usable type vars like typing.AnyStr. Here are a few proposals we came up with. Inline syntax using square brackets ===================== class C[T, S]: def foo[R=int](self) -> R: pass Some ideas how we could express upper bounds, constraints, and variance: * Upper bounds: [R(Base1 | Base2)] or [R from Base1 | Base2]. * Constraints: [R as Base1 | Base2] * Variance: [R(covariant=True)] Advantages: * Similar syntax to existing generics syntax. * Used by languages such as Go and Scala. Disadvantages: * Square brackets are already used for indexes/slices and generics. * We have a bit of a "bracket hell", which does not help readability: def foo[F=dict[str, Callable[[int], object]]](): pass Inline syntax using angle brackets ===================== class C<T, S>: def foo<R>(self) -> R: pass Advantages: * Angle brackets are currently unused in Python and lower than/greater than is not used within type context. * Using angle brackets could reduce "bracket hell". * Used by languages like C++, Java, C#, TypeScript, Delphi, so should be fairly similar to developers coming from popular languages. Disadvantages: * Slightly inconsistent with existing generic syntax. Decorators ======= @typevar("T") @typevar("S") class C: @typevar("R", default=int) def foo(self) -> R: pass Advantages: * Doesn't require syntax changes to Python. * Very readable (in my opinion), because it has one type var per line, especially with more complex type vars. * Is defined exactly like a TypeVar(), but without assigning it to a name. * Flexible and extensible as it's using regular decorator syntax. * Backwards compatible as it could be added to typing_extensions. Disadvantages: * More verbose than the other suggestions, especially when using simple type vars. * Non-standard syntax, compared to other languages. * Currently requires either quoted types or "from __future__ import annotations". ------------------------- My personal favorite is the decorator syntax, for all the advantages listed. The disadvantages seem minor to me or temporary. Being more verbose makes is actually more readable. When writing TypeScript, I often have monsters like this: function createReactStore< S, A extends Action = AnyAction, E = undefined, >( someArguments: ... ) { // ... } I believe that the Python equivalent using decorator would be more readable: @typevar("S") @typevar("A", bound=Action, default=AnyAction) @typevar("E", default=object) def createReactStore( someArguments: ... ): ... - Sebastian

11 21

Opened and changed typing issues week 2022-W15
by Typing Bot 18 Apr '22

18 Apr '22

No issues or pull requests with the label 'topic: feature' were opened or updated last week in the typing repository on GitHub. All issues and pull requests with the label 'topic: feature' can be viewed under the following URL: https://github.com/python/typing/issues?q=label%3A%22topic%3A+feature%22

1 0

PyCon 2022 Typing Summit Agenda
by Shannon Zhu 15 Apr '22

15 Apr '22

Hi everyone, PyCon 2022 is coming up! There will be many typing-related talks, tutorials booths, etc., but I wanted to provide a final update on the Typing Summit for everyone who is able to attend in person. The summit (which has become an annual event since the pandemic started) will be held this year at PyCon US on Thursday, April 28th from 1-5pm, in Salt Lake City. The agenda is as follows - all times are PM and are in Mountain Time: 1:00-1:20 – Settling in and introductions 1:20-1:35 – “New typing features in Python 3.10 and 3.11”, David Foster 1:35-1:55 – “Typing of Tensor Shapes and Type Arithmetic”, Alfonso Castaño 1:55-2:10 – “Too small for a PEP: minor new typing features in Python 3.11”, Jelle Zijlstra (2:10-2:25 – Buffer/Break) 2:25-2:40 – “Extending PEP 647: User-Defined Type Guards”, Rebecca Chen 2:40-3:05 – “The future of TypedDict” and “Runtime uses for type annotations: A survey of tools”, David Foster 3:05-3:35 – “Runtime Annotations: PEP 563 & 649 Overview”, Carl Meyer (3:35-3:50 – Buffer/Break) 3:50-4:25 – “Beyond Subtyping”, Kevin Millikin 4:25-4:55 – “Panel: Typing-sig and Python Core Dev”, Guido van Rossum, Greg P. Smith, Jelle Zijlstra, Pradeep Kumar Srinivasan, Matthew Rahtz You can now also find this information live on the PyCon website: https://us.pycon.org/2022/events/typing-summit/ Thanks to everyone who has put in the time and effort to prepare a topic for presentation and discussion! Note: Unfortunately we cannot support remote attendance this year, but we are working with PyCon on options to record at least the slide presentation sections to be made available afterwards. Excited to see everyone there! Shannon & Alfonso

1 0

Status of PEP 645? (Allow writing optional types as x?)
by David Foster 15 Apr '22

15 Apr '22

The typing-related PEP 645 - which introduces the syntax `T?` to mean `Optional[T]` - only has a single substantial related commit in the PEPs repo - the commit introducing the PEP (in Nov 2020) - and no discussions thread that I can find (on typing-sig or elsewhere). I'm personally quite interested in the potential capability to quickly signify `Optional[T]` since it is something I do a lot. What is the status of this PEP? Deferred for now? Withdrawn in favor of `T|None` syntax?† Something else? -- David Foster | Seattle, WA, USA Contributor to TypedDict, mypy, and Python's typing system † Through the grapevine I've been inferring that folks have been gravitating toward the slightly-shorter `T|None` syntax to spell `Optional[T]` in the meantime, and I even mention that spelling in my own PEP 655 (Required[]) [1] [1]: https://peps.python.org/pep-0655/#how-to-teach-this

4 5

SC requests for typing-sig: specifications & delegation
by encukou＠gmail.com 15 Apr '22

15 Apr '22

Dear typing-sig, Typing-related PEPs are becoming more common, and we'd like to propose some changes to how they are handled. First, we feel that it is time to convert PEPs (proposals) to specifications. We're getting to the point where we might refer to a feature as “PEP 589 as amended by PEP 655”. To understand a feature, users (and implementers of typing tools) must read through proposal documents that contain motivation, discussion, and sometimes even information that's obsolete today. The use of PEP numbers in everyday communication makes the field even less accessible to outsiders. The Python Packaging Authority (PyPA) used to be in a similar situation, and moved to a model of [specifications], which contain current information, and PEPs as change proposals, which contain a summary of each change and the reasoning (motivation, rationale, discussion summary) for each change. The work of converting existing PEPs to spec documents is not done, but the eventual goal is clear. We ask you to consider adopting a similar model. [specifications]: https://packaging.python.org/en/latest/specifications/ Second, we'd like to see some kind of typing-sig delegation. For PEPs like 655 (Marking individual TypedDict items as required or potentially-missing), which only affect typing.py and external tools, consensus on typing-sig is by far the most important consideration for accepting or rejecting a PEP. If you could appoint a person (or committee) you trust to speak on behalf of typing-sig -- that is, seek consensus on the list and fairly summarize it -- it would simplify the decision process. It could even be the first step toward delegating typing-related PEPs entirely. Happy typing! – Petr, on behalf of the Steering Council

6 11

Making class attributes first class citizens
by Tin Tvrtković 12 Apr '22

12 Apr '22

Hello. I've been working on an ORM and I've realized the typing tools we have aren't really well suited to this use case. Let's see if we can start improving this. For example, it's a challenge to find an ORM that can do statically checked query expressions, or statically checked projections (when you want to load only a subset of your table fields, as a tuple, say). I think SQLModel comes closest at this time with some undocumented magic. I have a small proposal to start, and it's about class attributes. Essentially every library in this space has you model your table/collection as a class, kind of like this: ``` class MyModel: my_int_field: int ``` Sometimes they make you inherit from something, sometimes there's a metaclass involved, sometimes decorators, sometimes there are magic descriptors. At the *core* of each, though, there are 3 entities: * the class itself (MyModel) * the instance attribute `my_int_field` * the class attribute `my_int_field` The class attribute is the one we need to improve/standardize, since it's the one with the weakest typing support. In practice there are 3 ways of getting to the class attribute: 1) getting it directly from the class object - `MyModel.my_int_field`. This is what the SQLAlchemy ORM and SQLModel do 2) getting it from a magic namedtuple attribute on the class - `MyModel.c.my_int_field`. This is what SQLAlchemy Core does, and technically attrs and dataclass (the field is ugly and hidden) 3) getting it from a function that returns a magic namedtuple - fields(MyModel).my_int_field. This is what attrs and dataclasses expose (although for dataclass it's not a namedtuple yet) I personally think 1) is bad for composability (two libraries can't use the model at the same time since they can't both own `MyModel.my_int_field`, the class can't be slotted without heavy magic) but it's very popular in the wild. My proposal is this: we figure out a way for libraries to export their own generic attribute type, and we have type checkers automatically parametrize it with the actual type information from the class. This could be kind of an extension to the dataclass_transform PEP. That PEP establishes generic support for declarative classes, we now just build on it with a system for custom class attributes. Here is what a potential direction for this might look like. I'm going to use the attrs way since its existing functionality is very close to this. `attrs.fields(MyModel)` already returns a namedtuple of (attr.Attribute[int],). We add an optional attribute to `fields`, `AttributeCls`. SQLAlchemy implements its own attribute type, `SQLAlchemyAttribute[T]`. `attrs.fields(MyModel, AttributeCls=SQLAlchemyAttribute)` now returns a namedtuple of (sqlalchemy.SQLAlchemyAttribute[int],). SQLAlchemy defines its own `fields` function for ergonomics: `c = partial(fields, AttributeCls=SQLAlchemyAttribute)`. Since SQLAlchemy has its own attribute type now, that attribute type is free to override `__eq__` so users can do queries with it: `c(MyModel).my_int == 5`, and since the attribute is generic, this can be statically checked. My library can define its own attribute type, `TinAttribute[T]`. Since my library targets a different database, I support a different API: `f(MyModel).my_int.gt(5).lt(10)`. An extension to the protocol could be made so we support special decorators, like the dataclass_transform PEP. So a decorator could exist that would signal to the type checker that these attributes are injected either into the class namespace itself, or a magic class attribute namedtuple like `c`. So a SQLAlchemy Core version could be: ``` sqlalchemy_decorator = partial(fields_decorator, attribute_cls=SQLAlchemyAttribute, magic_attribute=c) @sqlalchemy_decorator class MyModel: my_int: int ``` which would tell the type checker that `MyModel.c.my_int` is an instance of `SQLAlchemyAttribute[int]`. If you don't set the `magic_attribute` value, it would tell the typechecker the attributes get injected into the class object instead, so you'd get the SQLAlchemy ORM and SQLModel behavior. I don't think this solves everything (libraries might still need descriptors for other purposes, like maybe dirtiness tracking), but I think it may be a good start. Since I mentioned statically typed projections, this allows those too (although with a lot of overloads). My query function can be `async def query(MyModel, projection: tuple[TinAttribute[T]]) -> tuple[T]`. I think a system like this would be a good start to improving type safety in these kinds of libraries.

3 5