Mailman 3 July 2020 - Typing-sig

Any way to express that a dict is expected to have a key(s) but don't care about other keys?
by Brett Cannon 21 Apr '22

21 Apr '22

Looking at TypedDict it seems that it does not like the idea of superfluous keys existing in a dict. In my case I have a REST call returning JSON for an error and all I'm guaranteed is to get an "error" key. But there may be extra keys that provide more details about the specific error. Unfortunately I don't know the exhaustive list of keys so I can't use a TypedDict with `totality == False` to try to be complete. I can't think of any way to somehow say "require/expect this one key, but other keys are OK". Am I missing anything that would let me do this?

13 31

PEP 612: Parameter Specification Variables Discussion Pt. II
by Mark Mendoza 11 Aug '20

11 Aug '20

Hi All, I want to start off by thanking everyone that gave feedback on the first round of review of PEP 612. Your feedback illuminated a lot of things that I think have made this PEP a lot better. Over the past few weeks I have had the time to rewrite much of that document to integrate all of that feedback. Notably, this included integrating `typing.type_variable_operators.Concatenate` to support decorators that modify a finite number of positional parameters. With that being said, I'd appreciate another round of feedback from all of you to determine whether this form of the PEP is ready to move on to the Steering Council. Thanks again! Mark Mendoza

4 12

Tensor typing - next group call
by Matthew Rahtz 11 Aug '20

11 Aug '20

Hi everyone, Next week we'll be holding our next tensor typing meeting. WHEN? Monday 3rd August, 9:30am SF time. GCal link: https://calendar.google.com/event?action=TEMPLATE&tmeid=NzY5cXM1am5uNDNkMW1… WHERE? Google Meet. We'll post the link on the day itself. WHAT? 1. Status updates. What have folks been working on over the past month? 2. What should symbols mean? Jörg and I would like to make a second attempt at making the case that symbols should by default refer to semantic axis types (e.g. `Batch` => a batch-like axis) rather than actual shapes (`Batch` => a concrete value of '32'). We'll have a short presentation and then we'll open it up for discussion. 3. Other. Is there anything else folks would like to talk about? Cheers! Matthew

2 5

What is the intended use case for the typing.SuportsXyz protocols?
by Luciano Ramalho 02 Aug '20

02 Aug '20

I was excited to learn about SupportsInt, SupportsFloat, and SupportsComplex protocols decorated with @runtime_checkable because I thought they bridged the gap between ABCs and typing protocol classes, allowing consistent type checks by static tools and at runtime. However, I am unable to see how to use those classes in practice. For example, SupportsComplex: I understand at runtime the @runtime_checkable decorator merely checks whether __complex__ exists in the type, but that is not a good predictor of how to use a numeric value. Types that actually can be converted to complex fail the insinstance check with SupportsComplex. That's the case of built-in int and float, as well as NumPy integer and float types like float16 and uint8. On the other hand, SupportsInt suggests that I can convert a complex to int, but in fact the int and float types implement __complex__ only to provide a better error message, according to Guido van Rossum in an earlier thread here. >>> isinstance(1+0j, SupportsInt) True >>> int(1+0j) Traceback (most recent call last): ... TypeError: can't convert complex to int The issue with SupportsInt also happens with SupportsFloat. In addition, the way Mypy interprets these protocol classes is not the same as what we can see at runtime. Given the above, what are recommended use cases for the runtime checkable SupportsInt, SupportsFloat, and SupportsComplex? Cheers, Luciano -- Luciano Ramalho | Author of Fluent Python (O'Reilly, 2015) | http://shop.oreilly.com/product/0636920032519.do | Technical Principal at ThoughtWorks | Twitter: @ramalhoorg

2 1

assigning a NoReturn function call
by Martin DeMello 31 Jul '20

31 Jul '20

Should the following be a type error? ``` def f() -> NoReturn while True: ... x = f() ``` Neither pytype nor mypy currently catch it. martin

2 3

PEP 604 status update
by Guido van Rossum 29 Jul '20

29 Jul '20

Here's a status update for PEP 604: https://www.python.org/dev/peps/pep-0604/ - Ivan has asked the SC to find another PEP delegate for personal reasons, and the SC has asked me to step up in his place. Philippe, the PEP's author, has agreed. - Maggie Moss has volunteered to write an implementation which is currently in progress at https://github.com/python/cpython/pull/21515. - I am quite positive regarding the PEP, I just think we need to tighten the specification some more, especially in corner cases and for interoperability with types defined in typing.py. I will work with Maggie and Philippe on this. I see no problem getting this into 3.10. It will make a nice complement to the list[int] notation introduced in 3.9 by PEP 585. Here's a summary of the key points from the PEP: - `int | str` represents the union of `int` and `str` - There will be a new extension type, `types.Union` (1), to represent unions created using `|` - Such types will be acceptable in `isinstance(x, t)` and `issubclass(C, t)` The rest is elaboration; most semantics are derived from existing semantics of `typing.Union[]`. We also strive for decent interoperability with the `typing` module. (1) Beware! The existing union notation uses `typing.Union`; the new notation uses `types.Union`. -- --Guido van Rossum (python.org/~guido) *Pronouns: he/him **(why is my pronoun here?)* <http://feministing.com/2015/02/03/how-using-they-as-a-singular-pronoun-can-…>

3 4

[PEP 604] Clarify interaction with ForwardRef, and None
by Richard Eames 29 Jul '20

29 Jul '20

Hi list! I was asked to bring this up here. When PEP604 was first introduced last year I noticed a few under-specified parts, namely how it interacts with ForwardRef and the edge case of `None | None` eg: >>> from typing import Union >>> Union[int, 'str'] typing.Union[int, ForwardRef('str')] >>> int | 'str' # should this raise an error, or return the same as above? >>> Union[None, "ref1"] typing.Optional[ForwardRef('ref1')] >>> None | "ref1" # should this construct (None first) be forbidden, or does str.__or__ or NoneType.__or__ need to be implemented? >>> Union[None, None] <class 'NoneType'> >>> None | None # should be forbidden? It seems very unlikely to come up in practise even if typing.Union supports it >>> Union["ref1","ref2"] typing.Union[ForwardRef('ref1'), ForwardRef('ref2')] >>> "ref1" | "ref2" # would this be str.__or__ returning types.Union? In the discussion last year, it was suggested that there could be a `typing.Forward` that could be used instead of the bare string "ref" cases, or even requiring the union to be fully quoted eg `T = "Ref | OtherRef"`.

2 1

Creating custom types with metadata (similar to Annotated)
by Patrick Arminio 24 Jul '20

24 Jul '20

Hello again, I'm working on a library that uses the return type of methods at runtime to generate GraphQL types and I'm trying to deal with the case of circular imports and classes referencing each others. I was thinking of create a LazyType where you can pass the type name and the module, so that we are able to resolve the actual type at runtime without having to guess where it came from, like this: ``` import typing import strawberry if typing.TYPE_CHECKING: from .type_a import TypeA @strawberry.type class TypeB: @strawberry.field() def type_a(self, info) -> strawberry.LazyType["TypeA", ".type_a"]: from .type_a import TypeA return TypeA() ``` As previous post unfortunately I can't relay on sys.module, since the type is only imported when TYPE_CHECKING is True or inside the method. Now, I managed to make it work with a custom LazyType class, so the annotation above works; ``` strawberry.LazyType["TypeA", ".type_a"] ``` I don't dislike this to be honest, but looks like mypy isn't really happy, as you can see from the errors here: ``` tests/test_cyclic/type_b.py:13: error: "LazyType" expects no type arguments, but 2 given tests/test_cyclic/type_b.py:13: error: Invalid type comment or annotation tests/test_cyclic/type_b.py:16: error: Incompatible return value type (got "TypeA", expected "LazyType") ``` The first one could be resolve by using generics, but I'm not sure that's a good idea. The last one could be solved by creating a plugin, hopefully it won't be too difficult. But I have no clue about `Invalid type comment or annotation`. Why would that be the case? I noticed that the error disappears when removing the dot from the module name, but I'd like to be able to pass a dot there, so users of the library don't have to type the full module, since we can infer that. Any ideas? Or do I need to change the approach for this? Maybe doing a less typehint-y way: ``` def x() -> strawberry.LazyType("TypeA", ".type_a"): ``` Let me know! :)

2 3

How to evaluate Forward/Strings Refs
by Patrick Arminio 16 Jul '20

16 Jul '20

Hi folks, I'm working on a library that makes extensive of use of type hints at runtime and I found this use case where I need to evaluate a forward reference ```python def test_forward_references(): @dataclasses.dataclass class User: username: str @dataclasses.dataclass class Product: reviews: List["Review"] @dataclasses.dataclass class Review: text: str ``` as youy can see List["Review"] has a reference to the not yet declared class Review. My use case is to generate class based on the type hints of these 3 classes, unfortunately when I try to inspect the field type of Product.reviews, I get List[ForwardRef("Review)] and I'm not sure how I could get the actual type at run time. I've made a repl that shows what I've tried so far: https://repl.it/@patrick91/dcth-2 I've tried getting the global namespace for the module, but that won't work in this case, since the classes are defined inside a function. I have another option, which is to store all the decorated classes in a global dict (I'm using a different decorator that the dataclasses one), but I don't want to do that as my users could register multiple types with the same name. Any ideas on how I could solve this?

3 8

Working Higher Kinded Types emulation
by Никита Соболев 12 Jul '20

12 Jul '20

Hi everyone, I would love to share something I have been working on for the last year! I have implemented an emulation of Higher Kinded Types for mypy. Here I would love to describe how it works and (hopefully!) receive your feedback. Some context before we start: 1. What are Higher Kinded Types (or HKT)? It is basically a concept of "generics of generics". Here's the simplest example: ```python def build_new_object(typ: Type[T], value: V) -> T[V]: ... # won't work, because HKT is not natively supported ``` 2. What is "emulated" HKT support? There's a very popular approach to model HKT not as `T[V]` but as a `Kind[T, V]`. Original whitepaper: https://www.cl.cam.ac.uk/~jdy22/papers/lightweight-higher-kinded-polymorphi… 3. Why would anyone need this? That's a good question. HKT gives a lot of power. It is widely used with functional programming. You can find a full example here: https://sobolevn.me/2020/06/how-async-should-have-been With HKT one can rewrite `@overload`ed function for both async and sync use-cases as: ```python @kinded def fetch_resource_size( client_get: Callable[[str], Kind2[_IOBased, httpx.Response, Exception]], url: str, ) -> Kind2[_IOBased, int, Exception]: ... # implementation will be slightly different ``` 4. Is it ready for the production use? No! It is just an early prototype. There are a lot of things to polish. There are some known hacks and limitations right now (that can possibly stay forever). Now, let's dig into how it works! First of all, there's a new primitive to represent `Kind`, it is called `KindN` and has 3 (at the moment, it might change in the future) aliases `Kind1`, `Kind2`, `Kind3`. Why? Kind1 represents types with one type variable, like `List[X]` Kind2 represents types with two type variables, like `Result[Value, Error]` Kind3 represents types with three type variables, like `Generator[X, A, B]` KindN is a base type for all of them, it can possibly have any number of type variables. Source code: https://github.com/dry-python/returns/blob/master/returns/primitives/hkt.py Now, let's say I want to build a `map_` function to map a pure function over a `Functor` instance. How should I do that? I need to: 1. Implement `Mappable` (or `Functor`) interface 2. Implement some data type that implements your `Mappable` interface 3. Implement some hacks, that I've mentioned earlier, to make sure that you will be able to work with `KindN` values 4. Implement the `map_` function itself Let's start with the interface: ```python _MappableType = TypeVar('_MappableType', bound='MappableN') class MappableN(Generic[_FirstType, _SecondType, _ThirdType]): """ Allows to chain wrapped values in containers with regular functions. Behaves like a functor. See also: https://en.wikipedia.org/wiki/Functor """ @abstractmethod def map( # noqa: WPS125 self: _MappableType, function: Callable[[_FirstType], _UpdatedType], ) -> KindN[_MappableType, _UpdatedType, _SecondType, _ThirdType]: """Allows to run a pure function over a container.""" #: Type alias for kinds with one type argument. Mappable1 = MappableN[_FirstType, NoReturn, NoReturn] ``` Source: https://github.com/dry-python/returns/blob/master/returns/interfaces/mappab… Tests: https://github.com/dry-python/returns/blob/master/typesafety/test_interface… This is pretty straight-forward. Some things to notice: - `_MappableType` must be bound to `MappableN`, we use this type to annotate `self`, this is required - It returns modified `KindN` instance, we work with the first type argument here - method is abstract, because we would need an actual implementation from the child types Now, let's write some real data type to satisfy this contract. I will just use the code from here: https://github.com/dry-python/returns/blob/01cf56e023e81f165f4a2b5c2c81a415… Some things to notice: - We use `Kind1` as a supertype for our own data type, it means that we only have one type argument, this is required. Notice that we pass `'MyClass'` forward reference into `Kind1` - mypy makes sure that `map` method defintion exists and is correct, any violations of the type contract will make mypy to raise errors Now, let's try to implement the `map_` function for any `Mappable` and any pure function. Source: https://github.com/dry-python/returns/blob/master/returns/methods/map.py Tests: https://github.com/dry-python/returns/blob/master/typesafety/test_methods/t… That's how it will look like: ```python from returns.primitives.hkt import KindN, kinded _MappableKind = TypeVar('_MappableKind', bound=MappableN) @kinded def map_( container: KindN[_MappableKind, _FirstType, _SecondType, _ThirdType], function: Callable[[_FirstType], _UpdatedType], ) -> KindN[_MappableKind, _UpdatedType, _SecondType, _ThirdType]: ... ``` Things to note: - We need to mark this function as `@kinded`, because we need to tweak its return type with a custom plugin. We don't want `KindN[]` instances, we need real types! - We need to bound `_MappableKind` to our `MappableN` interface to make sure we will have an access to its methods (`.map` in our case) The only thing left is the implementation. The obvious way: `return container.map(function)` won't work. Because: 1. `KindN` does not have `.map` method, we need to somehow get to `MappableN` instead 2. It will have wrong return type: `KindN[MappableN, _UpdatedType, _SecondType, _ThirdType]` while we need `KindN[_MappableKind, _UpdatedType, _SecondType, _ThirdType]` (the difference is in the first type argument). So, we need to use the dirty hacks! ```python from returns.primitives.hkt import debound # ... new_instance, rebound = debound(container) return rebound(new_instance.map(function)) ``` What does it do? 1. It returns a proper `new_instance` with `MappableN` type 2. It also returns a callback to coerce the return type to the correct one The only thing left is to test it! Here are the tests: https://github.com/dry-python/returns/blob/master/typesafety/test_methods/t… They pass! To sum things up: 1. Current state of HKTs in Python allows defining complex type contracts 2. We are able to write `@kinded` methods and functions that do return the correct types in the end 3. We have some limitations: like max 3 type parameters at the moment 4. There are some minor API tweak when working with kinds: debound, dekind, Kinded[], @kinded 5. There are also some blockers from the mypy side to improve some API parts, like: https://github.com/python/mypy/issues/9001 That's it! I would love to answer any questions you have. And I hope to hear your feedback on my work. Best regards, Nikita Sobolev https://github.com/sobolevn

1 0

2022

2021

2020

2019

2018

Typing-sig July 2020