Mailman 3 October 2020 - Typing-sig

Any way to express that a dict is expected to have a key(s) but don't care about other keys?
by Brett Cannon 22 Apr '22

22 Apr '22

Looking at TypedDict it seems that it does not like the idea of superfluous keys existing in a dict. In my case I have a REST call returning JSON for an error and all I'm guaranteed is to get an "error" key. But there may be extra keys that provide more details about the specific error. Unfortunately I don't know the exhaustive list of keys so I can't use a TypedDict with `totality == False` to try to be complete. I can't think of any way to somehow say "require/expect this one key, but other keys are OK". Am I missing anything that would let me do this?

13 31

User-defined type guards
by Eric Traut 18 Jan '21

18 Jan '21

I occasionally receive questions from pyright users about extending type narrowing to handle cases that are not possible with today’s Python type system. Typescript provides a useful facility called [user-defined type guards]( https://www.typescriptlang.org/docs/handbook/advanced-types.html#user-defin…) that allows users to extend the notion of type narrowing. It would be straightforward to add this idea to Python. Do others think this would be useful, and is there any interest in standardizing this? Here’s a concrete proposal. We could add a new generic type called `TypeGuard` in the `typing` module. It would be a subclass of bool, defined simply as: ```python class TypeGuard(bool, Generic[_T]): pass ``` If any function or method accepts one parameter and returns a `TypeGuard` type, a type checker would assume that the argument passed to this function can be narrowed to the type specified in the `TypeGuard` type argument. Here are a few examples: ```python from typing import Any, Generic, List, Tuple, TypeGuard, TypeVar _T = TypeVar("_T") def is_two_element_tuple(val: Tuple[_T, ...]) -> TypeGuard[Tuple[_T, _T]]: return TypeGuard(len(val) == 2) def foo(names: Tuple[str, ...]): if is_two_element_tuple(names): reveal_type(names) # Tuple[str, str] def is_string_list(val: List[object]) -> TypeGuard[List[str]]: return TypeGuard(all(isinstance(x, str) for x in val)) def bar(stuff: List[Any]): if is_string_list(stuff): reveal_type(stuff) # List[str] ``` Thoughts? Suggestions? -Eric --- Eric Traut Contributor to Pyright and Pylance Microsoft Corp.

12 59

Variadic generics PEP draft
by Matthew Rahtz 16 Dec '20

16 Dec '20

Hi all, Something that's on many of our wishlists is support for variadic generics. Here's a first draft of a PEP detailing how they might work. https://docs.google.com/document/d/1oXWyAtnv0-pbyJud8H5wkpIk8aajbkX-leJ8JXs… (Attached is also an HTML render of the RST for easier reading.) This is an early draft so there are likely many things we're still missing. Please do take a look and give us your feedback (in the doc directly or by email, whichever is easiest for you). Thanks! Matthew

4 10

Should a single TypeVar be allowed in a function signature?
by Rebecca Chen 13 Dec '20

13 Dec '20

Hi all, I was hoping to get some opinions on whether something like the following is valid, and if so, how it should be interpreted: from typing import MutableSequence, TypeVar T = TypeVar('T') def f(seq: MutableSequence[T]) -> int: return len(seq) (This is, of course, *not* inside a generic class parameterized on `T`.) Currently, pytype emits an [invalid-annotation] error for this piece of code, with the explanation that `T` appears only once in `f`. Neither mypy nor pyre emits an error. As far as I know, how a single TypeVar should be treated is not covered in any of the typing PEPs. pytype's reasons for making this an error are: * The error messaging helps users avoid mistakes caused by misunderstanding how to use a TypeVar. * pytype's behavior when encountering a single TypeVar (especially when a bound or constraints come into play) is ill-defined, so we don't want anyone depending on it. The reasons I've seen for not making it an error are: * The TypeVar provides a way to annotate `seq` as a sequence of anything, while still allowing usage errors to be caught in the body of `f`. `Any` swallows all errors, and `object` does not work for invariant containers. * There are potential future uses for a single TypeVar. One that came up in the tensor typing call this morning is using a ListVariadic as a placeholder to represent an unused part of an annotation. * PEP 484 does not explicitly say that this should be an error, so it makes sense to err on the side of allowing it. * In spirit, `T` is an unused variable; flagging such things is not the job of a type checker. Thoughts? Best, Rebecca

4 10

Defaults for type vars
by Sebastian Rittau 26 Nov '20

26 Nov '20

This was previously discussed in https://github.com/python/typing/issues/307. It would often be useful to be able to specify a default type for type vars. A few examples: * Counter is most likely used with ints in the vast majority of cases. Being able to write "Counter" instead of "Counter[int]" makes sense to me. * Currently, when marking a previously non-generic type as generic, it will break existing type annotations. Having a default value works around this. To use Counter as an example again: Currently it is non-generic, but making it generic would break existing annotations. Adding a default of "int" would work around this problem. * Another example from the issue above: Using a default of None can prevent overloads, as in the following example: "def nullcontext(enter_result: _T = ...) -> ContextManager[_T]: ...". Without the default we need an explicit overload for the case where no argument is provided, because otherwise the type checker can't know the type of _T. While this is not a big problem in stubs, I find overloads quite obnoxious in implementations. While of course not completely comparable, I find myself using generic defaults quite often when working with TypeScript. A few examples as anectodal evidence: * We have functions that fetch JSON from an API. They are generic over the return type and default to a general JSON type, but can be overwritten if the response shape is known. (Because we control the API.) * We have a few cases, where an object is generic over a list of strings. Take a class wrapping HTML <select> elements. Sometimes all the possible options are known to the application, and have semantic meaning. (For example for choosing a view.) So it makes sense to declare a class like "class Select[Literal["foo", "bar"]]" (using Python syntax). It's then possible to have "Select.value" have the type "Literal["foo", "bar"]". In other cases, the possible options are unknown (taken from a database, for example). In that case, using "class Select[str]" would make sense. The latter is also a sensible default. * Generally I tend to use more type variables per type in TypeScript than in Python, partially due to the fact that some of them can be set to sensible defaults. While it would be obnoxious to always define four generics everytime you use a type, in typeshed I only have to define one of them with the others having sensible defaults. What do other people think about this and how would we proceed if this is deemed useful? - Sebastian

3 2

Tensor typing meeting 12/10/2020
by Matthew Rahtz 25 Nov '20

25 Nov '20

Hey all, Our next tensor typing meeting will be next week. WHEN? Monday 12th October, 9:30am San Francisco Time, 5:30pm London time. WHERE? http://meet.google.com/fft-dzjq-ksu WHAT? We have a relatively sparse agenda this time. I'll lead a discussion, but if anyone else would like to present/discuss something, let me know! 1. Status updates 2. Discussion: Use of ... (Ellipsis) for unspecified parts of shapes Cheers! Matthew

3 13

Multiple Dispatch for Binary Operators in Python
by Neil Girdhar 29 Oct '20

29 Oct '20

Hi everyone, My friend and I have been interested in multiple dispatch for a while, and we noticed that it might be worth considering using it to resolve some fundamental difficulties type-checkers have understanding binary operators. We wrote up a collab summarizing our ideas: https://colab.research.google.com/drive/1-HQa5GOmV-uhTbv6EgnMfJYVgwuYMIrQ#s… I initially posted a short version of this idea to python-ideas, but Guido suggested I flesh out the idea and post it here. Looking forward to everyone's insight and comments, Best, Neil

2 1

Higher Kinded Types released!
by Никита Соболев 26 Oct '20

26 Oct '20

Hi everyone, I am happy to announce that the first version of Higher Kinded Types emulation is released with full `mypy` support as a part of `dry-python/returns(a)0.15`. I have made a post that describes all its features and (what is even more exciting) all the hacks we have to apply to get this working: https://sobolevn.me/2020/10/higher-kinded-types-in-python Long story short: the biggest problem is that `mypy` treats `Kind` as a regular type and not as a SpecialForm with some extra features. This forces us to use direct inheritance from Kind and do some other hacks inside our mypy plugin. The second problem is that we don't have variadic generics. But, I know that it is in the work right now. If mypy developers are interested, I can slowly work on adding Kind type and all the required machinery into mypy itself (the good news is that even `mypyc` works correctly with Kind). If so, please, guide me about the first steps :) Source code and docs: https://github.com/dry-python/returns Simple example from the article to experiment: https://gist.github.com/sobolevn/7f8ffd885aec70e55dd47928a1fb3e61 Best regards, Nikita Sobolev https://github.com/sobolevn

1 0

Update: Work on Optional Syntax PEP
by maggiemoss＠fb.com 08 Oct '20

08 Oct '20

Hi there, Recently I have been working on an implementation for PEP 604 (https://www.python.org/dev/peps/pep-0604/). While working on this, I started reviewing and exploring what new Optional syntax would look like in the Python language. I've written a rough draft of a PEP and wanted to present it to this email list before submitting a PR. I know this is a topic that has been discussed in many settings (Github issues, in person typing meetups, etc. )previously, and wanted to solicit feedback here first. Let me know if submitting a PR would be a better way to gauge sentiment and feedback on this proposal. Draft Proposal (branch compare): https://github.com/python/peps/compare/master...MaggieMoss:optional-syntax-… Gist: https://gist.github.com/MaggieMoss/c848cb3a581979f445d075c15629c950 Thanks again, Maggie

6 9

Validating type completeness for py.typed packages
by Eric Traut 03 Oct '20

03 Oct '20

PEP 561 indicates that “package maintainers who wish to support type checking of their code MUST add a marker file named py.typed…”. It doesn’t define what “support type checking” means or what expectations are implied. This has led to a situation where packages claim to support type checking but omit many type annotations. There’s currently no tooling that validates the level of “type completeness” for a package, so even well-intentioned package maintainers are unable to confirm that their packages are properly and completely annotated. This leads to situations where type checkers and language servers need to fall back on type inference, which is costly and gives inconsistent results across tools. Ideally, all py.typed packages would have their entire public interface completely annotated. I’m working on a new feature in Pyright that allows package maintainers to determine whether any of the public symbols in their package are missing type annotations. To do this, I need to clearly define what constitutes a “public symbol”. In most cases, the rules are pretty straightforward and follow the naming guidelines set forth in PEP 8 and PEP 484. For example, symbols that begin with an underscore are excluded from the list of public symbols. One area of ambiguity is related to import statements. PEP 484 indicates that within stub files, a symbol is not considered exported unless it is used within an import statement of the form `import x as y` or `from x import y as z` or `from x import *`. The problem is that this rule applies only to “.pyi” files and not to “.py” files. For packages that use inlined types, it’s ambiguous whether an import statement of the form `import x` or `from y import x` should treat `x` as a public symbol that is exported from that module. I can think of a few solutions here: 1. For py.typed packages, type checkers should always apply PEP 484 import rules for “.py” files. If a symbol `x` is imported with an `import x` or `from y import x`, it is treated as “not public”, and any attempt to import it from another package will result in an error. 2. For py.typed packages, PEP 484 rules are _not_ applied for import statements. This maintains backward compatibility. Package maintainers can opt in to PEP 484 rules using some well-defined mechanism. For example, we could define a special flag “stub_import_rules” that can be added to a “py.typed” file. Type checkers could then conditionally use PEP 484 rules for imports. Option 1 will likely break some assumptions for existing packages. Option 2 avoids that break, but it involves more complexity. Any suggestions? Thoughts? -Eric --- Eric Traut Contributor to Pyright and Pylance Microsoft

4 14

2022

2021

2020

2019

2018

Typing-sig October 2020