Mailman 3 June 2023 - Typing-sig

Re: Typing Meetup
by S Pradeep Kumar 19 Sep '23

19 Sep '23

Updating the Typing Meetup date to Wednesday as there were some conflicts. Let me know if there are any major time conflicts. Date: *Wednesday, November 03, 2021.*Time: 10 am San Francisco time (UTC-7) / 5 pm London time (UTC) Duration: 1 hour Potential discussion topics: + Typing **kwargs using a TypedDict (Franek Magiera?, Pradeep) [1] + Remaining questions about Callable syntax (Steven) + Non-required Protocol fields (Sebastian Rittau?) [2] + Type support for sealed classes (Adrian Freund?) [3] + Safe, typed JSON validation + Type system documentation (Shannon?) If you're working on these or any other topics and would like to start a discussion at the meetup, reach out to me. If you have interesting topics you'd like to see discussed, share them too. Once we have topics, we can always start a discussion even if there's no proposer yet. We will probably have time for 3-4 topics in this session. On Fri, Oct 15, 2021 at 12:55 PM S Pradeep Kumar <gohanpra(a)gmail.com> wrote: >>> >>>> Hello all, >>>> >>>> We plan to hold a virtual Typing Meetup to discuss key typing-related >>>> issues in real-time. The idea is that we can make progress on important or >>>> controversial discussions, get early feedback on in-progress PEPs, and >>>> gauge interest in new proposals. >>>> >>>> A virtual meetup will be more accessible to people in other timezones >>>> than Bay Area in-person meetups. It should also help consolidate >>>> discussions that are often spread out across mailing lists or Github repos. >>>> The tentative plan is to have a monthly meetup, but we can change that as >>>> needed. >>>> >>>> Date: Monday, November 01, 2021. >>>> Time: 10 am San Francisco time (UTC-7) / 5 pm London time (UTC) >>>> Duration: 1 hour >>>> >>>> Time and date are flexible. I'll share the zoom link and agenda the >>>> week before. I'll also record the meetup and share afterwards. >>>> >>>> If you're interested in presenting a new proposal or your work on some >>>> existing proposal, please respond here. We can shoot for 15 minutes per >>>> presentation. I know there are quite a few interesting threads on >>>> typing-sig that haven't seen much activity in a while :) >>>> -- >>>> S Pradeep Kumar >>>> _______________________________________________ >>>> Typing-sig mailing list -- typing-sig(a)python.org >>>> To unsubscribe send an email to typing-sig-leave(a)python.org >>>> https://mail.python.org/mailman3/lists/typing-sig.python.org/ >>>> Member address: guido(a)python.org >>> >>> Links: [1] Typing **kwargs using TypedDict: https://mail.python.org/archives/list/typing-sig@python.org/thread/IUCXT2IQ… [2] Non-required Protocol fields: https://mail.python.org/archives/list/typing-sig@python.org/thread/P6FCGRJU… [3] Sealed classes: https://mail.python.org/archives/list/typing-sig@python.org/thread/QSCT2N4R… -- S Pradeep Kumar

10 60

Type narrowing for TypeGuard in the negative case
by Eric Traut 04 Aug '23

04 Aug '23

Since the introduction of PEP 647 (User-defined Type Guards), we've received a steady stream of input from users saying that they don't like the limitation that type narrowing is applied only in the positive case and is not applied in the negative case. In general, type narrowing is not safe to perform if a user-defined type guard returns False, so I've stood by the original decision not to provide type narrowing in the negative case, but there are cases where such narrowing is safe and desirable. Here's a recent thread where this was discussed in some detail: https://github.com/python/typing/issues/996#issuecomment-1002716830 In this discussion, Ilya Kamen proposed a solution. The proposal is to extend the existing `TypeGuard` to support an optional second type argument. If present, the second argument indicates the narrowed type in the negative type narrowing situation. Here's a simple (admittedly contrived) example: ```python def is_str(val: str | int) -> TypeGuard[str, int]: return isinstance(val, str) def func(val: str | int): if is_str(val): reveal_type(val) # str else: reveal_type(val) # int ``` I've implemented this proposal in the latest published version of pyright (1.1.202) so folks can experiment with it and see if they like it. If there's general consensus that it's the right approach, I can file an amendment for the existing PEP 647 to include this functionality. It was trivial to add to pyright, so I'm optimistic that it would likewise be easy to add to the other type checkers. Another common request for PEP 647 is the desire for an "assert" form of TypeGuard — a way to indicate that a function performs runtime validation of a type, raising an exception if the type is incorrect. It occurred to me that this two-argument form of `TypeGuard` could also be used to handle this use case. The second argument would be specified as a `NoReturn`. I've provisionally implemented this in pyright as well (although it's not in the currently-published version — you'll need to wait until 1.1.203 is published). Here's what this would look like: ```python def validate_sequence(val: Sequence[_T | None]) -> TypeGuard[Sequence[_T], NoReturn]: """Verifies that the sequence contains no None values""" if len([x for x in val if x is not None]) > 0: raise Exception() return True def func(val: Sequence[int | None]): validate_sequence(val) reveal_type(val) # Sequence[int] ``` I'm interested in input on these proposals. -Eric -- Eric Traut Contributor to Pyright & Pylance Microsoft

7 8

Union with single type
by Patrick Arminio 29 Jun '23

29 Jun '23

Hi folks! (Context) I've recently updated my GraphQL library to allow declaring unions like this: ``` MyGraphQLUnion = Annotated[Union[A, B], strawberry.union(name="MyGraphQLUnion")] ``` The previous syntax was this: ``` MyGraphQLUnion = strawberry.union(name="MyGraphQLUnion", types=(A, B)) ``` Which wasn't type-friendly (we used to use a mypy plugin for it), so we added support for Annotated 😊 Unfortunately, we did find an issue with this and it's the fact that Unions in Python don't seem to support only one element, while GraphQL does, here's what I get when I try to create a Union with only one type ``` Python 3.11.0 (main, Jun 13 2023, 14:51:07) [Clang 14.0.3 (clang-1403.0.22.14.1)] on darwin Type "help", "copyright", "credits" or "license" for more information. >>> from typing import Union >>> int == Union[int] True >>> Union[int] <class 'int'> >>> ``` My question is, is this ok? Do we want to allow creating unions with only one type? Does it make sense in Python? (In GraphQL it does make sense since adding new members to a union changes the way you fetch the data)

4 6

Opened and changed typing issues week 2023-W25
by Typing Bot 26 Jun '23

26 Jun '23

The following is an overview of all issues and pull requests in the typing repository on GitHub with the label 'topic: feature' that were opened or updated last week, excluding closed issues. --------------------------------------------------- The following issues and pull requests were updated last week: #1372 Add PartialApplication opened by @NeilGirdhar https://github.com/python/typing/issues/1372 --------------------------------------------------- All issues and pull requests with the label 'topic: feature' can be viewed under the following URL: https://github.com/python/typing/issues?q=label%3A%22topic%3A+feature%22

1 0

Adding a type to denote string literals as being an attribute name i.e AttributeName[SomeClass]
by Matt Barker 24 Jun '23

24 Jun '23

Oftentimes a string literal must be used to specify an attribute name in python, typically when the dot attribute reference operator is not available because the property being accessed is not available or not known until run time. This tends to break a lot of the functionality of static type checkers however. An example from the pydantic validators documentation shows this multiple times: @validator('password2') # String literal of property name def passwords_match(cls, v, values, **kwargs): if 'password1' in values and v != values['password1']: # And also here, twice raise ValueError('passwords do not match') return v where 'password1' and 'password2' are assumed to be a property names in the pydantic model. But there is currently no convenient way of telling the static type checker to warn us if 'password1' is not a property of the model, and when refactoring to change the name of 'password1' to something else the string literal will be missed in the update. This problem could be solved with a new type that denotes a string literal as being a property name of an object. In the pydantic validator example given above the syntax might look like: @validator('password2') def passwords_match(cls, v, values, **kwargs): pw_str: AttributeName[Self] = 'password1' if pw_str in values and v != values[pw_str]: raise ValueError('passwords do not match') return v And the validator decorator might also make use of the type hint as bellow: def validator(attr_name: AttributeName[Self]): ... Additionally often times developers will have to work with a non python object representation of some data that uses the same attribute names, such as when performing database operations. collection.update_one( {'_id': document_id}, {'$set': {'job': 'teacher'}} # here 'job' is a property of a person dataclass stored unmodified in a MongoDB database ) This could be rewritten in a way that respects the relationship the string literal 'job' has with the Person class that was originally stored in the database: update: AttributeName[Person] = {'job': 'teacher'} collection.update_one( {'_id': document_id}, {'$set': update} ) Furthermore a database query to retrieve this object would do so in the form of a dictionary, which could be typed as dict[AttributeName[Person], str] for autocomplete reasons etc. May also be a legitimate use case for inlined type hints to reduce the verbosity of having an extra variable to type hint as with the examples above. Anyone else think this would be a useful python enhancement?

2 2

PEP 718: subscriptable functions
by James H-B 24 Jun '23

24 Jun '23

Hi, I'm cross posting this for visibility. I've recently proposed PEP 718 (https://pep-previews--3179.org.readthedocs.build/pep-0718/) which allows for subscripting function, method and builtin functions/methods at runtime. I didn't imagine it'd require massive changes to type checkers but I thought it'd be nice to let everyone know about this (unless I'm being very naive). Thanks, James H-B.

1 0

How about adding an additional proerty type to TypeDict
by Justin Black 22 Jun '23

22 Jun '23

When handling json schema data, there is the concept of additional properties. For a given map there are - required key value pairs with known string keys and known value types - optional key value pairs with known string keys and known value types (pairs may be omitted) - optional key value pairs with unknown string keys and known value types (pairs may be omitted) Here is an example of what an implementation could look like it uses additional_prop_type to define the additional properties type ``` class SomeMapRequired(typing.AddPropsTypedDict): a: int class SomeMapOptional(typing.AddPropsTypedDict, total=False, additional_prop_type=float): b: str class SomeMap(SomeMapRequired, SomeMapOptional): pass inst: SomeMap = {'a': 1} inst: SomeMap = {'a': 1, 'b': ''} inst: SomeMap = {'a': 1, 'someAddProp': 1.23} inst: SomeMap = {'a': 1, 'someAddProp': None} # fails, additional_prop_type wrong ``` Having this would help immensely with storing type hints for json schema data.

5 5

Release: typing-extensions 4.7.0rc1
by Jelle Zijlstra 21 Jun '23

21 Jun '23

I just released: https://pypi.org/project/typing-extensions/4.7.0rc1/ This is the first time we are using a release candidate for typing-extensions. After the last feature release broke some projects, several users suggested using release candidates as a means to detect such breakages before a full release. Please try out the release candidate on your projects and let us know (by opening an issue https://github.com/python/typing_extensions/issues/new) if you encounter any problems. If there are no major issues, I expect to release 4.7.0 final one week from today, on June 28. We also started running the test suites of some prominent third-party projects against the main branch of typing-extensions (thanks to Alex Waygood), which already helped us find several problems. As of this release, typing-extensions will re-export all non-deprecated names in the typing module so that users don't have to remember whether to import some particular name from typing or typing-extensions. In addition, the release adds new introspection helpers get_protocol_members() and is_protocol(), officially declares support for Python 3.12, and makes many tweaks and bug fixes related to Protocol and TypedDict. Full changelog: - Add `typing_extensions.get_protocol_members` and `typing_extensions.is_protocol` (backport of CPython PR #104878). Patch by Jelle Zijlstra. - `typing_extensions` now re-exports all names in the standard library's `typing` module, except the deprecated `ByteString`. Patch by Jelle Zijlstra. - Due to changes in the implementation of `typing_extensions.Protocol`, `typing.runtime_checkable` can now be used on `typing_extensions.Protocol` (previously, users had to use `typing_extensions.runtime_checkable` if they were using `typing_extensions.Protocol`). - Align the implementation of `TypedDict` with the implementation in the standard library on Python 3.9 and higher. `typing_extensions.TypedDict` is now a function instead of a class. The private functions `_check_fails`, `_dict_new`, and `_typeddict_new` have been removed. `is_typeddict` now returns `False` when called with `TypedDict` itself as the argument. Patch by Jelle Zijlstra. - Declare support for Python 3.12. Patch by Jelle Zijlstra. - Fix tests on Python 3.13, which removes support for creating `TypedDict` classes through the keyword-argument syntax. Patch by Jelle Zijlstra. - Fix a regression introduced in v4.6.3 that meant that ``issubclass(object, typing_extensions.Protocol)`` would erroneously raise ``TypeError``. Patch by Alex Waygood (backporting the CPython PR https://github.com/python/cpython/pull/105239). - Allow `Protocol` classes to inherit from `typing_extensions.Buffer` or `collections.abc.Buffer`. Patch by Alex Waygood (backporting https://github.com/python/cpython/pull/104827, by Jelle Zijlstra). - Allow classes to inherit from both `typing.Protocol` and `typing_extensions.Protocol` simultaneously. Since v4.6.0, this caused `TypeError` to be raised due to a metaclass conflict. Patch by Alex Waygood. - Backport several deprecations from CPython relating to unusual ways to create `TypedDict`s and `NamedTuple`s. CPython PRs #105609 and #105780 by Alex Waygood; `typing_extensions` backport by Jelle Zijlstra. - Creating a `NamedTuple` using the functional syntax with keyword arguments (`NT = NamedTuple("NT", a=int)`) is now deprecated. - Creating a `NamedTuple` with zero fields using the syntax `NT = NamedTuple("NT")` or `NT = NamedTuple("NT", None)` is now deprecated. - Creating a `TypedDict` with zero fields using the syntax `TD = TypedDict("TD")` or `TD = TypedDict("TD", None)` is now deprecated. - Fix bug on Python 3.7 where a protocol `X` that had a member `a` would not be considered an implicit subclass of an unrelated protocol `Y` that only has a member `a`. Where the members of `X` are a superset of the members of `Y`, `X` should always be considered a subclass of `Y` iff `Y` is a runtime-checkable protocol that only has callable members. Patch by Alex Waygood (backporting CPython PR https://github.com/python/cpython/pull/105835).

1 0

Any way to express that a dict is expected to have a key(s) but don't care about other keys?
by Brett Cannon 20 Jun '23

20 Jun '23

Looking at TypedDict it seems that it does not like the idea of superfluous keys existing in a dict. In my case I have a REST call returning JSON for an error and all I'm guaranteed is to get an "error" key. But there may be extra keys that provide more details about the specific error. Unfortunately I don't know the exhaustive list of keys so I can't use a TypedDict with `totality == False` to try to be complete. I can't think of any way to somehow say "require/expect this one key, but other keys are OK". Am I missing anything that would let me do this?

17 35

Further questions on interaction of LiteralString with Generics
by Mikhail Golubev 19 Jun '23

19 Jun '23

Hi typing people! We have been implementing LiteralString (PEP 675) support in PyCharm and even though the PEP is written wonderfully and most of the proposal seems straightforward to implement, I just can't wrap my head around how this feature should work with generics without breaking existing code. There is a brief section about it in the PEP but it doesn't cover my concerns, so I decided to bring it here. Sorry if it has been discussed already, I couldn't find anything relevant in the typing-sig archives. The main question is "how far" inferred LiteralStrings should be propagated through substitution of type parameters during the inference. Consider the following example def expects_literal_string(s: LiteralString) -> None: pass s: str = ... xs = ['foo', 'bar'] # inferred type: list[LiteralString] expects_literal_string(''.join(xs)) # ok! it's appealing to infer the type list[LiteralString] for xs here right away to make the result of ''.join(xs) be LiteralString as well (there is an overload for str.join in Typeshed) however doing so will trigger a typing error for the following unassuming code xs.append(s) # type error: expected LiteralString, got str Of course, the same reasoning applies to user-defined generics, not having their own literal syntax T = TypeVar('T') class Box(Generic[T]): def __init__(self, x: T) -> None: self.value = x def set(self, x: T) -> None: self.value = x box = Box('foo') box.set(s) # type error: expected LiteralString, got str One approach (though very limiting) might be to infer list[LiteralString] only in cases where a collection literal is used directly, not assigned to a name, e.g. expects_literal_string(''.join(['foo', 'bar'])) # ok! but even then it might cause some problems, such as here, where a type containing LiteralString "over-constraints" possible values for other parameters sharing the same TypeVar def couple(first: T, second: T) -> tuple[T, T]: return first, second couple(['foo'], [s]) # type error: expecting list[LiteralString], got list[str] I would argue that even if the first argument was explicitly annotated here (for instance to pass it to str.join), it would still be a surprise for a user that an existing call to couple now leads to a type checker error. xs: list[LiteralString] = ['foo'] expects_literal_string(''.join(xs)) couple(xs, [s]) # type error: expecting list[LiteralString], got list[str] I see that Mypy support of LiteralString is still in progress. I'm curious how other type checkers (pyright, Pyre) handle such cases gracefully. I guess there are a number of different strategies to allow gradually introducing LiteralString to existing type hinted code base. For instance: Do you upcast LiteralString to str, "erasing" it, whenever it's captured in a type parameter? Do you require all generic types possibly containing LiteralString values to be explicitly type hinted? Thanks for sharing the knowledge! -- Mikhail Golubev Software Developer JetBrains http://www.jetbrains.com The drive to develop

3 3