I don't see much overlap between user-defined type guards and `typing.cast`. `cast` is never used for conditional type narrowing, nor can it affect control flow within a program. `cast` also doesn't involve any logic at runtime; it simply returns the value that it was passed. Plus, as you've pointed out, `cast` does not enforce that the output type is a narrower type than the input type. A user-defined type guard is a user-defined function that is called in a conditional expression within an if/else statement. It contains logic that executes at runtime, and the bool response changes the flow of control in the program. At static type checking time, a user-defined type guard allows the type checker to understand more about the implied type of an expression within the if/else code blocks.

I'm specifically trying to address cases where overloads (and other existing mechanisms) do not solve the problem. The examples I provided at the top of this thread cannot be handled with overloads. -Eric -- Eric Traut Contributor to Pyright and Pylance Microsoft Corp.

Unfortunately, that doesn't provide the desired affect. That only works if the type checker already knows the type of the parameter matches `Tuple[_T, _T]` because it's going to be matching the call to the overload that works (versus we need to pick how to narrow the type at the call site based on the return value). Take this full example: ``` from typing import Literal, Tuple, TypeVar, overload _T = TypeVar("_T") @overload def is_two_element_tuple(val: Tuple[_T, _T]) -> Literal[True]: ... @overload def is_two_element_tuple(val: Tuple[_T, ...]) -> bool: ... def is_two_element_tuple(val): return len(val) == 2 def print_coord(c: Tuple[int, ...]): if is_two_element_tuple(c): reveal_type(c) # Should be Tuple[int, int] x, y = c print(f"x={x}, y={y}") else: reveal_type(c) # Should be Tuple[int, ...] print("I dunno.") ``` In this case, the code is intending to have the check verify that the tuple has two elements for some arbitrary tuple. We don't know that c has two elements until we do the check. mypy says: main.py:16: note: Revealed type is 'builtins.tuple[builtins.int]' main.py:20: note: Revealed type is 'builtins.tuple[builtins.int]' pyright says: 16:21 - warning: Type of "c" is "Tuple[int, ...]" 20:21 - warning: Type of "c" is "Tuple[int, ...]" Compare that to the original example: ``` def is_two_element_tuple(val: Tuple[_T, ...]) -> TypeGuard[Tuple[_T, _T]]: return TypeGuard(len(val) == 2) ``` Here, the input is the generic type ("I will prove something about a value of this type"), and the output is the specific one ("I will either tell you True if it's of type Tuple[_T, _T] or False otherwise").

On Sat, Oct 3, 2020 at 12:56 PM Jake Bailey via Typing-sig < typing-sig@python.org> wrote:

I've been thinking about this one, and I think that it would be better to not make `TypeGuard` an instantiable type, instead having it be a special form that type checkers treat as `bool`. There's already some precedence for types that "don't really exist" here with `Literal`, where you wouldn't write `return Literal(2)`, but just `return 2`.

It's true for all special forms, right? You don't use Union when a return type is `Union[A, B]`, and you don't use Callable when returning a function.

Firstly, I think this will make the functions read better. E.g.:

``` def is_two_element_tuple(val: Tuple[_T, ...]) -> TypeGuard[Tuple[_T, _T]]: return len(val) == 2

def is_string_list(val: List[object]) -> TypeGuard[List[str]]: return all(isinstance(x, str) for x in val) ```

Secondly (like `Literal`), not introducing a type here would mean that existing functions could be converted to type guards without modification. For example, there are a few functions in `inspect` that could be converted to type guards:

``` # Before def ismodule(object: object) -> bool: ... # After, maybe from types import ModuleType def ismodule(object: object) -> TypeGuard[ModuleType]: ... ```

If we required type guard functions to return an instance of `TypeGuard`, then no existing function could become a guard, because they will return `TypeGuard`'s parent type.

I don't understand the latter phrase (what's TypeGuard's parent type anyway?) but indeed it seems like a friendlier API. That said, I think I've wanted this kind of functionality a few times, and long ago I recall the mypy team discussing this. For example in https://github.com/python/mypy/issues/5206, which mentions TypeScript type guards. I also feel however that the need for this is much less in Python than in TypeScript -- in most cases (though not the examples Eric showed) you can just use isinstance(), since Python doesn't have the feature that makes this needed in TypeScript (Python's class instances aren't dicts). So I wouldn't hurry to implement this. Possibly pyright could prototype this as a pyright extension and you could propose a PEP based on your experience using that. -- --Guido van Rossum (python.org/~guido) *Pronouns: he/him **(why is my pronoun here?)* http://feministing.com/2015/02/03/how-using-they-as-a-singular-pronoun-can-c...

Am 30.09.20 um 23:40 schrieb Eric Traut:

I occasionally receive questions from pyright users about extending type narrowing to handle cases that are not possible with today’s Python type system.

Typescript provides a useful facility called [user-defined type guards]( https://www.typescriptlang.org/docs/handbook/advanced-types.html#user-define...) that allows users to extend the notion of type narrowing. It would be straightforward to add this idea to Python. Do others think this would be useful, and is there any interest in standardizing this?

In typescript I find user-defined type guards to be quite helpful and there have been a few occasions where I wished to have them in Python. Unfortunately I don't remember the situations exactly. But parsing JSON is one example where it can be useful: ``` class SomeShape(TypedDict):     ... def has_some_shape(j: object) -> TypeGuard[SomeShape]:     ... def do_stuff(shape: SomeShape) -> None:     ... j = json.loads(...) if has_some_shape(j):     do_stuff(j) elif has_some_other_shape(j):     do_other_stuff(j) else:     raise ValueError(...) ``` do_stuff() and do_other_stuff() can be safely typed then.

Here’s a concrete proposal. We could add a new generic type called `TypeGuard` in the `typing` module. It would be a subclass of bool, defined simply as:

def is_two_element_tuple(val: Tuple[_T, ...]) -> TypeGuard[Tuple[_T, _T]]: return TypeGuard(len(val) == 2)

Why is wrapping the response in TypeGuard() necessary? Couldn't we just do the following in a TypeGuard-aware type checker? def is_two_element_tuple(val: Tuple[_T, ...]) -> TypeGuard[Tuple[_T, _T]]: return len(val) == 2 - Sebastian

I've implemented a prototype of `TypeGuard` in the latest version of pyright if you'd like to try it out. I've made a few small changes to my original proposal: 1. `TypeGuard` is now effectively treated as an alias for `bool`, eliminating the need to construct an actual `TypeGuard` object for the return value. ```python from typing_extensions import TypeGuard def is_two_element_tuple(val: Tuple[_T, ...]) -> "TypeGuard[Tuple[_T, _T]]": return len(val) == 2 ``` 2. In my original proposal, I said that type guard functions should be limited to a single parameter. I've found uses for multiple parameters, and I don't see a justification for this limitation. The expression passed as the argument for the first parameter is assumed to be the one that is narrowed. The other arguments are treated just like any others by the type checker. ```python def is_string_list(val: List[Union[str, float]], allow_empty: bool) -> "TypeGuard[List[str]]": if len(val) == 0: return allow_empty return all(isinstance(x, str) for x in val) ``` 3. I'm not enforcing that the output type be strictly narrower than the input type. This would preclude some valuable use cases, such as the previous one. I will write and submit a PEP as you suggested. -- Eric Traut Contributor to Pyright and Pylance Microsoft Corp.

Am 20.10.20 um 23:50 schrieb Eric Traut:

As Guido suggested, I've created a new draft PEP for "User-defined Type Guards". Input is welcome.

Draft Proposal (in fork): https://github.com/erictraut/peps/blob/master/pep-9999.rst Draft Proposal (branch compare): https://github.com/python/peps/compare/master...erictraut:master

Thank you for the PEP, I really like it. A few remarks: * In the first example of the chapter "TypeGuard Type" you return "TypeGuard(len(val) == 2)" without explanation, while in the second example you just return "True" (my preference). * " User-defined type guards apply narrowing only in the positive case (the if clause)." Why? In the examples you give, type checkers should be able to safely narrow the "else" as well, shouldn't they? There is one use case (that might be out of scope for this PEP) that is not covered, but is quite useful. I call it "type assertions". For example "assert isinstance(foo, Bar)" already narrows "foo" in the following code. I'll give one real-life example that we use quite a lot: def parse_foo_json_request(j: Any) -> None:     validate_foo_json(j)     # Now the shape of j is known and data can be safely extracted. def validate_foo_json(j: Any) -> None:     # Do various checks on j, raise a complex exception if is doesn't     # match the expected shape.     pass This can't be rewritten using a type guard, since the exception raised within the validation function depends on the exact problem with the given shape. This will later be used to give a detailed, structured explanation of went wrong to the caller of a web API. With the current PEP I would have to write this is something like this: def parse_foo_json_request(j: Any) -> None:     if not validate_foo_json(j):         # dummy, will never happen         raise RuntimeError():     # Now the shape of j is known and data can be safely extracted. def validate_foo_json(j: Any) -> TypeGuard[Foo]:     # Do various checks on j, raise a complex exception if is doesn't     # match the expected shape.     return True One idea I have (not though through) is to be able to write something like this: def validate_foo_json(j: TypeGuard[Foo]) -> None:     # Do various checks on j, raise a complex exception if is doesn't     # match the expected shape.     return True I.e. if the type guard is at an argument position, the validation function will not return if it doesn't match the type. But as I said, this might be out of scope of the current PEP.  - Sebastian

I'm very interested in this -- however this week the Python Core Dev sprint is eating up all my time. Sometime in the next few weeks I hope to catch up with various typing-sig topics (also the variadics pep). On Tue, Oct 20, 2020 at 2:50 PM Eric Traut wrote:

As Guido suggested, I've created a new draft PEP for "User-defined Type Guards". Input is welcome.

Draft Proposal (in fork): https://github.com/erictraut/peps/blob/master/pep-9999.rst Draft Proposal (branch compare): https://github.com/python/peps/compare/master...erictraut:master

-- Eric Traut Contributor to Pyright and Pylance Microsoft Corp. _______________________________________________ Typing-sig mailing list -- typing-sig@python.org To unsubscribe send an email to typing-sig-leave@python.org https://mail.python.org/mailman3/lists/typing-sig.python.org/ Member address: guido@python.org

-- --Guido van Rossum (python.org/~guido) *Pronouns: he/him **(why is my pronoun here?)* http://feministing.com/2015/02/03/how-using-they-as-a-singular-pronoun-can-c...

Thanks Eric for the PEP. I have some questions: *Type checkers should assume that type narrowing should be applied to the expression that is passed as the first argument to a user-defined type guard.*What should happen in case of instance methods, class methods or static methods? The first two cases have an implicit self/class argument? Would the type guyard apply to that argument or to the first explicit parameter? What about properties? Would they be supported? *User-defined type guards apply narrowing only in the positive case (the if clause).*Like Sebastian Rittau mentioned, what is the rationale behind this? On Wed, Oct 21, 2020 at 2:41 PM Jake Bailey via Typing-sig < typing-sig@python.org> wrote:

We talked about this on gitter, but I'll sum it up here as well. I think `filter` could be modified to use a guard:

``` @overload def filter(__function: Callable[[_T1], TypeGuard[_T2]], __iterable: Iterable[_T1]) -> Iterator[_T2]: ... ```

Unfortunately, you have to encode the narrowing behavior into the signature (as there's no code flow graph), but I suppose this makes some sense.

`filterfalse`, not so much, because the way `TypeGuard` has to work only allows proving of truths. You mentioned TS's `Exclude`, which would allow proving of truths by having a `TypeGuard` that could say something like "I prove that the type of the first parameter is not of type `_T2`", for example. _______________________________________________ Typing-sig mailing list -- typing-sig@python.org To unsubscribe send an email to typing-sig-leave@python.org https://mail.python.org/mailman3/lists/typing-sig.python.org/ Member address: skreft@gmail.com

-- Sebastian Kreft

Thanks Eric for updating the PEP. I have some questions: *If a type guard function is implemented as an instance method or class method, the first explicit argument maps to the second parameter (after "self" or "cls").* What's the rationale behind this? AFAIK in Typescript you can write user-defined typeguards asserting about the nature of the calling object. Using this is SomeInterface. This could be useful to narrow an object implementing a protocol. What's the rationale for user-defined type guards to only apply in the positive case? In fact, of all the type guards listed in the motivation, only the truthy-typeguard applies only in the positive case, meaning user-defined typeguards are going to be less powerful than builtin ones. Maybe we could make TypeGuard exhaustive by default (working on positive and negative cases) and making the type in the negative case to be T - guarded_type. Then we could receive an optional second argument, so that the type in the negative case is: TypeGuard[guarded_type] -> (T - guarded_type) TypeGuard[guarded_type, negative_extra_types] -> Union[T - guarded_type, negative_extra_types] in that way you could write all the guards listed in the motivation as user-defined ones and even narrow the type further os some existing guards: # As shown in the motivation the negative case still has type Optional[str] def truthy_string(s: Optional[string]) -> Typeguard[str, str]: return bool(s) # In the negative case it will have type Union[None, Literal['']] def truthy_string(s: Optional[string]) -> TypeGuard[str, Literal['']]: return bool(s) To mimic the current proposal of guards only working in the positive case one could write: def guard(t: T) -> TypeGuard[U, T]: return ... Finally, the example is_str_list should probably read: if len(val) == 0: return allow_empty Otherwise, it will always return True when the list is empty. On Mon, Nov 9, 2020 at 12:02 PM Eric Traut wrote:

Thanks for the feedback everyone. I've updated the language in the PEP to clear up some points that were raised.

I also fixed the bug in the code sample that Sebastian noted. It is not be necessary to wrap the result in a TypeGuard object. Any bool return result is fine.

There were questions about what happens when a type guard function is implemented as a method (instance or class). In this case, the value being tested is still the first argument of the bound method. It maps to the second parameter of the method — the one after the "self" or "cls" parameter.

I also clarified what I mean by " User-defined type guards apply narrowing only in the positive case (the if clause)." If that's still not clear, let me know.

Sebastian, you asked about "type assertions". For that use case, I recommend a different approach. The validation routine should return the typed value that it's validating and raise an exception if it doesn't match that type.

```python def validate_foo_json(j: Any) -> Foo: if xxx: raise RuntimeError() return cast(Foo, j)

def parse_foo_json_request(j: Any) -> None: validated_j = validate_foo_json(j) ```

Guido, I'm interested in your feedback on the PEP once you have a chance to review. _______________________________________________ Typing-sig mailing list -- typing-sig@python.org To unsubscribe send an email to typing-sig-leave@python.org https://mail.python.org/mailman3/lists/typing-sig.python.org/ Member address: skreft@gmail.com

-- Sebastian Kreft

Thanks Eric for the answers, in general I think it would be valuable to add some of these reasons to the PEP itself. One point that hasn't been raised is what happens in multithreaded code, as nothing would guarantee that the guard is still met. Another point not mentioned is whether typeguards will work for async functions. On Sat, Nov 14, 2020 at 2:03 PM Eric Traut wrote:

...

What's the rationale behind ["the first explicit argument maps to the second parameter"]?

Without this, it would be impossible to implement a user-defined type guard function as a method. If you want to apply a type guard to self, you can pass `self` as the first argument, which maps to the first explicit parameter.

That would be really not ergonomic to define and use. Imagine looking at something like result.has_data(result) or def has_data(self, self) (see example below for more context). It's hard for me to imagine type guards really using multiple arguments, especially self or class. I get your example about allowing empty when checking for a list of strings, but that could be written as two different guards. What would be the semantics of evaluating multi-argument type guards which are instance or classmethods. What would happen if you modify self or the class, which may affect the result of the typeguard itself. (The same applies for any multi-argument typeguard.). For what is worth, I couldn't find any typeguards with multiple arguments in any of the JS dependencies we are using at work (node, google apis, lodash, etc).

...

What's the rationale for user-defined type guards to only apply in the positive case?

The problem is that `T - guarded_type` is not well defined in the general case. This sort of type algebra is defined only in very specific cases (such as with unions). In general, the negative case must assume no narrowing. For comparison, this is how user-defined type guards work in TypeScript, by necessity.

Thanks, I was looking for something like this. However that's not how TypeScript works in all cases. In fact the very documentation https://www.typescriptlang.org/docs/handbook/advanced-types.html#user-define... explains that they do narrow the type in the negative case at least for unions. See https://www.typescriptlang.org/play?#code/MYGwhgzhAEBiCWEAW0DeBYAUNaEDu8AtgB... Maybe a comparison with user-defined typeguards in TS would be really helpful.

Can you think of an example of a user-defined type guard (one that is not handled by built-in type guards) that could handle the negative case? None of the examples I provided in the draft PEP would work.

All the PEP examples would work in the negative case, however expressing the resulting type is not possible with current types. One case would be something like the following snippet based on some real TS code written by a third party. class QueryResult(Generic[T]): _data: Optional[bytes] def data(self) -> Optional[T]: return expensive_parse_operation(self._data) def empty(self) -> TypeGuard[MissingQueryResult]: return self._data is None where class MissingQueryResult(Protocol): def data(self) -> None: ... def empty(self) -> Literal[True]: ... Note that without an option to provide the types for the negative case we have to implement another guard def has_data(self) -> TypeGuard[ExistingQueryResult]: return self._data is not None which is not that terrible from the implementor's POV, however for the user of the API, it's a problem as now they do need to remember which variant to use in which case.

Incidentally, I've posted more detailed documentation for the [type narrowing behaviors and all of the built-in type guards that pyright supports]( https://github.com/microsoft/pyright/blob/master/docs/type-concepts.md#type-... ). _______________________________________________ Typing-sig mailing list -- typing-sig@python.org To unsubscribe send an email to typing-sig-leave@python.org https://mail.python.org/mailman3/lists/typing-sig.python.org/ Member address: skreft@gmail.com

In the rejected section Enforcing Strict Narrowing ( https://github.com/erictraut/peps/blob/master/pep-9999.rst#enforcing-strict-...), could you provide some examples of use cases that would be eliminated? Maybe mentioning that the variance of the type, and maybe some other restrictions could make this impractical. However, it may be worth considering enforcing this by default and skip it if another argument is passed, or if the passed type is a protocol or a generic. For example, I would expect the type checker to complain if it finds code like: def unsound_guard(arg: int) -> TypeGuard[str]: return True def unsound_guard_union(arg: TypeA | TypeB) -> TypeGuard[Typea]: # Note the typo here return True For both of the cases above TS does emit an error: A type predicate's type must be assignable to its parameter's type. Type 'string' is not assignable to type 'boolean'. A type predicate's type must be assignable to its parameter's type. Type 'string' is not assignable to type 'number | boolean'. It even checks compatibility of interfaces (protocols). -- Sebastian Kreft

Unfortunately, that doesn't provide the desired affect.

Based on PEP 586 §"Interactions with narrowing" it appears that the syntax mentioned by Ivan Levkivskyi is *supposed* to work, even though mypy (and pyright?) don't appear to implement it. If I copy the example from PEP 586, with minimal changes, and typecheck it with mypy and reveal_type(): from typing import * @overload def is_int_like(x: Union[int, List[int]]) -> Literal[True]: ... @overload def is_int_like(x: object) -> bool: ... def is_int_like(x): ... def foo(scalar: Union[int, str]) -> None: if is_int_like(scalar): reveal_type(scalar) # mypy says: Revealed type is 'Union[builtins.int, builtins.str]' scalar += 3 # PEP says: Type checks: type of 'scalar' is narrowed to 'int' else: reveal_type(scalar) # mypy says: Revealed type is 'Union[builtins.int, builtins.str]' scalar += "foo" # PEP says: Type checks: type of 'scalar' is narrowed to 'str' Notice the annotations in the comments. So one might argue that mypy/pyright should be updated to support the "returns Literal[True]" pattern put forward by PEP 586. I *do* think the TypeGuard-based syntax is easier to understand and more-straightforward than implementing an overload that returns Literal[True]. It will also be easier for folks coming from TypeScript background to understand since there's a direct analogue for TypeGuard in TypeScript. -- David Foster | Seattle, WA, USA Contributor to TypedDict support for mypy

A few comments: 1. The current syntax appears to require that TypeGuard only apply to the *first* parameter of a function (ignoring self or cls, if present) but it might be valuable to apply to a different parameter in certain cases. For example: def conforms_to(type: Type[T], value: object) -> TypeGuard[value=T]: ... tentative spelling ^^^^^^ Considering prior art, TypeScript's version of TypeGuard (a "type predicate") allows naming the parameter it applies to: function isNumber(x: any): x is number { ... } I might suggest that a syntax like `TypeGuard[param=T]` be permitted to specify a particular parameter whose type should be narrowed. Note that keyword argument syntax can't be used in square brackets [] unless/until PEP 637 ("Support for indexing with keyword arguments") at [1] is accepted. Also that PEP is currently slated for Python 3.11+ whereas this PEP is currently slated for Python 3.10+. I could potentially see breaking off the addition of keyword syntax to TypeGuard to a later PEP, to avoid making this PEP depend on PEP 637 ("Support for indexing with keyword arguments"). 2. The discussions in the "Rejected Ideas" section of this PEP need some more elaboration IMHO: 2.1. §"Decorator Syntax":

The use of a decorator was considered for defining type guards.

Here, it would be useful to show an example of what the proposed decorator syntax was. Presumably you're referring to the syntax from PEP 586 §"Interactions with narrowing" and Ivan's comment: from typing import * @overload def is_int_like(x: Union[int, List[int]]) -> Literal[True]: ... @overload def is_int_like(x: object) -> bool: ... def is_int_like(x): ... def foo(scalar: Union[int, str]) -> None: if is_int_like(scalar): scalar += 3 # type of 'scalar' should be narrowed to 'int' else: scalar += "foo" # type of 'scalar' should be narrowed to 'str' 2.2. §"Enforcing Strict Narrowing":

Strict type narrowing enforcement was considered, but this eliminates numerous valuable use cases for this functionality.

I don't actually know what this is talking about. [1]: https://www.python.org/dev/peps/pep-0637/ -- David Foster | Seattle, WA, USA Contributor to TypedDict support for mypy

Catching up... On Sun, Nov 29, 2020 at 11:25 AM David Foster wrote:

A few comments:

1. The current syntax appears to require that TypeGuard only apply to the *first* parameter of a function (ignoring self or cls, if present) but it might be valuable to apply to a different parameter in certain cases. For example:

def conforms_to(type: Type[T], value: object) -> TypeGuard[value=T]: ... tentative spelling ^^^^^^

Considering prior art, TypeScript's version of TypeGuard (a "type predicate") allows naming the parameter it applies to:

function isNumber(x: any): x is number { ... }

I might suggest that a syntax like `TypeGuard[param=T]` be permitted to specify a particular parameter whose type should be narrowed.

Note that keyword argument syntax can't be used in square brackets [] unless/until PEP 637 ("Support for indexing with keyword arguments") at [1] is accepted. Also that PEP is currently slated for Python 3.11+ whereas this PEP is currently slated for Python 3.10+.

I could potentially see breaking off the addition of keyword syntax to TypeGuard to a later PEP, to avoid making this PEP depend on PEP 637 ("Support for indexing with keyword arguments").

Heh. I talked to Anders about this and he said he couldn't recall a single example of a function where the parameter under consideration isn't the first parameter (in fact, he couldn't remember any examples where there even was another parameter). So I think this is pretty theoretical, and I would like not to complicate the proposal to handle this. **But...** Sebastian Kreft seems to have executed a search that came to the same conclusion as Anders (typeguards are one-argument functions), but he is pushing for a way to define a typeguard as a method that guards `self`, e.g. `if query.empty(): ...`. I'm not sure how to address this without adopting David's idea except to propose that if `self` is the *only* argument the typeguard applies to `self`. But that's pretty ad-hoc and ugly. (Note that the TS equivalent would have to involve `this`, which is not explicit in TS, so apparently in TS there's no great need for this.) So in the end David's suggestion of using PEP 637 keyword indexing could be adopted if in the future the need for this use case becomes overwhelming. (It's interesting that PEP 637 seems so popular in typing-sig -- it's needed for variadic generics as well. :-) 2. The discussions in the "Rejected Ideas" section of this PEP need some

more elaboration IMHO:

2.1. §"Decorator Syntax":

...

The use of a decorator was considered for defining type guards.

Here, it would be useful to show an example of what the proposed decorator syntax was. Presumably you're referring to the syntax from PEP 586 §"Interactions with narrowing" and Ivan's comment:

from typing import *

@overload def is_int_like(x: Union[int, List[int]]) -> Literal[True]: ... @overload def is_int_like(x: object) -> bool: ... def is_int_like(x): ...

def foo(scalar: Union[int, str]) -> None: if is_int_like(scalar): scalar += 3 # type of 'scalar' should be narrowed to 'int' else: scalar += "foo" # type of 'scalar' should be narrowed to 'str'

I don't think it was that. The PEP 586 example doesn't actually work, and can't work without changing the specification of @overload. The idea was probably something like this: ``` @typeguard(Union[int, List[int]]) def is_int_like(x: object) -> bool: >implementation> ``` This is inferior because it requires runtime evaluation of the type in the decorator, which would constrain forward references. Definitely worth elaborating.

2.2. §"Enforcing Strict Narrowing":

...

Strict type narrowing enforcement was considered, but this eliminates numerous valuable use cases for this functionality.

I don't actually know what this is talking about.

Presumably this is about whether the type T in TypeGuard[T] must be a subtype of the argument type. (The PEP mentions a reason why this is a bad idea: container types that are invariant.) Sebastian Kreft pointed out that allowing non-strict narrowing by default might cause false negatives for some obvious nonsense (e.g. `def f(a: int) -> TypeGuard[str]`). As a remedy, the type checker could either double check that the types have some overlap (e.g. their join isn't `object`); or (Sebastian's proposal) we could require passing an extra flag to allow non-strict narrowing. I'll let Eric choose here; I'd be okay with either (though probably the ergonomics of the implicit form are slightly better). Again the PEP could use some more words on this topic. Per Sebastian Kreft's suggestion, the PEP could also use some words comparing the proposal to TypeScript. Sebastian Kreft had something about negative type guards (the problem that you can't be sure that the target is *not* of the given type if the assertion fails). I would like to give up on this and proceed as the PEP proposes. Another Sebastian (Rittau) asked about "type assertions" which raise an exception instead of returning False. I wonder if that couldn't be handled by writing an explicit assert statement, e.g. ``` def f(x: Tuple[str, ...]): assert is_two_element_tuple(x) # Now x has type Tuple[T, T] ``` Maybe if the typeguard function raises a more user-friendly exception, it could still be declared as a TypeGuard, and the assert would just exist to help the type checker? Conclusion: There are some loose ends, but I will sponsor this PEP and likely approve it after only minor updates. Eric, please use PEP number 647. -- --Guido van Rossum (python.org/~guido) *Pronouns: he/him **(why is my pronoun here?)* http://feministing.com/2015/02/03/how-using-they-as-a-singular-pronoun-can-c...

Apologies, I got my mypy link slightly wrong. I meant https://github.com/python/mypy/issues/5756 although the issue I linked is related. On Tue, 22 Dec 2020 at 21:55, Shantanu Jain wrote:

One concern I have is an inability to type narrow without creating a whole separate and fairly verbose function.

For instance, I'd often want to do an `is_str_list` check like this (rather than incur a linear cost): ``` if not val or isinstance(val[0], str): print(" ".join(val)) ``` But to accomplish this I'd have to create another function and give it an awkward name like `is_str_list_based_on_first_element` so that it's clear that it's not safe in general.

I'm not convinced that this is much of an improvement over a possible alternative of mandating that type checkers support redefinitions with cast. The following reads really clearly to me: ``` if not val or isinstance(val[0], str): val = cast(List[str], val) print(" ".join(val)) ```

If you're like me and you inline most of the time you do this kind of narrowing, this is much less invasive to get things to type check.

If we got around to adding a safe_cast a la https://github.com/python/mypy/issues/5687, this would also allow the user to explicitly opt in to strict or non strict type narrowing.

This would also easily allow narrowing of object members, as in Sebastian Kreft's QueryResult example.

Cast redefinitions feel really natural to me. I don't know what else one would want a cast redefinition to mean, so there seems little cost for type checkers to go ahead and support it, and we'd have a way to make the motivating use cases type check. Of course, we could then return to TypeGuard if we found it insufficient in practice.

On Tue, 22 Dec 2020 at 19:27, Guido van Rossum wrote:

...

Catching up...

On Sun, Nov 29, 2020 at 11:25 AM David Foster wrote:

...

A few comments:

1. The current syntax appears to require that TypeGuard only apply to the *first* parameter of a function (ignoring self or cls, if present) but it might be valuable to apply to a different parameter in certain cases. For example:

def conforms_to(type: Type[T], value: object) -> TypeGuard[value=T]: ... tentative spelling ^^^^^^

Considering prior art, TypeScript's version of TypeGuard (a "type predicate") allows naming the parameter it applies to:

function isNumber(x: any): x is number { ... }

I might suggest that a syntax like `TypeGuard[param=T]` be permitted to specify a particular parameter whose type should be narrowed.

Note that keyword argument syntax can't be used in square brackets [] unless/until PEP 637 ("Support for indexing with keyword arguments") at [1] is accepted. Also that PEP is currently slated for Python 3.11+ whereas this PEP is currently slated for Python 3.10+.

I could potentially see breaking off the addition of keyword syntax to TypeGuard to a later PEP, to avoid making this PEP depend on PEP 637 ("Support for indexing with keyword arguments").

Heh. I talked to Anders about this and he said he couldn't recall a single example of a function where the parameter under consideration isn't the first parameter (in fact, he couldn't remember any examples where there even was another parameter). So I think this is pretty theoretical, and I would like not to complicate the proposal to handle this. **But...**

Sebastian Kreft seems to have executed a search that came to the same conclusion as Anders (typeguards are one-argument functions), but he is pushing for a way to define a typeguard as a method that guards `self`, e.g. `if query.empty(): ...`. I'm not sure how to address this without adopting David's idea except to propose that if `self` is the *only* argument the typeguard applies to `self`. But that's pretty ad-hoc and ugly. (Note that the TS equivalent would have to involve `this`, which is not explicit in TS, so apparently in TS there's no great need for this.)

So in the end David's suggestion of using PEP 637 keyword indexing could be adopted if in the future the need for this use case becomes overwhelming. (It's interesting that PEP 637 seems so popular in typing-sig -- it's needed for variadic generics as well. :-)

2. The discussions in the "Rejected Ideas" section of this PEP need some

...

more elaboration IMHO:

2.1. §"Decorator Syntax":

...

The use of a decorator was considered for defining type guards.

Here, it would be useful to show an example of what the proposed decorator syntax was. Presumably you're referring to the syntax from PEP 586 §"Interactions with narrowing" and Ivan's comment:

from typing import *

@overload def is_int_like(x: Union[int, List[int]]) -> Literal[True]: ... @overload def is_int_like(x: object) -> bool: ... def is_int_like(x): ...

def foo(scalar: Union[int, str]) -> None: if is_int_like(scalar): scalar += 3 # type of 'scalar' should be narrowed to 'int' else: scalar += "foo" # type of 'scalar' should be narrowed to 'str'

I don't think it was that. The PEP 586 example doesn't actually work, and can't work without changing the specification of @overload.

The idea was probably something like this: ``` @typeguard(Union[int, List[int]]) def is_int_like(x: object) -> bool: >implementation> ``` This is inferior because it requires runtime evaluation of the type in the decorator, which would constrain forward references.

Definitely worth elaborating.

...

2.2. §"Enforcing Strict Narrowing":

...

Strict type narrowing enforcement was considered, but this eliminates numerous valuable use cases for this functionality.

I don't actually know what this is talking about.

Presumably this is about whether the type T in TypeGuard[T] must be a subtype of the argument type. (The PEP mentions a reason why this is a bad idea: container types that are invariant.) Sebastian Kreft pointed out that allowing non-strict narrowing by default might cause false negatives for some obvious nonsense (e.g. `def f(a: int) -> TypeGuard[str]`). As a remedy, the type checker could either double check that the types have some overlap (e.g. their join isn't `object`); or (Sebastian's proposal) we could require passing an extra flag to allow non-strict narrowing. I'll let Eric choose here; I'd be okay with either (though probably the ergonomics of the implicit form are slightly better).

Again the PEP could use some more words on this topic.

Per Sebastian Kreft's suggestion, the PEP could also use some words comparing the proposal to TypeScript.

Sebastian Kreft had something about negative type guards (the problem that you can't be sure that the target is *not* of the given type if the assertion fails). I would like to give up on this and proceed as the PEP proposes.

Another Sebastian (Rittau) asked about "type assertions" which raise an exception instead of returning False. I wonder if that couldn't be handled by writing an explicit assert statement, e.g. ``` def f(x: Tuple[str, ...]): assert is_two_element_tuple(x) # Now x has type Tuple[T, T] ``` Maybe if the typeguard function raises a more user-friendly exception, it could still be declared as a TypeGuard, and the assert would just exist to help the type checker?

Conclusion: There are some loose ends, but I will sponsor this PEP and likely approve it after only minor updates. Eric, please use PEP number 647.

-- --Guido van Rossum (python.org/~guido) *Pronouns: he/him **(why is my pronoun here?)* http://feministing.com/2015/02/03/how-using-they-as-a-singular-pronoun-can-c... _______________________________________________ Typing-sig mailing list -- typing-sig@python.org To unsubscribe send an email to typing-sig-leave@python.org https://mail.python.org/mailman3/lists/typing-sig.python.org/ Member address: hauntsaninja@gmail.com

But to me, basically any cast() has a smell, and I wouldn't want to legitimize it by recommending that idiom.

I think I don't feel this as much as you do... a type guard is just a verbose conditional cast. Using cast has the advantage of not requiring runtime changes / requiring fairly minimal changes to type checkers, but if it's a concern maybe we could give it its own (hopefully less smelly) bikeshed, say assert_static_is_subtype.

If you don't want to define a two-line function, is that because you're writing short scripts? Or is it because you're worried about the runtime cost of the function call? (Then you should be worried about the cost of cast() also. :-)

My reluctance to define a function comes from a couple things: - For examples given in the PEP draft, like `len(val) == 2` or `all(isinstance(x, str) for x in val)`, I'd always write those inline today. Having to move around my code to define a two line but clunky looking function feels less ergonomic than adding a cast or an annotation. Especially if the function just gets used once or twice. - Runtime type checking has a lot of caveats. For my example of `not val or isinstance(val[0], str)`, picking a name like is_str_list_based_on_first_element to describe those caveats is clunkier than the actual code that does the check. - I sometimes write short scripts! None of that is deal breaking, but since I didn't love TypeGuard, I thought it was worth bringing up cast as a lightweight construct that could be used to make code that is currently inexpressible in our type system expressible. In my mind, if cast supported redefinition like this today, the bar for the acceptance of a type guard like construct would be higher. ... discussion of safe cast ...

I bungled this minor suggestion — my terminology and links were a mess :-) I meant to refer to https://github.com/erictraut/peps/blob/master/pep-9999.rst#enforcing-strict-... and some previous discussion on this thread. That is, should it be possible to further check that we only narrow to a subtype of the original type, e.g. `Tuple[int, ...]` to `Tuple[int, int]` as compared to `List[object]` to `List[str]` (or `int` to `str`). If this were desirable, it could be accomplished by use of a different cast function, perhaps called cast_narrower or downcast. Hm. In my experience typeguard functions often naturally occur (though

perhaps not for the examples given in the PEP) and they often abstract away checks that should not be inlined. So perhaps the use cases are just different?

Yeah, the use cases are definitely complementary / do not preclude one another. Like you say, I think there's an unmet need for `assert isinstance(x, TypeForm)` and cast redefinition could provide a solution for that. That said, I'd like to know more about what use cases you envision for TypeGuard beyond code that currently tends to be inlined / maybe the PEP could benefit from a discussion of those. The TypedDict example in the current draft is well taken, though :-) (I'll note that you could still use cast redefinition in the appropriate scope, but it might feel annoying). Shantanu P.S.: I feel the need to further clarify my confusing mention of an ill conceived safe_cast. This is basically irrelevant to the topic at hand, so please ignore. The kind of casting that is typically safe is an "upcast" or "cast_broader". This corresponds to: ``` cast_broader(Union[int, str, bytes], 5) ``` Relevant mypy issue link: https://github.com/python/mypy/issues/5756 This relates to type narrowing chiefly in that it makes it possible to avoid unwanted narrowing. This was relevant in the previous typing-sig discussion about whether a variable annotation should effectively perform an upcast. See https://github.com/python/mypy/issues/2008 and https://mail.python.org/archives/list/typing-sig@python.org/thread/GYVM5KEE6... There's also "downcast" or "cast_narrower", which is pertinent here. This corresponds to: ``` x: Union[int, str, bytes] cast_narrower(int, x) + 1 ``` This obviously isn't safe. But it's less susceptible to obviously evil things if you check that int is a subtype of Union[int, str, bytes]. This corresponds to the discussion of "strict narrowing" in the draft TypeGuard PEP. Relevant mypy issue link: https://github.com/python/mypy/issues/5687#issuecomment-425654603

On Wed, Dec 23, 2020 at 5:00 AM Shantanu Jain wrote:

But to me, basically any cast() has a smell, and I wouldn't want to

...

legitimize it by recommending that idiom.

I think I don't feel this as much as you do... a type guard is just a verbose conditional cast. Using cast has the advantage of not requiring runtime changes / requiring fairly minimal changes to type checkers, but if it's a concern maybe we could give it its own (hopefully less smelly) bikeshed, say assert_static_is_subtype.

Okay, let's just agree to disagree. Not everybody has to feel as bad about casts as I do, and not everybody has to feel the need to abstract type guards as conditional functions as much as I do.

If you don't want to define a two-line function, is that because you're

...

writing short scripts? Or is it because you're worried about the runtime cost of the function call? (Then you should be worried about the cost of cast() also. :-)

My reluctance to define a function comes from a couple things:

- For examples given in the PEP draft, like `len(val) == 2` or `all(isinstance(x, str) for x in val)`, I'd always write those inline today. Having to move around my code to define a two line but clunky looking function feels less ergonomic than adding a cast or an annotation. Especially if the function just gets used once or twice. - Runtime type checking has a lot of caveats. For my example of `not val or isinstance(val[0], str)`, picking a name like is_str_list_based_on_first_element to describe those caveats is clunkier than the actual code that does the check. - I sometimes write short scripts!

For a short script I probably would inline the check as well. Then again for short scripts I rarely bother calling mypy. (Though now that I'm using VS Code that excuse is weaker. :-)

None of that is deal breaking, but since I didn't love TypeGuard, I thought it was worth bringing up cast as a lightweight construct that could be used to make code that is currently inexpressible in our type system expressible. In my mind, if cast supported redefinition like this today, the bar for the acceptance of a type guard like construct would be higher.

Hm, I don't see this as a feature of cast -- I see it as a feature of redefinition (which currently isn't well standardized). I would be really disappointed if redefinition was allowed when the RHS is a cast() but not when the RHS is some other expression. Somehow to me that breaks the abstraction that cast() is just an expression, and that would weigh heavily on me. It's true that I accept some other things that have the form of an expression but are treated as special syntax, e.g. TypeVar() and NewType(). But for those that's the *only* way it can be used. A cast() is legal in any position where an expression is allowed, and that makes it hard for me to swallow that when combining two separate things that each have well-defined meanings (assignments and casts) the semantics of their combination cannot be derived from their combined semantics (in particular, you are proposing that `var = cast(T, ...)` be allowed in some cases where `var = <some expression that has type T>` would not be allowed. I would much rather put an effort in better standardization of redefinition semantics (which would make your example work without special-casing cast() in that position).

... discussion of safe cast ...

...

I bungled this minor suggestion — my terminology and links were a mess :-) I meant to refer to https://github.com/erictraut/peps/blob/master/pep-9999.rst#enforcing-strict-... and some previous discussion on this thread. That is, should it be possible to further check that we only narrow to a subtype of the original type, e.g. `Tuple[int, ...]` to `Tuple[int, int]` as compared to `List[object]` to `List[str]` (or `int` to `str`). If this were desirable, it could be accomplished by use of a different cast function, perhaps called cast_narrower or downcast.

I thought the discussion ended with agreement that we should not enforce strict narrowing for type guards (because of invariant types). That doesn't mean there aren't use cases for downcasts, but they are different than the use cases for type guards.

Hm. In my experience typeguard functions often naturally occur (though

...

perhaps not for the examples given in the PEP) and they often abstract away checks that should not be inlined. So perhaps the use cases are just different?

Yeah, the use cases are definitely complementary / do not preclude one another. Like you say, I think there's an unmet need for `assert isinstance(x, TypeForm)` and cast redefinition could provide a solution for that.

Let's start a separate thread for that so it doesn't distract us from the typeguard discussion further. (Or we can keep it in the tracker until such time as you feel it's ready to be turned into a PEP.)

That said, I'd like to know more about what use cases you envision for TypeGuard beyond code that currently tends to be inlined / maybe the PEP could benefit from a discussion of those. The TypedDict example in the current draft is well taken, though :-) (I'll note that you could still use cast redefinition in the appropriate scope, but it might feel annoying).

The main use case really is two-liners that you write over and over, not once or twice. It's annoying that you are *forced* to inline those just because otherwise the type checker won't narrow the type in the "if true" branch. I could think of cases where I'd even abstract a single isinstance() call into a function just because the type I'm checking for has a name that's long or confusing. But it's definitely something that becomes more important for larger code bases. -- --Guido van Rossum (python.org/~guido) *Pronouns: he/him **(why is my pronoun here?)* http://feministing.com/2015/02/03/how-using-they-as-a-singular-pronoun-can-c...

Just catching up on the thread.

...

Sebastian Kreft seems to have executed a search that came to the same conclusion as Anders (typeguards are one-argument functions), but he is

Guido said: pushing for a way to define a typeguard as a method that guards self, e.g. if query.empty(): .... I'm not sure how to address this without adopting David's idea except to propose that if self is the only argument the typeguard applies to self.

Applying a user-defined type guard to `self` strikes me as a very unusual use case — one that probably involves anti-patterns such as a base class having knowledge of its derived classes. I don't think we should accommodate this with any special handling. The proposed mechanism already handles this if you pass self as an explicit argument, as in:

The real world example I provided does not require knowledge of any subclasses, it just asserts the object conforms to a specialized protocol which better describes the types for the current state of the class. Note

On Wed, Dec 23, 2020 at 5:48 PM Eric Traut wrote: that the example is based on some real Typescript code. For the provided example one could instead model the query result as instances of one of the two classes `ExistingResult` or `MissingResult`. However, that violates another principle of not returning Unions. If you could provide an alternative to better model the example code that would be great. I think this whole point could be postponed if we remove the ability to specify typeguards with multiple arguments.

```python class Foo: def is_bar(self, x: "Foo") -> TypeGuard[Bar]: return isinstance(x, Bar)

def func(self): if self.is_bar(self): reveal_type(self) # Bar ```

I agree strongly with Guido that `cast` should not be overloaded for all the reasons provided. From my perspective, `cast` should be used only rarely and as a last resort. It's effectively saying "disable all type checking safety here because I know better". It's dangerous and leads to fragile code. I wouldn't want to further legitimize its use. Also, I agree that it would be inappropriate to implicitly redefine the type of the expression passed as the first argument to `cast`. That would be inconsistent, unintuitive, and result in backward compatibility problems for existing code.

David said:

...

it's possible to trivially implement safe_cast()...

Yes, but I'll point out that you would need to reverse the parameters because the expression that is being narrowed (in your example, the `value` parameter) must be the first param of the type guard function.

...

Conclusion: There are some loose ends, but I will sponsor this PEP and

Guido said: likely approve it after only minor updates. Eric, please use PEP number 647.

OK, thanks. I'll work on incorporating the feedback and post a new draft soon. _______________________________________________ Typing-sig mailing list -- typing-sig@python.org To unsubscribe send an email to typing-sig-leave@python.org https://mail.python.org/mailman3/lists/typing-sig.python.org/ Member address: skreft@gmail.com

-- Sebastian Kreft

Sebastian said:

If you could provide an alternative to better model the example code that would be great.

Here are three techniques that would work with the current proposal. ```python class QueryResult(Generic[T]): # Technique 1: Accept an explicit `value` parameter expecting the caller # to pass in an instance; they can pass `self` if that's the instance they # want to test. def is_empty1(self, value: "QueryResult[S]") -> TypeGuard[MissingQueryResult]: return value._data is None # Technique 2: Use a static method, expecting the caller to pass in "self" (or # whatever other instance they want to test) @staticmethod def is_empty2(value: "QueryResult[S]") -> TypeGuard[MissingQueryResult]: return value._data is None # Technique 3: Define it as a separate utility method, not part of the class. def is_empty_query_result(value: "QueryResult[S]") -> TypeGuard[MissingQueryResult]: return value.empty() ``` Do you agree that we want to support type guards that are instance or class methods? If so, then these functions will need to accept one explicit `value` parameter in addition to the implicit "self" or "cls" parameter. I think there's a case to be made for disallowing type guards as instance or class methods. That would mean that all type guard functions would need to be either a static method or a (non-method) function. Alternatively, we could allow type guards as instance/class methods but assume that the first implicit parameter (`self` or `cls`) is the value being tested. I don't think that multiple arguments is the problem here. I've come across a couple of cases in real code where I wouldn't have been able to use the TypeGuard mechanism if it was limited to just one argument, so I'm reluctant to place that limitation on it.

On 12/23/20 12:48 PM, Eric Traut wrote:

David said:

...

it's possible to trivially implement safe_cast()...

Yes, but I'll point out that you would need to reverse the parameters because the expression that is being narrowed (in your example, the `value` parameter) must be the first param of the type guard function.

(The example of safe_cast() I provided doesn't actually use TypeGuard so I'm assuming you meant cast(), whose definition is already fixed by the existing API to have a typing form as the first parameter.) Ah yes. It would be necessary to use explicit syntax in TypeGuard to declare which (non-first) parameter was involved. Here's a corrected definition of cast() that uses TypeGuard: ``` def cast(form: TypeForm[T], value: object) -> TypeGuard[value=T]: return value ``` On 12/23/20 4:48 PM, Sebastian Kreft wrote:

[...] Alternatively, we could allow type guards as instance/class methods but assume that the first implicit parameter (`self` or `cls`) is the value being tested.

I like your last proposal, having `self` or `cls` the value being tested.

This also makes sense to me. Although it does seem unusual to have a TypeGuard instance method that redefines self/cls. Would it be narrowing the type of self/cls? To what end? I'd like to think of a realistic example here... On 12/23/20 4:48 PM, Sebastian Kreft wrote:

I don't think that multiple arguments is the problem here. I've come across a couple of cases in real code where I wouldn't have been able to use the TypeGuard mechanism if it was limited to just one argument, so I'm reluctant to place that limitation on it.

Note that it'd be just a deferral until we get more insights on how typeguards are used in Python. Then we could extend TypeGuard to accept an optional second argument specifying which argument is the one being guarded, something like TypeGuard[T, 'second'].

I'll still advocate for a future syntax more like `TypeGuard[second=T]` if the ability to explicitly label the applicable parameter appears later. :) To stress an earlier point, I do *not* think it's necessary to add that kind of parameter-labeling syntax in the initial PEP, especially given that for most uses of TypeGuard (ignoring combinations with TypeForm) it's always the *first* parameter of the function that's being narrowed, which doesn't require the labeling syntax. (If keyword arguments inside [] were already available, I'd have a different opinion.) Best, -- David Foster | Seattle, WA, USA Contributor to TypedDict support for mypy

On 12/26/20 9:15 PM, Guido van Rossum wrote:> ```

def cast(form: TypeForm[T], value: object) -> TypeGuard[value=T]: return value ```

Um, that's not a valid type guard according to Eric's PEP. The type guard function must return a bool. Or is this part of an alternate proposal where that's not the case? Maybe you could add some more

context? Doh. Right. Somehow I got the concept of a TypeGuard (a conditional returnable-type-assertion) and an "always-true returnable-type-assertion" mixed up in my head, and managed to omit the actual return type! I'm having trouble thinking of common cases for an "always-true returnable-type-assertion" that would be useful, so I think I'll drop the subject for the time being. If I *were* to give the alternate concept a spelling, I'd hazard something like: ``` def cast(form: TypeForm[T], value: V) -> V, TypeIs[value=T]: if TYPE_CHECKING: assert isinstance(value, form) # not actually valid Python for arbitrary typing forms return value ``` (Notice the regular return type for when the function returns normally, followed by some number of type-assertions, all in a comma-separated [tuple] expression.) On 12/26/20 9:15 PM, Guido van Rossum wrote:

I'll still advocate for a future syntax more like

`TypeGuard[second=T]`

if the ability to explicitly label the applicable parameter appears later. :)

To stress an earlier point, I do *not* think it's necessary to

add that

kind of parameter-labeling syntax in the initial PEP, especially

given

that for most uses of TypeGuard (ignoring combinations with TypeForm) it's always the *first* parameter of the function that's being narrowed, which doesn't require the labeling syntax. (If keyword arguments

inside

[] were already available, I'd have a different opinion.)

I agree that with PEP 637 syntax this would look nicer. However it wouldn't work in Python versions before 3.10, so maybe we could offer Sebastian's suggestion as a backwards compatible syntax option (like for variadics we're leaning towards `*Ts` in 3.10 with `Expand[Ts]` for earlier versions).

Makes sense. Best, -- David Foster | Seattle, WA, USA Contributor to TypedDict support for mypy

Thanks! I think it's ready for the next stage, actually submitting a PR for the peps repo. Once that passes tests I will merge it, at which point we should update the Post-history header. (I have a few nits that are best handled in the PR stage.) May 2021 be an improvement over 2020, --Guido On Sun, Dec 27, 2020 at 10:02 AM Eric Traut wrote:

I hope everyone's having an enjoyable, relaxing, and healthy holiday season!

I've updated the [draft PEP]( https://github.com/erictraut/peps/blob/master/pep-0647.rst) to incorporate the feedback.

* I updated the PEP number and added Guido as the sponsor. * Sebastian was interested in more examples of a multi-parameter type guard functions, so I added a second example (`is_set_of`, which also demonstrates that type guard functions can be generic). * I added more details in the "rejected ideas" section about a decorator-based syntax that was considered but abandoned. * I added an extensive justification in the "rejected ideas" section for why I don't think we should enforce any strict narrowing requirements, including the looser form that Guido proposed. Guido, if you disagree with my reasoning here, let's discuss further. * I added text in the "rejected ideas" section explaining why we decided not to support the narrowing of arbitrary parameters and always assume the first parameter is the value being narrowed. I mention in this section that we could extend the PEP in the future if this becomes important. * I added text in the "rejected ideas" section explaining why we are not providing any special mechanism for narrowing the implied "self" or "cls" parameters in instance/class methods.

-Eric -- Eric Traut Contributor to Pyright and Pylance Microsoft Corp. _______________________________________________ Typing-sig mailing list -- typing-sig@python.org To unsubscribe send an email to typing-sig-leave@python.org https://mail.python.org/mailman3/lists/typing-sig.python.org/ Member address: guido@python.org

-- --Guido van Rossum (python.org/~guido) *Pronouns: he/him **(why is my pronoun here?)* http://feministing.com/2015/02/03/how-using-they-as-a-singular-pronoun-can-c...

The expression passed as the first argument should always take on the type indicated by the TypeGuard within the guarded block. In your example, `x` should take on the type `float` within the `if` statement. The name of the type guard function in your example would more accurately be `is_nonzero_float` because that's what you're testing for. The expression `isinstance(x, float)` will always evaluate to false if you pass it an `int`. Perhaps you intended for the type guard function to test for both `float` and `int`? ```python def is_nonzero(x: object) -> TypeGuard[float]: return isinstance(x, (float, int)) and x != 0 ```

You can definitely use type variables within a type guard function. The PEP already contains a couple of examples of that. Here's how I'd recommend modifying your sample. Note that there's no need for a union type or an `isinstance` check because of the use of the bound TypeVar. ```python T = TypeVar("T", bound=float) def is_nonzero(x: T) -> TypeGuard[T]: return x != 0 a = 3 if is_nonzero(a): reveal_type(a) # int b = 3.14 if is_nonzero(b): reveal_type(b) # float ``` This example is a bit contrived in that "non-zeroness" isn't captured by the type indicated by the TypeGuard, so there's nothing of value a static type checker can do with the information provided by this type guard function. I don't think we should apply the conditional rule you proposed. The rule would have to be more complex than what you stated to handle unions and constrained TypeVars (which are similar to unions). The rule adds complexity in terms of the implementation but more importantly in the mental model for the user. It would introduce an inconsistency while providing little or no additional value. If you think it's advisable to add a new section to the PEP, I can do so.

On 12/27/20 10:02 AM, Eric Traut wrote:

I've updated the [draft PEP](https://github.com/erictraut/peps/blob/master/pep-0647.rst) to incorporate the feedback.

Looks good. A few comments below: § "Narrowing of Arbitrary Parameters":

TypeScript's formulation of user-defined type guards allows for any input parameter to be used as the value tested for narrowing. The TypeScript language authors could not recall any real-world examples in TypeScript where the parameter being tested was not the first parameter. For this reason, it was decided unnecessary to burden the Python implementation of user-defined type guards with additional complexity to support a contrived use case. If such use cases are identified in the future, there are ways the TypeGuard mechanism could be extended. This could involve the use of keyword indexing, as proposed in PEP 637.

To make "This could involve the use of keyword indexing" more concrete, suggest actually putting in an example, such as: ``` def is_str_list(failure_message: str, val: List[object]) -> TypeGuard[val=List[str]]: non_strs = [x for x in val if not isinstance(x, str)] if len(non_strs) != 0: print(failure_message % non_strs) return len(non_strs) == 0 parsed_list = ... assert is_str_list('Non-strings: %s', parsed_list) ``` Admittedly this particular is contrived, but is still illustrative.

Discussions-To: Python-Dev

Nit: "Python-Dev" -> "Typing-Sig" § "Conditionally Applying TypeGuard Type"

It was suggested that the expression passed as the first argument to a type guard function should retain 👉its👈 existing type if the type of the expression was

Nit: See typo above: 👉its👈 Best, -- David Foster | Seattle, WA, USA Contributor to TypedDict support for mypy

There was some discussion in this thread of type guards vs conditional casts. Given that Shantanu's point that a type guard is similar to an if-clause followed by cast makes sense at least to me, maybe it would make sense to add a section under "Rejected Ideas" for basically "add no features; use conditional casts"? (or the slightly stronger suggestion to have checkers allow name redefinition in casts.) That way the PEP could explain the rationale for why TypeGuard approach is better than what can be achieved by conditional casts. As far as I can tell, some of the features which were discussed like enforcing only strict narrowing, and narrowing both the positive and negative cases, have been dropped.

Also: - If a type guard definition has insufficient arguments, should that be an error? E.g. ``` def booh() -> TypeGuard[int]: return False ``` On Sun, Jan 10, 2021 at 9:26 PM Guido van Rossum wrote:

I'm trying to implement type guards in mypy, and we have run into a few edge cases (several due to Jukka's sharp observations).

- Keyword arguments. The target of isinstance() *must* be passed as a positional argument. Can the PEP require this for type guards too? Given that the target is always the first argument, I think that passing it as a keyword argument (or through `*args` or `**kwargs`) would just confuse the reader. (If the intention of the PEP is to explicitly allow this, the wording could be clearer, or at least there should be an example.)

- If a class defines a method that is a type guard, and a subclass overrides that method, should the subclass also declare its method as a type guard, since it may be used as a type guard? (I notice that pyright allows the subclass to declare the method override with `-> bool`. Bug?)

-- --Guido van Rossum (python.org/~guido) *Pronouns: he/him **(why is my pronoun here?)* http://feministing.com/2015/02/03/how-using-they-as-a-singular-pronoun-can-c...

-- --Guido van Rossum (python.org/~guido) *Pronouns: he/him **(why is my pronoun here?)* http://feministing.com/2015/02/03/how-using-they-as-a-singular-pronoun-can-c...

On Sun, Jan 10, 2021 at 9:39 PM Eric Traut wrote:

My intent was that type guards require the first argument to be passed by position. I thought that was clear in the PEP since it says "the first explicit argument", but we could add the word "positional" to make it even clearer.

Please use "positional" -- it's a clear technical term. "Explicit" could be interpreted as intending to distinguish between default argument values and values passed as arguments. After all, there's nothing implicit about f(x=1) compared to f(). ;-)

If a subclass overrides a type guard and doesn't provide the `TypeGuard` return type, a type checker could choose to flag it as an error or warning, just as it could choose to flag any incompatible override. I don't consider that related to the type guard PEP.

A PEP that introduces a new type system feature should define how that feature fits into the existing framework for subtype checking. This PEP should answer the question of whether Callable[..., bool] is a subtype of Callable[..., TypeGuard[X]]. If the answer is that it depends, that's also good to mention. But I'd like an answer, because it also affects a question about overloading which I didn't bring up yet because there was a bug in pyright here. But since you fixed it, now I can pose the question: If two overloads only differ in one parameter and that parameter's type is Callable[..., TypeGuard[X]] in one case and Callable[..., bool] in the other overload, can the return type vary between these cases? My example is ``` @overload def filter(f: Callable[[T], TypeGuard[R]], it: Iterable[T]) -> Iterator[R]: ... @overload def filter(f: Callable[[T], bool], it: Iterable[T]) -> Iterator[T]: ... def filter(f, it): <impl> ``` This intends to produce a different return type based on whether the callable is a type guard or not. E.g. given a list of objects and an is_int() *type guard* it returns Iterator[int], but with a non-type-guard function it's Iterator[object]. I would think that the subtype question is relevant to the question of whether this should work.

If a type guard definition has insufficient _parameters_, it can't be invoked with "a first explicit argument", so it won't be usable as a type guard function. I suppose that a type checker could choose to warn the user of this at the point where the function is declared. I don't plan to add any such warning in Pyright.

This I am fine with, it will just not work as a type guard. -- --Guido van Rossum (python.org/~guido) *Pronouns: he/him **(why is my pronoun here?)* http://feministing.com/2015/02/03/how-using-they-as-a-singular-pronoun-can-c...

This PEP doesn't dictate how or where a type checker applies type guards. It simply provides a way to specify a user-defined type guard. Type checkers should apply user-defined type guards in all situations where they already apply built-in type guards. For all type checkers I'm aware of, that would apply to assert statements, since an assert is simply shorthand for `if not <conditional check>: raise AssertionError()`.