
I'm a +1 on using Annotated in this manner. Guido mentioned that it was intended for only third-parties though. I'd like to know more about why this isn't a good pattern for use by Python libraries. On Sun, 2021-02-14 at 16:29 +0100, Adrian Freund wrote:
Here's another suggestion:
PEP 593 introduced the `Annotated` type annotation. This could be used to annotate a TypeGuard like this:
`def is_str_list(val: List[object]) -> Annotated[bool, TypeGuard(List[str])]`
Note that I used ( ) instead of [ ] for the TypeGuard, as it is no longer a type.
This should fulfill all four requirements, but is a lot more verbose and therefore also longer. It would also be extensible for other annotations.
For the most extensible approach both `-> TypeGuard(...)` and `-> Annotated[bool, TypeGuard(...)]` could be allowed, which would open the path for future non-type-annotations, which could be used regardless of whether the code is type-annotated.
-- Adrian
On Sat, Feb 13, 2021 at 07:48:10PM -0000, Eric Traut wrote:
I think it's a reasonable criticism that it's not obvious that a function annotated with a return type of `TypeGuard[x]` should return a bool.
[...]
As Guido said, it's something that a developer can easily look up if they are confused about what it means.
Yes, developers can use Bing and Google :-)
But it's not the fact that people have to look it up. It's the fact
they need to know that this return annotation is not what it seems, but a special magic value that needs to be looked up.
That's my objection: we're overloading the return annotation to be something other than the return annotation, but only for this one special value. (So far.) If you don't already know that it is special, you won't know that you need to look it up to learn that its special.
I'm open to alternative formulations that meet the following requirements:
1. It must be possible to express the type guard within the function signature. In other words, the implementation should not need to be present. This is important for compatibility with type stubs and to guarantee consistent behaviors between type checkers.
When you say "implementation", do you mean the body of the function?
Why is this a hard requirement? Stub files can contain function bodies, usually `...` by convention, but alternatives are often useful, such as docstrings, `raise NotImplementedError()` etc.
https://mypy.readthedocs.io/en/stable/stubs.html
I don't think that the need to support stub files implies that the type guard must be in the function signature. Have I missed something?
2. It must be possible to annotate the input parameter types _and_ the resulting (narrowed) type. It's not sufficient to annotate just one or the other.
Naturally :-)
That's the whole point of a type guard, I agree that this is a
hard requirement.
3. It must be possible for a type checker to determine when narrowing can be applied and when it cannot. This implies the need for a bool response.
Do you mean a bool return type? Sorry Eric, sometimes the terminology you use is not familiar to me and I have to guess what you mean.
4. It should not require changes to the grammar because that would prevent this from being adopted in most code bases for many years.
Fair enough.
Mark, none of your suggestions meet these requirements.
Mark's suggestion to use a variable annotation in the body meets requirements 2, 3, and 4. As I state above, I don't think that requirement 1 needs to be a genuinely hard requirement: stub files can include function bodies.
To be technically precise, stub functions **must** include function bodies. It's just that by convention we use typically use `...` as
body.
Gregory, one of your suggestions meets these requirements:
```python def is_str_list(val: Constrains[List[object]:List[str]) -> bool: ... ```
That still misleadingly tells the reader (or naive code analysis software) that parameter val is of type
Contrains[List[object]:List[str]]
whatever that object is, rather than what it *actually* is, namely `List[object]`. I dislike code that misleads the reader.
As for choosing the name of the annotation
[...]
`TypeGuard` is the term that is used in other languages to describe this notion, so it seems reasonable to me to adopt this term
Okay, this reasoning makes sense to me. Whether spelled as a decorator or an annotation, using TypeGuard is reasonable.
Steven, you said you'd like to explore a decorator-based
Let's explore that. Here's what that it look like if we were to meet all of the above requirements.
```python @type_guard(List[str]) def is_str_list(val: List[object]) -> bool: ... ```
Okay.
I note that this could be easily extended to support narrowing in
negative case as well:
@type_guard(List[str], List[float]) def is_str_list(val: List[Union[str, float]]) -> bool: ...
The problem here, as I mention in the "rejected ideas" section of
PEP, is that even with postponed type evaluations (as described in PEP 563), the interpreter cannot postpone the evaluation of an expression if it's used as the argument to a decorator. That's because it's not being used as a type annotation in this context.
Agreed.
So while Mark is correct to point out that there has been a mechanism available for forward references since PEP 484,
That was actually me who pointed out the quoting mechanism for forward references. (Unless Mark also did so.)
we've been trying to eliminate the use of quoted type expressions in favor of postponed evaluation. This would add a new case that can't be handled through postponed evaluation. Perhaps you still don't see that as a strong enough justification for rejecting the decorator-based formulation. I'm not entirely opposed to using a decorator here, but I think on balance that the `TypeGuard[x]` formulation is better. Once again, that's a subjective opinion.
I understand the desire to minimize the use of quoted forward references. But the cost to save two quote characters seems high: changing an obvious and straight-forward return annotation to an actively misleadingly special case. (Also, see below for the `Callable` case.)
I'm not convinced that forward references will be common. Did I miss something, or are there no examples in the PEP that require a forward-reference?
# Spam is not defined yet, so a forward reference is needed.
def is_list_of_spam(values:List[object]) -> TypeGuard[List[Spam]]: return all(isinstance(x, Spam) for x in values)
# versus decorator
@type_guard('List[Spam]') def is_list_of_spam(values:List[object]) -> bool: return all(isinstance(x, Spam) for x in values)
Of course, neither of these examples will actually work, because Spam doesn't exist so you can't refer to it in the body. I don't get the sense that this will require forward-references very often. At least not often enough to justify obfuscating the return type.
This obfuscation appears to have a critical consequence too. Please correct me if I am wrong, but quoting your PEP:
""" In all other respects, TypeGuard is a distinct type from bool. It is not a subtype of bool. Therefore, Callable[..., TypeGuard[int]] is not assignable to Callable[..., bool]. """
If I am reading this correctly that implies that if I define these functions:
``` def is_str_list(val: List[object]) -> TypeGuard[List[str]]: return all(isinstance(x, str) for x in val)
def my_filter(func:Callable[object, bool], values:List[object]) -> List[object]: return [x for x in values if func(x)] ```
the type checker would be expected to flag this as invalid:
my_filter(is_str_list, ['a', 'b'])
If I have not misunderstood, surely this is a critical flaw with
PEP?
Eric, for what it's worth, I think that this will be an excellent feature for type checkers, thank you for writing the PEP. It's just
syntax gives me the willies:
- special case handling of TypeGuard in a way that obfuscates the actual return type;
- actively misleads the reader, and any naive code analysis tools
don't know about type guards;
- prevents user type guards from being seen as `Callable[..., bool]` even though they actually are.
And the justification for eliminating decorators seems to be weak to me.
However, I will give you one possible point in its favour: runtime introspection of the annotations by typeguard-aware tools. If an introspection tool is aware of the special meaning of `TypeGuard` in the return annotation, then it is easy to introspect that value at runtime:
# untested... T = func.__annotations__['return'] if T.startswith("TypeGuard"): print("return type is bool") print("narrows to type ...") # TODO: extract from T else: print(f"return type is {T}")
With a decorator, we would need some other solution for runtime introspection of the type guard. That's easy enough to solve, the obvious solution is to just add the information to the function as an attribute. But it is one additional complication.
Still, I think this is a minor issue, especially compared to the critical issue of `Callable[..., bool]`.
I think that the runtime introspection issue will probably rule out Mark's "variable annotation in the body" idea. So considering all
On February 14, 2021 2:20:14 PM GMT+01:00, Steven D'Aprano <steve@pearwood.info> wrote: that truly the formulation. the the the the that the
factors as I see them, including familiarity to TypeScript devs:
* Decorator: +1
* Mark's variable annotation, if runtime introspection is neglected: +0.1
* Gregory's idea to annotate the parameter itself: -0.1
* The PEP's return value annotation: -0.2
* Mark's variable annotation, if runtime introspection is required: -1
_______________________________________________ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-leave@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-d ev@python.org/message/KFKTZ6L5MI6S7KZY4W6PGZZWRR2PQTQF/ Code of Conduct: http://python.org/psf/codeofconduct/