typing.Maybe type annotation
Hi all, I recently ran into a problem trying to adapt mypy into an older codebase, and it got me thinking about a potential addition to the typing library. This idea comes in two parts (that can even be taken individually): the first is to allow the False singleton literal to be used in type annotations, much like how None is now allowed. False will in fact mean "any object for which the `not` operator will return True". The second part is to add a generic shorthand to the typing module: `typing.Maybe[T]`, that will evaluate to `Union[T, False]`. This will allow us to annotate the results of `and` and `or` operators more effectively. ``` raw_value : str = input() # suppose we want to convert raw_value to an int, but leave it as an arbitrary falsish value if raw_value is an empty string, we can now do this in the following ways: # annotate value as a union (type hint is misleading) value_union = raw_value and int(raw_value) # value: Union[str, int] inferred # annotate the value as optional on first assignment within an if clause (verbose, also type hint is confusing when assigning to a literal) # variants of this method include putting the first assignment in its own clause, or putting an empty variable declaration before any assignment, # but these are all just adding verbosity to what should be a very simple operation value_optional_clause: Optional[int] = None if raw_value: value_optional_clause = int(raw_value) # annotate as an optional, and use the ternary operator (unpythonic and overly verbose) value_ternary = int(raw_value) if raw_value else None # value: Optional[int] inferred # with the new typing.Literal class, we can explain the type of value precisely (it will require a cast however, and be very verbose) value_literal_union: Union[int, Literal['']] = cast(Union[int, Literal['']], raw_value and int(raw_value)) ``` Adding `typing.Maybe`/`False` will open up new possibilities and easier type inference when using the logical operators, at least the following: ``` # T0, T1 and T2 are typevars x : Maybe[T0] y : T1 z: T2 v = x or y # z: Union[T0, T1] inferred v = y and z # z: Maybe[T2] inferred if x: # x: T0 inferred ... if not y: # y: False inferred ... f: False if f: # unreachable code ... ``` Any thoughts?
On Tue, Mar 9, 2021 at 9:15 PM Ben Avrahami <avrahami.ben@gmail.com> wrote:
I'm hoping this is nothing more than a weak example, and it isn't something you'd actually do in your code, because IMO this has bigger problems than the type annotation - namely, the inconsistency of actual type. I'd much rather write this as: value = int(raw_value or 0) and then it's consistently an integer. It'll still be usable the exact same ways, but it's much easier to work with. (Case in point, albeit not from Python: An API gave me back the JavaScript "undefined" value in a context where I was expecting false or 0, and it caused problems when I used it subsequently with something that treated undefined as "toggle" instead of as true or false. Was very annoying.) ChrisA
I was astonished that `typing.Maybe[T]` doesn't already exist, but if it did exist, surely it would have to be `Union[T, None]`. Ah wait, that would be spelled "Optional". As for the pseudo-type `False` meaning "any falsey value", I don't think that tracking the *value* of variables is something that the current generation of type checkers are capable of. Consider: text = input() flag = "x" in text We can infer that text is a string, but not whether it is a truthy string or a falsey string; we can infer that flag is a bool, but not whether it is True or False. Adding variable annotations is no help, because the annotations have to be added when we edit the code, but whether or not the values are truthy or falsey isn't generally known until the code is run. And the type checker runs at the intermediate time where the code is compiled: 1. Edit time. 2. Compile time (includes linters, the parser, etc). 3. Run time. We can only annotate information known at stage 1; the type checker can only check things known at stage 2; but the value of most things are only known at stage 3. -- Steve
You can also use typing.cast for the handful of cases where your particular type checker (e.g. MyPy) fails to refine the type properly in a case like the following: def lookup_usernames(db, user_ids: Optional[List[int]] = None) -> List[str]: if user_ids is None: return [] user_ids = typing.cast(List[int], user_ids) db.executemany('select name from users where id = ?', [(user_id,) for user_id in user_ids]) return db.fetchall()
On Tue, Mar 9, 2021 at 9:15 PM Ben Avrahami <avrahami.ben@gmail.com> wrote:
I'm hoping this is nothing more than a weak example, and it isn't something you'd actually do in your code, because IMO this has bigger problems than the type annotation - namely, the inconsistency of actual type. I'd much rather write this as: value = int(raw_value or 0) and then it's consistently an integer. It'll still be usable the exact same ways, but it's much easier to work with. (Case in point, albeit not from Python: An API gave me back the JavaScript "undefined" value in a context where I was expecting false or 0, and it caused problems when I used it subsequently with something that treated undefined as "toggle" instead of as true or false. Was very annoying.) ChrisA
I was astonished that `typing.Maybe[T]` doesn't already exist, but if it did exist, surely it would have to be `Union[T, None]`. Ah wait, that would be spelled "Optional". As for the pseudo-type `False` meaning "any falsey value", I don't think that tracking the *value* of variables is something that the current generation of type checkers are capable of. Consider: text = input() flag = "x" in text We can infer that text is a string, but not whether it is a truthy string or a falsey string; we can infer that flag is a bool, but not whether it is True or False. Adding variable annotations is no help, because the annotations have to be added when we edit the code, but whether or not the values are truthy or falsey isn't generally known until the code is run. And the type checker runs at the intermediate time where the code is compiled: 1. Edit time. 2. Compile time (includes linters, the parser, etc). 3. Run time. We can only annotate information known at stage 1; the type checker can only check things known at stage 2; but the value of most things are only known at stage 3. -- Steve
You can also use typing.cast for the handful of cases where your particular type checker (e.g. MyPy) fails to refine the type properly in a case like the following: def lookup_usernames(db, user_ids: Optional[List[int]] = None) -> List[str]: if user_ids is None: return [] user_ids = typing.cast(List[int], user_ids) db.executemany('select name from users where id = ?', [(user_id,) for user_id in user_ids]) return db.fetchall()
participants (4)
-
Ben Avrahami -
Chris Angelico -
Greg Werbin -
Steven D'Aprano