Adding `typing.reveal_type`

I just opened https://bugs.python.org/issue46414 to propose adding `typing.reveal_type` to Python 3.11. Copying my message there: --- The `reveal_type()` primitive is injected by type checkers into the builtins. When the type checker sees a call, it prints the inferred type of the argument. This has been implemented across all type checkers, but adding an implementation to `typing` would help document the behavior and make it more discoverable for users. Also, it means code with `reveal_type()` calls can run without runtime errors, useful if you want to run your tests at the same time as you're debugging a typing issue. The runtime implementation can be very simple: def reveal_type(obj: _T, /) -> _T: print("Runtime type is", type(obj)) return obj reveal_type() is supported by all type checkers that I'm aware of (docs include https://google.github.io/pytype/faq.html#can-i-find-out-what-pytype-thinks-t... for pytype and https://mypy.readthedocs.io/en/stable/common_issues.html#reveal-type for mypy). One area of divergence is the return value. Pyright returns the inferred type of the expression as a string (and uses that in its test suite for testing type inference). Mypy returns the argument, which has the advantage that you can insert `reveal_type()` in the middle of an expression without having to put it on its own line. Also, the Pyright behavior cannot sensibly be implemented at runtime. Therefore, I suggest using Mypy's behavior for `typing.reveal_type`. ---- I'm hoping this will be simple and noncontroversial enough that we can add it quickly, but if there are concerns from any type checker authors, I'd love to hear them.

As a static type checking amateur, it's not clear to me what the aim of adding this to stdlib is. Is this to avoid the pitfalls of code containing calls to `reveal_type` when not running mypy or other static type checker? Is the intent for the static type checker to monkey-patch the typing library to provide something different when it's running? On Mon, 2022-01-17 at 07:14 -0800, Jelle Zijlstra wrote:

El lun, 17 ene 2022 a las 8:37, Paul Bryan (<pbryan@anode.ca>) escribió:
There's a couple of advantages: - Discoverability. I'd been thinking about this for a while, but my prompt for writing it up this morning was Petr Viktorin's query over on python-dev about a use of reveal_type() in PEP 673. Documenting reveal_type() in the typing module will make it easier for users to find out about it. Two type checkers (pyre and pyright) currently don't define reveal_type() in their documentation, but use it in examples anyway. - Standardization. Because reveal_type() has been spreading organically between type checkers, there are some differences in exact behavior. Standardizing that behavior will make code more portable across type checkers. - Type checker-independent use. If reveal_type() is in typing, we can use it in type checker-independent documentation, such as PEPs and https://typing.readthedocs.io/en/latest/. - Runtime use. Right now you'll get a NameError if you try to run code that uses reveal_type(). I don't think this will be a very common use case, but I could imagine wanting to run your test suite at the same time as you're using reveal_type() to debug typechecking.
Is the intent for the static type checker to monkey-patch the typing library to provide something different when it's running?
Static type checkers don't use the runtime definitions in typing.py, so there's no need to monkeypatch anything.

This proposal makes sense to me. I would recommend adding `reveal_locals` as well. It's not used as often as `reveal_type`, but it's kind of a "sister function". They're both described in PEP 484, and AFAIK, all type checkers support both. -Eric -- Eric Traut Contributor to Pyright & Pylance Microsoft

Still not making sense to me, and I find no mention of `reveal_type` or `reveal_locals` in PEP 484. The proposal is to add functions to Python's stdlib runtime to provide a common API for external static type checkers? I'm not against there being a common API per se, but I'm not seeing such a need rising to the level of adding it to stdlib. What am I missing? On Mon, 2022-01-17 at 17:21 +0000, Eric Traut wrote:

I stand corrected. I thought these were both part of PEP 484, but you're correct that they were added later by mypy. As Jelle said, they are both well known and used pretty extensively in code samples within the typing world. The benefit of having them implemented is to allow these samples to run in the interpreter. If there's good reason not to add these two functions to `typing`, a reasonable fallback is to add them to `typing_extensions`.

I know of at least one popular typing context wherein `reveal_type` isn't provided: PyCharm. I'm not sure exactly how that affects the argument for/against. On the one hand, if you're using PyCharm without another type checker, all the examples using reveal_type just don't make any sense unless you already know the PyCharm equivalent (lookup Quick Documentation for member). On the other hand, at first blush it seems a little odd to add something to the standard lib to make up for this situation. On the gripping hand, what else is going to fix this little problem with the communication about types? On Mon, Jan 17, 2022 at 11:35 AM Eric Traut <eric@traut.com> wrote:

This is the first I've heard of reveal_locals! Sounds like a useful thing to add, though, I would be in favour of documenting it on par with reveal_type. Pytype also supports an assert_type function, added based on user requests. It's a check you can leave in your code to ensure the type checker is inferring what you think it is. martin On Mon, Jan 17, 2022 at 9:21 AM Eric Traut <eric@traut.com> wrote:

We've been discussing `assert_type()` in https://github.com/python/typing/discussions/1030. I would be interested in standardizing that too. El mar, 18 ene 2022 a las 12:35, Martin DeMello via Typing-sig (< typing-sig@python.org>) escribió:

Am 17.01.22 um 17:38 schrieb Paul Bryan:
This is an idea I wanted to bring up as well, but never came around to. For me, it's mostly about convenience: Often when debugging, I add reveal_type() statements and run the type checker and the tests alternately. But currently this means I have to remember to comment out the reveal_type() statement whenever I run the tests (and I usually don't remember). This would help with this nuisance. - Sebastian

To me it seems weird, perhaps even wrong, to add these to stdlib, as they're expected to be implemented by external static type checking tools. If static type checkers want to share an API convention, wouldn't it be more appropriate to establish a shared namespace where they can implement these functions and keep them out of stdlib? On Mon, 2022-01-17 at 19:12 +0100, Sebastian Rittau wrote:

Hey :) Perhaps I've misunderstood you, but isn't the "typing" module exactly a shared namespace for external type checkers? As a consumer, it would be great to not have to worry about commenting out reveal_type or wrapping it in `if TYPE_CHECKING:`. On Mon, 17 Jan 2022, 18:51 Paul Bryan, <pbryan@anode.ca> wrote:

It's similar to `typing.cast`, which is part of stdlib. The `cast` call acts as a no-op at runtime but has special meaning to type checkers.

To add to this, you could potentially consider `typing.NewType` as precedent. Whilst it is an introspectable class at runtime, `NewType.__call__` is just also just a no-op and only gains meaning when a type checker is applied. (forgive the bad example) ``` Volume = NewType("Volume", int) type(N) # <class 'typing.NewType'> tv_volume = Volume(50) type(tv_volume) # <class `int`> ``` On Mon, Jan 17, 2022 at 7:19 PM Eric Traut <eric@traut.com> wrote:

That's an interesting case; I'm planning on including code in our library to handle `NewType` at runtime, to: a) enforce types of parameters and dataclass attributes, and b) use its underlying type for serialization and deserialization of values. On Mon, 2022-01-17 at 19:55 +0000, Luis Miguel Morera De La Cruz wrote:

If that's the stated purpose of the typing module, it's been lost on me. I've used it extensively for runtime type hint introspection. If there are existing examples of where static type checkers are expected/required to override (e.g. monkey patch) what's implemented in the typing module, I'd be quite interested in knowing what they are. I get the desire to avoid writing conditional code; I'm just not convinced that this should be baked into the standard library. Given the example by Jelle simply prints the type and returns the object, it doesn't appear to have any value unless overriden by static type checkers for use in debugging. One could choose another module name, not included in stdlib, implemented soley by static type checkers, to achive this goal. On Mon, 2022-01-17 at 18:59 +0000, Luis Miguel Morera De La Cruz wrote:

On Mon, Jan 17, 2022 at 10:12 AM Sebastian Rittau <srittau@rittau.biz> wrote:
Indeed -- I have used this workflow (alternating between running a static checker and running the code) and sometimes I've even done things like if not TYPE_CHECKING: reveal_type = print # Good enough :-) Still, I have a concern. Linters etc. tend to complain about unused imports. That means that you cannot leave the `from typing import reveal_type` in your code once you've removed the last reveal_type() call. But now when you want to use it, you'll have to add the import back in. So it would be much more convenient if it was an actual builtin. Regarding reveal_locals(), that's much less commonly used in my experience (it was added to mypy rather late) and I am hesitant to standardize it prematurely. (Alas, spotting usage "in the wild" of these functions is not easy, since their essence is that you have to remove them before running your code.) -- --Guido van Rossum (python.org/~guido) *Pronouns: he/him **(why is my pronoun here?)* <http://feministing.com/2015/02/03/how-using-they-as-a-singular-pronoun-can-c...>

El lun, 17 ene 2022 a las 19:10, Guido van Rossum (<guido@python.org>) escribió:
I see your point, but I'm not sure we'll be able to meet the bar for adding a new builtin. Also, I'm not proposing that type checkers remove the current behavior where reveal_type() is always an implicit builtin during type checking. So you'd only need to add the explicit import in two cases: - When you want to also run your code at the same time (e.g. to run your test suite) - When you want to communicate to readers where reveal_type() comes from (e.g. in a PEP example)

On Tue, Jan 18, 2022 at 12:25 PM Jelle Zijlstra <jelle.zijlstra@gmail.com> wrote:
Yeah, probably not.
I'd still be hesitant to add this to PEPs, even new PEPs, because once you know what reveal_type is, it rarely need an explanation (witness people here thinking that it was defined in PEP 484 -- in fact I searched that PEP myself :-). But the first bullet feels true, and just having the docs and the specification make it useful, so I am in favor of this. I think we should also add reveal_locals() and assert_type(), to promote, document and standardize them. -- --Guido van Rossum (python.org/~guido) *Pronouns: he/him **(why is my pronoun here?)* <http://feministing.com/2015/02/03/how-using-they-as-a-singular-pronoun-can-c...>

Would it be worthwhile also include filename, line and column numbers like type checkers do so you could easily track down where the call is from at runtime or should you just run a type checker to get them?

I realize I never responded to this suggestion. Sebastian had a similar idea in https://github.com/python/typing/issues/277#issuecomment-961061916: use the warnings module to emit a message. Personally I don't care much for the exact runtime behavior, but I'm wary of making it too complicated. I don't feel like tracking down where a call comes from is a very compelling use case: it's easy to just grep for, and your type checker should give you an error for any call to reveal_type() anyway. El mar, 18 ene 2022 a las 13:02, James H-B (<gobot1234yt@gmail.com>) escribió:

As a static type checking amateur, it's not clear to me what the aim of adding this to stdlib is. Is this to avoid the pitfalls of code containing calls to `reveal_type` when not running mypy or other static type checker? Is the intent for the static type checker to monkey-patch the typing library to provide something different when it's running? On Mon, 2022-01-17 at 07:14 -0800, Jelle Zijlstra wrote:

El lun, 17 ene 2022 a las 8:37, Paul Bryan (<pbryan@anode.ca>) escribió:
There's a couple of advantages: - Discoverability. I'd been thinking about this for a while, but my prompt for writing it up this morning was Petr Viktorin's query over on python-dev about a use of reveal_type() in PEP 673. Documenting reveal_type() in the typing module will make it easier for users to find out about it. Two type checkers (pyre and pyright) currently don't define reveal_type() in their documentation, but use it in examples anyway. - Standardization. Because reveal_type() has been spreading organically between type checkers, there are some differences in exact behavior. Standardizing that behavior will make code more portable across type checkers. - Type checker-independent use. If reveal_type() is in typing, we can use it in type checker-independent documentation, such as PEPs and https://typing.readthedocs.io/en/latest/. - Runtime use. Right now you'll get a NameError if you try to run code that uses reveal_type(). I don't think this will be a very common use case, but I could imagine wanting to run your test suite at the same time as you're using reveal_type() to debug typechecking.
Is the intent for the static type checker to monkey-patch the typing library to provide something different when it's running?
Static type checkers don't use the runtime definitions in typing.py, so there's no need to monkeypatch anything.

This proposal makes sense to me. I would recommend adding `reveal_locals` as well. It's not used as often as `reveal_type`, but it's kind of a "sister function". They're both described in PEP 484, and AFAIK, all type checkers support both. -Eric -- Eric Traut Contributor to Pyright & Pylance Microsoft

Still not making sense to me, and I find no mention of `reveal_type` or `reveal_locals` in PEP 484. The proposal is to add functions to Python's stdlib runtime to provide a common API for external static type checkers? I'm not against there being a common API per se, but I'm not seeing such a need rising to the level of adding it to stdlib. What am I missing? On Mon, 2022-01-17 at 17:21 +0000, Eric Traut wrote:

I stand corrected. I thought these were both part of PEP 484, but you're correct that they were added later by mypy. As Jelle said, they are both well known and used pretty extensively in code samples within the typing world. The benefit of having them implemented is to allow these samples to run in the interpreter. If there's good reason not to add these two functions to `typing`, a reasonable fallback is to add them to `typing_extensions`.

I know of at least one popular typing context wherein `reveal_type` isn't provided: PyCharm. I'm not sure exactly how that affects the argument for/against. On the one hand, if you're using PyCharm without another type checker, all the examples using reveal_type just don't make any sense unless you already know the PyCharm equivalent (lookup Quick Documentation for member). On the other hand, at first blush it seems a little odd to add something to the standard lib to make up for this situation. On the gripping hand, what else is going to fix this little problem with the communication about types? On Mon, Jan 17, 2022 at 11:35 AM Eric Traut <eric@traut.com> wrote:

This is the first I've heard of reveal_locals! Sounds like a useful thing to add, though, I would be in favour of documenting it on par with reveal_type. Pytype also supports an assert_type function, added based on user requests. It's a check you can leave in your code to ensure the type checker is inferring what you think it is. martin On Mon, Jan 17, 2022 at 9:21 AM Eric Traut <eric@traut.com> wrote:

We've been discussing `assert_type()` in https://github.com/python/typing/discussions/1030. I would be interested in standardizing that too. El mar, 18 ene 2022 a las 12:35, Martin DeMello via Typing-sig (< typing-sig@python.org>) escribió:

Am 17.01.22 um 17:38 schrieb Paul Bryan:
This is an idea I wanted to bring up as well, but never came around to. For me, it's mostly about convenience: Often when debugging, I add reveal_type() statements and run the type checker and the tests alternately. But currently this means I have to remember to comment out the reveal_type() statement whenever I run the tests (and I usually don't remember). This would help with this nuisance. - Sebastian

To me it seems weird, perhaps even wrong, to add these to stdlib, as they're expected to be implemented by external static type checking tools. If static type checkers want to share an API convention, wouldn't it be more appropriate to establish a shared namespace where they can implement these functions and keep them out of stdlib? On Mon, 2022-01-17 at 19:12 +0100, Sebastian Rittau wrote:

Hey :) Perhaps I've misunderstood you, but isn't the "typing" module exactly a shared namespace for external type checkers? As a consumer, it would be great to not have to worry about commenting out reveal_type or wrapping it in `if TYPE_CHECKING:`. On Mon, 17 Jan 2022, 18:51 Paul Bryan, <pbryan@anode.ca> wrote:

It's similar to `typing.cast`, which is part of stdlib. The `cast` call acts as a no-op at runtime but has special meaning to type checkers.

To add to this, you could potentially consider `typing.NewType` as precedent. Whilst it is an introspectable class at runtime, `NewType.__call__` is just also just a no-op and only gains meaning when a type checker is applied. (forgive the bad example) ``` Volume = NewType("Volume", int) type(N) # <class 'typing.NewType'> tv_volume = Volume(50) type(tv_volume) # <class `int`> ``` On Mon, Jan 17, 2022 at 7:19 PM Eric Traut <eric@traut.com> wrote:

That's an interesting case; I'm planning on including code in our library to handle `NewType` at runtime, to: a) enforce types of parameters and dataclass attributes, and b) use its underlying type for serialization and deserialization of values. On Mon, 2022-01-17 at 19:55 +0000, Luis Miguel Morera De La Cruz wrote:

If that's the stated purpose of the typing module, it's been lost on me. I've used it extensively for runtime type hint introspection. If there are existing examples of where static type checkers are expected/required to override (e.g. monkey patch) what's implemented in the typing module, I'd be quite interested in knowing what they are. I get the desire to avoid writing conditional code; I'm just not convinced that this should be baked into the standard library. Given the example by Jelle simply prints the type and returns the object, it doesn't appear to have any value unless overriden by static type checkers for use in debugging. One could choose another module name, not included in stdlib, implemented soley by static type checkers, to achive this goal. On Mon, 2022-01-17 at 18:59 +0000, Luis Miguel Morera De La Cruz wrote:

On Mon, Jan 17, 2022 at 10:12 AM Sebastian Rittau <srittau@rittau.biz> wrote:
Indeed -- I have used this workflow (alternating between running a static checker and running the code) and sometimes I've even done things like if not TYPE_CHECKING: reveal_type = print # Good enough :-) Still, I have a concern. Linters etc. tend to complain about unused imports. That means that you cannot leave the `from typing import reveal_type` in your code once you've removed the last reveal_type() call. But now when you want to use it, you'll have to add the import back in. So it would be much more convenient if it was an actual builtin. Regarding reveal_locals(), that's much less commonly used in my experience (it was added to mypy rather late) and I am hesitant to standardize it prematurely. (Alas, spotting usage "in the wild" of these functions is not easy, since their essence is that you have to remove them before running your code.) -- --Guido van Rossum (python.org/~guido) *Pronouns: he/him **(why is my pronoun here?)* <http://feministing.com/2015/02/03/how-using-they-as-a-singular-pronoun-can-c...>

El lun, 17 ene 2022 a las 19:10, Guido van Rossum (<guido@python.org>) escribió:
I see your point, but I'm not sure we'll be able to meet the bar for adding a new builtin. Also, I'm not proposing that type checkers remove the current behavior where reveal_type() is always an implicit builtin during type checking. So you'd only need to add the explicit import in two cases: - When you want to also run your code at the same time (e.g. to run your test suite) - When you want to communicate to readers where reveal_type() comes from (e.g. in a PEP example)

On Tue, Jan 18, 2022 at 12:25 PM Jelle Zijlstra <jelle.zijlstra@gmail.com> wrote:
Yeah, probably not.
I'd still be hesitant to add this to PEPs, even new PEPs, because once you know what reveal_type is, it rarely need an explanation (witness people here thinking that it was defined in PEP 484 -- in fact I searched that PEP myself :-). But the first bullet feels true, and just having the docs and the specification make it useful, so I am in favor of this. I think we should also add reveal_locals() and assert_type(), to promote, document and standardize them. -- --Guido van Rossum (python.org/~guido) *Pronouns: he/him **(why is my pronoun here?)* <http://feministing.com/2015/02/03/how-using-they-as-a-singular-pronoun-can-c...>

Would it be worthwhile also include filename, line and column numbers like type checkers do so you could easily track down where the call is from at runtime or should you just run a type checker to get them?

I realize I never responded to this suggestion. Sebastian had a similar idea in https://github.com/python/typing/issues/277#issuecomment-961061916: use the warnings module to emit a message. Personally I don't care much for the exact runtime behavior, but I'm wary of making it too complicated. I don't feel like tracking down where a call comes from is a very compelling use case: it's easy to just grep for, and your type checker should give you an error for any call to reveal_type() anyway. El mar, 18 ene 2022 a las 13:02, James H-B (<gobot1234yt@gmail.com>) escribió:
participants (9)
-
Dustin Wyatt
-
Eric Traut
-
Guido van Rossum
-
James H-B
-
Jelle Zijlstra
-
Luis Miguel Morera De La Cruz
-
Martin DeMello
-
Paul Bryan
-
Sebastian Rittau