It would be great to have better type hinting in IDEs. My usecase: logger = logging.getLogger(__name__) try: ... except FooException, exc: logger.warn('...') I remember there was a way to show the traceback via logger.warn(). I could use my favorite search engine, but a short cut via the IDE would be much easier. How can the IDE know what kind of duck "logger" is? Many IDEs parse the docstrings, but a lot of code does not provide it. How can this be improved? Regards, Thomas Güttler PS: I don't mention the name of my IDE intentionally :-) It does not matter for this question.
Thomas Güttler schrieb am 05.06.2015 um 07:36:
It would be great to have better type hinting in IDEs.
Sounds more like a topic for python-list than python-ideas.
Yes it does. It sounds like you want to use an IDE instead that supports the above. Or install a plugin for the one you're using that improves its capabilities for type introspection. There are a couple of IDE plugins that embed jedi, for example. Stefan
On Jun 4, 2015, at 22:36, Thomas Güttler <guettliml@thomas-guettler.de> wrote:
It would be great to have better type hinting in IDEs.
Is PEP 484 not sufficient for this purpose? Of course you'll have to wait until 3.5 or use an external backport (I think the goal is for MyPy to include stub files for 2.7/3.3/3.4, and/or for them to be published separately on PyPI?), and even longer for every library you depend on to get on board. And of course your favorite IDE has to actually do something with these type hints, and integrate an inference engine (whether that means running something like MyPy in the background or implementing something themselves). But I don't think there's anything Python itself can do to speed any of that up. And meanwhile, I suppose it's possible that the PEP 484 design will turn out to be insufficient, but there's no way we're going to know that until the IDEs try to use it and fail.
Is there a reason you're using syntax that's deprecate since Python 2.6 and doesn't work in 3.x? Any proposal for the future of the Python language isn't going to help you if you're still using 2.5.
On Thu, Jun 4, 2015, at 11:01 PM, Andrew Barnert via Python-ideas wrote:
It's really not. For one thing, PEP 484 isn't going to result in the standard library being hinted all up (though I assume someone may make stubs). But really, the specific issue that the OP is running into is because of the signature of logging.warn -- msg, *args, **kwargs. These kinds of signatures are very useful in certain circumstances but they are also completely opaque. They're intentionally taking "anything" and passing it off to another function. PEP 484 doesn't say anything about the realities of logging.warn, as all the work is being done in the private _log where we can examine and learn that it takes two optional keyword parameters named "exc_info" and "extra", or what those mean or what valid values are for them. All my preferred IDE tells me is, "msg, *args, **kwargs", which leaves me befuddled if I don't remember the signature or have the docs in hand. If it were to display the docstring too, I'd know "exc_info" is a valid keyword argument that does something useful, but I'd still have no idea about "extra" (and I actually have no idea what extra does and am not looking it up right now on purpose). I don't really think that this is a problem for Python the language, but maybe the style guide: don't use *args or **kwargs unless you either document the details of what those should be, ooor, maybe include a @functools.passes (fictional device) that in some fashion documents "look at this other function for the things I'm passing along blindly). The problem the OP is demonstrating is really completely out of scope for what PEP 484 is addressing, I think. It has little to do with type hinting and more to do with, IMHO, "should the stdlib provide more introspectable signatures" (which then IDE's could use). -- Stephen Hansen m e @ i x o k a i . i o
On 5 June 2015 at 16:19, Stephen Hansen <me+python@ixokai.io> wrote:
Doing exactly that is a core part of the PEP 484 effort, since it's needed to assist in Python 2 -> 3 migrations: https://github.com/JukkaL/typeshed One of the advantages of that is that more specific signatures can be added to stdlib stubs and benefit IDEs in existing Python versions, rather than having to wait for more explicit signatures in future Python versions. Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia
On Fri, Jun 5, 2015, at 12:10 AM, Nick Coghlan wrote:
How so? Nothing in PEP 484 addresses signatures which take *args and **kwargs arguments. Please, correct me if I'm wrong, but my understanding is you can uselessly specify types for a Mapping, but you can't specify what actual keys are valid in that mapping. That's the problem with logging's specification, its functions take "anything" and pass on "anything", in their signature... In reality they take up two keyword arguments -- exc_info and extra. Args really is "anything" as its just formatted against the string. Unless I'm missing something, PEP484 only allows defining *types* of specific arguments -- but this isn't about types. This is about what arguments are valid (and then, after you have that bit of knowing, what types come next). When used with API's that take *args and **kwargs, I don't see how PEP484 is useful at all. I'm not arguing against PEP484. but it has nothing at all to do with the specific problem mentioned here. Dynamic API's that take "any args" and "any kwargs" are opaque things it doesn't tell anything about. Logger.warn (and, logging.warn) is one such API. On Fri, Jun 5, 2015, at 12:21 AM, Thomas Güttler wrote:
Same difference. Logging.warn is just a thin wrapper around the root logger's warn(). They still have the same completely opaque signature, of msg, *args, **kwargs that it passes along, which is why your IDE can't report a useful signature that tells you that exc_info=True is what you want. --S
Am 05.06.2015 um 08:19 schrieb Stephen Hansen:
I am using logger.warn() not logging.warn(). The question is: How to know which kind of duck "logger" is? "logger" was created by "logging.getLogger(__name__)" It is not the question how to implement better guessing in the IDE. The basics needs to be solved. Everything else is "toilet paper programming" (Ah, smell inside, ... let's write an wrapper ...) Regards, Thomas Güttler
This question is unanswerable unless you actually execute the code at runtime under the exact same conditions as you expect to encounter it. Because Python allows for monkey patching at runtime by any other code running in the process you can make no assumptions about what kind of duck that will be. Even without monkey patching you can’t know, because someone may have adjusted sys.path ahead of time, causing you to import an entirely unexpected module called ‘logging’. In certain specialised cases, if you limit yourself to special rules, you *might* be able to statically assert the type of this object, but in the general case it simply cannot be done. So the only way to improve this is to implement better guessing in the IDE. Hence the improvements that were proposed to you.
Cory Benfield schrieb am 05.06.2015 um 09:36:
That doesn't mean that it's impossible to find enough type information to make an IDE present something helpful to a user. In all interesting cases, the object returned by logging.getLogger() will be a logger instance with a well-known interface, and tools can just know that. Tools like Jedi and PyCharm show that this is definitely possible. Stefan
On Jun 5, 2015, at 00:21, Thomas Güttler <guettliml@thomas-guettler.de> wrote:
That is _exactly_ what PEP 484 addresses. If `logging.getLogger` is annotated or stubbed to specify that it returns a `logging.Logger` (which it will be), then a static type checker (whether MyPy or a competing checker or custom code in the IDE) can trivially infer that `logger` is a `logging.Logger`. If you needed to annotate exactly which subclass of `Logger` was returned (unlikely, but not impossible--maybe you conditionally do a `logging.set_logger_class`, and _you_ know what the type is going to be even though a static analyzer can't infer it, and your subclass has a different API than the base class), then you can use a variable type comment. As Stephen Hansen argues, that may still not solve all of your problems. But it definitely does solve the "how to know which kind of duck" problem you're asking about.
Have you read PEP 484? What part of the basics do you think it's not solving? Because it sounds an awful lot like you're just demanding that someone write something exactly like PEP 484.
Am 05.06.2015 um 10:38 schrieb Andrew Barnert:
Now I read it and it does exactly what I was looking for.
Unfortunately we still use Python2.7, but maybe it is time for change ... Just one thing left:
**If** `logging.getLogger` is annotated ....
What is the policy of the standard library? Will there be type hints for methods like logging.getLogger() in the standard library in the future? Since it is quite easy to add them, will patches be accepted? Regards, Thomas Güttler
On 5 Jun 2015 23:34, "Thomas Güttler" <guettliml@thomas-guettler.de> wrote:
Am 05.06.2015 um 10:38 schrieb Andrew Barnert:
On Jun 5, 2015, at 00:21, Thomas Güttler <guettliml@thomas-guettler.de>
wrote: the
The typeshed project provides stubs for both Python 2 & 3: <https://github.com/JukkaL/typeshed>https <https://github.com/JukkaL/typeshed>:// <https://github.com/JukkaL/typeshed> github.com <https://github.com/JukkaL/typeshed>/ <https://github.com/JukkaL/typeshed>JukkaL <https://github.com/JukkaL/typeshed>/ <https://github.com/JukkaL/typeshed> typeshed <https://github.com/JukkaL/typeshed> Type hinting your own code where appropriate would be easier in Python 3 (since you can use inline type hints)
The standard library won't be getting native annotations any time soon, but the typeshed annotations are expected to fill the gap.
Since it is quite easy to add them, will patches be accepted?
Contributions to the typeshed stubs would be preferable. Cheers, Nick.
Thomas Güttler schrieb am 05.06.2015 um 07:36:
It would be great to have better type hinting in IDEs.
Sounds more like a topic for python-list than python-ideas.
Yes it does. It sounds like you want to use an IDE instead that supports the above. Or install a plugin for the one you're using that improves its capabilities for type introspection. There are a couple of IDE plugins that embed jedi, for example. Stefan
On Jun 4, 2015, at 22:36, Thomas Güttler <guettliml@thomas-guettler.de> wrote:
It would be great to have better type hinting in IDEs.
Is PEP 484 not sufficient for this purpose? Of course you'll have to wait until 3.5 or use an external backport (I think the goal is for MyPy to include stub files for 2.7/3.3/3.4, and/or for them to be published separately on PyPI?), and even longer for every library you depend on to get on board. And of course your favorite IDE has to actually do something with these type hints, and integrate an inference engine (whether that means running something like MyPy in the background or implementing something themselves). But I don't think there's anything Python itself can do to speed any of that up. And meanwhile, I suppose it's possible that the PEP 484 design will turn out to be insufficient, but there's no way we're going to know that until the IDEs try to use it and fail.
Is there a reason you're using syntax that's deprecate since Python 2.6 and doesn't work in 3.x? Any proposal for the future of the Python language isn't going to help you if you're still using 2.5.
On Thu, Jun 4, 2015, at 11:01 PM, Andrew Barnert via Python-ideas wrote:
It's really not. For one thing, PEP 484 isn't going to result in the standard library being hinted all up (though I assume someone may make stubs). But really, the specific issue that the OP is running into is because of the signature of logging.warn -- msg, *args, **kwargs. These kinds of signatures are very useful in certain circumstances but they are also completely opaque. They're intentionally taking "anything" and passing it off to another function. PEP 484 doesn't say anything about the realities of logging.warn, as all the work is being done in the private _log where we can examine and learn that it takes two optional keyword parameters named "exc_info" and "extra", or what those mean or what valid values are for them. All my preferred IDE tells me is, "msg, *args, **kwargs", which leaves me befuddled if I don't remember the signature or have the docs in hand. If it were to display the docstring too, I'd know "exc_info" is a valid keyword argument that does something useful, but I'd still have no idea about "extra" (and I actually have no idea what extra does and am not looking it up right now on purpose). I don't really think that this is a problem for Python the language, but maybe the style guide: don't use *args or **kwargs unless you either document the details of what those should be, ooor, maybe include a @functools.passes (fictional device) that in some fashion documents "look at this other function for the things I'm passing along blindly). The problem the OP is demonstrating is really completely out of scope for what PEP 484 is addressing, I think. It has little to do with type hinting and more to do with, IMHO, "should the stdlib provide more introspectable signatures" (which then IDE's could use). -- Stephen Hansen m e @ i x o k a i . i o
On 5 June 2015 at 16:19, Stephen Hansen <me+python@ixokai.io> wrote:
Doing exactly that is a core part of the PEP 484 effort, since it's needed to assist in Python 2 -> 3 migrations: https://github.com/JukkaL/typeshed One of the advantages of that is that more specific signatures can be added to stdlib stubs and benefit IDEs in existing Python versions, rather than having to wait for more explicit signatures in future Python versions. Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia
On Fri, Jun 5, 2015, at 12:10 AM, Nick Coghlan wrote:
How so? Nothing in PEP 484 addresses signatures which take *args and **kwargs arguments. Please, correct me if I'm wrong, but my understanding is you can uselessly specify types for a Mapping, but you can't specify what actual keys are valid in that mapping. That's the problem with logging's specification, its functions take "anything" and pass on "anything", in their signature... In reality they take up two keyword arguments -- exc_info and extra. Args really is "anything" as its just formatted against the string. Unless I'm missing something, PEP484 only allows defining *types* of specific arguments -- but this isn't about types. This is about what arguments are valid (and then, after you have that bit of knowing, what types come next). When used with API's that take *args and **kwargs, I don't see how PEP484 is useful at all. I'm not arguing against PEP484. but it has nothing at all to do with the specific problem mentioned here. Dynamic API's that take "any args" and "any kwargs" are opaque things it doesn't tell anything about. Logger.warn (and, logging.warn) is one such API. On Fri, Jun 5, 2015, at 12:21 AM, Thomas Güttler wrote:
Same difference. Logging.warn is just a thin wrapper around the root logger's warn(). They still have the same completely opaque signature, of msg, *args, **kwargs that it passes along, which is why your IDE can't report a useful signature that tells you that exc_info=True is what you want. --S
Am 05.06.2015 um 08:19 schrieb Stephen Hansen:
I am using logger.warn() not logging.warn(). The question is: How to know which kind of duck "logger" is? "logger" was created by "logging.getLogger(__name__)" It is not the question how to implement better guessing in the IDE. The basics needs to be solved. Everything else is "toilet paper programming" (Ah, smell inside, ... let's write an wrapper ...) Regards, Thomas Güttler
This question is unanswerable unless you actually execute the code at runtime under the exact same conditions as you expect to encounter it. Because Python allows for monkey patching at runtime by any other code running in the process you can make no assumptions about what kind of duck that will be. Even without monkey patching you can’t know, because someone may have adjusted sys.path ahead of time, causing you to import an entirely unexpected module called ‘logging’. In certain specialised cases, if you limit yourself to special rules, you *might* be able to statically assert the type of this object, but in the general case it simply cannot be done. So the only way to improve this is to implement better guessing in the IDE. Hence the improvements that were proposed to you.
Cory Benfield schrieb am 05.06.2015 um 09:36:
That doesn't mean that it's impossible to find enough type information to make an IDE present something helpful to a user. In all interesting cases, the object returned by logging.getLogger() will be a logger instance with a well-known interface, and tools can just know that. Tools like Jedi and PyCharm show that this is definitely possible. Stefan
On Jun 5, 2015, at 00:21, Thomas Güttler <guettliml@thomas-guettler.de> wrote:
That is _exactly_ what PEP 484 addresses. If `logging.getLogger` is annotated or stubbed to specify that it returns a `logging.Logger` (which it will be), then a static type checker (whether MyPy or a competing checker or custom code in the IDE) can trivially infer that `logger` is a `logging.Logger`. If you needed to annotate exactly which subclass of `Logger` was returned (unlikely, but not impossible--maybe you conditionally do a `logging.set_logger_class`, and _you_ know what the type is going to be even though a static analyzer can't infer it, and your subclass has a different API than the base class), then you can use a variable type comment. As Stephen Hansen argues, that may still not solve all of your problems. But it definitely does solve the "how to know which kind of duck" problem you're asking about.
Have you read PEP 484? What part of the basics do you think it's not solving? Because it sounds an awful lot like you're just demanding that someone write something exactly like PEP 484.
Am 05.06.2015 um 10:38 schrieb Andrew Barnert:
Now I read it and it does exactly what I was looking for.
Unfortunately we still use Python2.7, but maybe it is time for change ... Just one thing left:
**If** `logging.getLogger` is annotated ....
What is the policy of the standard library? Will there be type hints for methods like logging.getLogger() in the standard library in the future? Since it is quite easy to add them, will patches be accepted? Regards, Thomas Güttler
On 5 Jun 2015 23:34, "Thomas Güttler" <guettliml@thomas-guettler.de> wrote:
Am 05.06.2015 um 10:38 schrieb Andrew Barnert:
On Jun 5, 2015, at 00:21, Thomas Güttler <guettliml@thomas-guettler.de>
wrote: the
The typeshed project provides stubs for both Python 2 & 3: <https://github.com/JukkaL/typeshed>https <https://github.com/JukkaL/typeshed>:// <https://github.com/JukkaL/typeshed> github.com <https://github.com/JukkaL/typeshed>/ <https://github.com/JukkaL/typeshed>JukkaL <https://github.com/JukkaL/typeshed>/ <https://github.com/JukkaL/typeshed> typeshed <https://github.com/JukkaL/typeshed> Type hinting your own code where appropriate would be easier in Python 3 (since you can use inline type hints)
The standard library won't be getting native annotations any time soon, but the typeshed annotations are expected to fill the gap.
Since it is quite easy to add them, will patches be accepted?
Contributions to the typeshed stubs would be preferable. Cheers, Nick.
participants (6)
-
Andrew Barnert -
Cory Benfield -
Nick Coghlan -
Stefan Behnel -
Stephen Hansen -
Thomas Güttler