Add an option that allow check Type Hints in runtime
Hi, all, Is it possible to add an option that allow check Type Hints in runtime? So that I can enable dynamic type checking when debugging. And disable it in final deploy, no performance lose. regards.
This exists already. All the annotations love on the '__annotations__' attribute is a function or method. Check that at runtime however you like! On Jul 7, 2016 11:39 PM, "Ma Lin" <animalize81@hotmail.com> wrote:
Hi, all,
Is it possible to add an option that allow check Type Hints in runtime? So that I can enable dynamic type checking when debugging. And disable it in final deploy, no performance lose.
regards. _______________________________________________ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
I forget to add python-ideas@python.org<mailto:python-ideas@python.org> in CC, let me retell to the others: I mean real check by the interpreter, not an '__annotations__' attribute. And I just had a look, mypy is a static checker, without actually running the code. My thought is modifying CALL_FUNCTION and RETURN_VALUE of the interpreter, so that allow them to check in each function calling when option enalbed. Cite a case: def fun(a: str, b: int) -> int: return len(a) + b fun('arg1', 'arg2') Whit the option and run this code, the interpreter will give an error prompt: Type of argument b is wrong. (or something like this) This is very useful for development and debugging. Without the option, the interpreter just ignores Type Hints in runtime, so no performance lose in project's final deploy. IMO, this will maximize the benefit of Type Hints within a small cost. 在 7-8 周五 14:42, David Mertz 写道: This exists already. All the annotations love on the '__annotations__' attribute is a function or method. Check that at runtime however you like! On Jul 7, 2016 11:39 PM, "Ma Lin" <animalize81@hotmail.com<mailto:animalize81@hotmail.com>> wrote: Hi, all, Is it possible to add an option that allow check Type Hints in runtime? So that I can enable dynamic type checking when debugging. And disable it in final deploy, no performance lose. regards. _______________________________________________ Python-ideas mailing list Python-ideas@python.org<mailto:Python-ideas@python.org> https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
On Fri, Jul 8, 2016 at 6:19 PM, Ma Lin <animalize81@hotmail.com> wrote:
I mean real check by the interpreter, not an '__annotations__' attribute. And I just had a look, mypy is a static checker, without actually running the code.
My thought is modifying CALL_FUNCTION and RETURN_VALUE of the interpreter, so that allow them to check in each function calling when option enalbed.
Cite a case:
def fun(a: str, b: int) -> int: return len(a) + b fun('arg1', 'arg2')
Whit the option and run this code, the interpreter will give an error prompt: Type of argument b is wrong. (or something like this)
This is very useful for development and debugging.
You should still be able to do it without changing the core interpreter. def enforce_types(func): @functools.wraps(func) def wrapped(*args, **kw): if not args_valid(args, kw, func.__annotations__): raise TypeError ret = func(*args, **kw) if not type_valid(ret, func.__annotations__["return"]): raise TypeError return wrapped @enforce_types def fun(a: str, b: int) -> int: return len(a) + b fun('arg1', 'arg2') Now all you have to do is write the type_valid function (takes an object and an annotation, returns True if it matches) and args_valid (takes the positional and keyword args, pairs them up against the function's params, and returns True if all(type_valid(...)) for the args). The only benefit of hacking the interpreter core for this is that you'd save the hassle of pairing up positional/keyword args against the parameters. ChrisA
On 8 July 2016 at 09:30, Chris Angelico <rosuav@gmail.com> wrote:
On Fri, Jul 8, 2016 at 6:19 PM, Ma Lin <animalize81@hotmail.com> wrote:
I mean real check by the interpreter, not an '__annotations__' attribute. And I just had a look, mypy is a static checker, without actually running the code.
My thought is modifying CALL_FUNCTION and RETURN_VALUE of the interpreter, so that allow them to check in each function calling when option enalbed. [...]
Now all you have to do is write the type_valid function (takes an object and an annotation, returns True if it matches) and args_valid (takes the positional and keyword args, pairs them up against the function's params, and returns True if all(type_valid(...)) for the args). The only benefit of hacking the interpreter core for this is that you'd save the hassle of pairing up positional/keyword args against the parameters.
Just to clarify for the OP - the reason you're getting pushback on making this a built in interpreter function is twofold - first, an external facility could be used *now*, and would not need to be limited to Python 3.6+, and second, adding checks to the function call protocol is not free, and would therefore affect not just the minority of people who want to add checks for debugging, but also all other users (which include people for whom Python's performance is a key issue). The Python interpreter's function calling process is known to be slower than (some) people would like, and making it slower - even marginally - isn't something we would want to do lightly. With the ability to use external static analyzers like mypy, as well as the possibility of writing custom solutions like Chris described above, people wanting such checks have a number of options already, and so the cost of adding the feature to the interpreter (and possibly the language - do you see this as a CPython implementation detail, or would you expect it to be a language-level facility, so other Python implementations such as PyPy and Jython would be expected to have it as well?) needs to be justified. Personally, I'd view this feature as an interesting addition, but not something I'd use myself, and so I'm inclined towards the "why should I pay for something I won't use?" position. Paul
On Fri, Jul 8, 2016 at 6:50 PM, Paul Moore <p.f.moore@gmail.com> wrote:
Just to clarify for the OP - the reason you're getting pushback on making this a built in interpreter function is twofold - first, an external facility could be used *now*, and would not need to be limited to Python 3.6+, and second, adding checks to the function call protocol is not free, and would therefore affect not just the minority of people who want to add checks for debugging, but also all other users (which include people for whom Python's performance is a key issue). The Python interpreter's function calling process is known to be slower than (some) people would like, and making it slower - even marginally - isn't something we would want to do lightly.
Exactly. The main reason IMO is the first one - you can add this checker to any Python 3 program, rather than wait for it to get coded, debugged, and shipped. Oh, and you can easily have a flag that disables checking for performance: if no_type_checks: def enforce_types(func): return func Now your decorator does nothing, and there is literally ZERO run-time cost (the only cost is a single cheap check when the function is defined, which usually means on module import), closures aside. That wouldn't be possible with a core interpreter change, unless it's made a compile-time option *for the interpreter* - something like "./configure --with-annotations-enforced" - and then you run python3.6 for the fast one, or python3.6tc for the type-checking one. That would be a pain. ChrisA
On Fri, Jul 8, 2016 at 7:02 PM, Chris Angelico <rosuav@gmail.com> wrote:
Now your decorator does nothing, and there is literally ZERO run-time cost (the only cost is a single cheap check when the function is defined, which usually means on module import), closures aside. That wouldn't be possible with a core interpreter change...
I really shouldn't say impossible. Literally the moment I clicked Send, I thought of a way it could be done with minimal run-time cost: just create a completely separate opcode for "type-checked call". I don't think that would materially slow down the regular call operation. As a bonus, any functions that don't have annotations could use the normal call opcode, so they wouldn't be slowed down. But the main point is, this would all entail additional complexity inside CPython, where the decorator method keeps the complexity in your application - and doesn't need nearly as much of it. The one advantage that CPython has is, as mentioned, the matching of arguments to parameters; maybe that could be provided in functools somewhere? ChrisA
I see, Chris' decorator is amazing, I didn't think that way. I mean add an runtime option, not a build option, it can be only enabled in development/debugging: python -tc a.py Without the option, the interpreter just ignores Type Hints: python a.py Anyway, I just throw an idea to here. Greet to all of you for bring Python to me.
On Fri, Jul 8, 2016 at 7:17 PM, Ma Lin <animalize81@hotmail.com> wrote:
I see, Chris' decorator is amazing, I didn't think that way.
That's one of the beauties of Python's model of "everything's an object". Functions can be manipulated (usually in broadly the same way as my example: creating a wrapper function that does stuff before and/or after calling the original), and you can basically do anything you like with the attributes. It's SO handy, in so many places! That said, though, you *can* go too far with that. I discovered recently that Ruby lacks a lot of fairly important features... which are monkey-patched in by Rails. So a Ruby on Rails application has a lot more functionality, even on core types (because you can add methods to the string type by monkey-patching), than a non-Rails Ruby app. There are some things that definitely do need interpreter support, which is why we're now seeing async and await becoming keywords.
I mean add an runtime option, not a build option, it can be only enabled in development/debugging: python -tc a.py Without the option, the interpreter just ignores Type Hints: python a.py
Right, but this means that the code to call a function has to check "was the --tc option specified?" before actually calling the function. Even though the full type checks aren't done, this check has a nonzero cost; and since calling functions happens rather a lot in Python programs (remember, lots of things implicitly become function or method calls), it would add up. Everyone's code would be slowed down slightly to support the occasional case where you want the checking.
Anyway, I just throw an idea to here. Greet to all of you for bring Python to me.
Keep on throwing ideas this way! Ideas are great. Well-thought-out ideas make for interesting discussions, even if they don't get implemented. Often the answer ends up being "here's a way to do this with the current version", but it doesn't carry with it a tag of "and you're an idiot for even suggesting it". We like ideas here - otherwise we wouldn't subscribe to this list :) ChrisA
On 8 July 2016 at 10:35, Chris Angelico <rosuav@gmail.com> wrote:
Anyway, I just throw an idea to here. Greet to all of you for bring Python to me.
Keep on throwing ideas this way! Ideas are great. Well-thought-out ideas make for interesting discussions, even if they don't get implemented. Often the answer ends up being "here's a way to do this with the current version", but it doesn't carry with it a tag of "and you're an idiot for even suggesting it". We like ideas here - otherwise we wouldn't subscribe to this list :)
Agreed! I'd have never even thought of Chris' decorator approach before this discussion occurred, so thanks for raising the subject. I may never use it, but knowing how to do something like this is great. Paul
On 8 July 2016 at 20:34, Paul Moore <p.f.moore@gmail.com> wrote:
On 8 July 2016 at 10:35, Chris Angelico <rosuav@gmail.com> wrote:
Anyway, I just throw an idea to here. Greet to all of you for bring Python to me.
Keep on throwing ideas this way! Ideas are great. Well-thought-out ideas make for interesting discussions, even if they don't get implemented. Often the answer ends up being "here's a way to do this with the current version", but it doesn't carry with it a tag of "and you're an idiot for even suggesting it". We like ideas here - otherwise we wouldn't subscribe to this list :)
Agreed! I'd have never even thought of Chris' decorator approach before this discussion occurred, so thanks for raising the subject. I may never use it, but knowing how to do something like this is great.
The runtime eval loop switching being considered to enable Pyjion's method JIT would also let folks opt in to running a dynamically typechecked runtime (assuming someone decided to write one). Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia
I provided a decorator recipe to do just what Chris Angelico describes a year or two before PEP 484 described static typechecking. I think what I do is a consistent subset of PEP 484 still, modulo the difference between runtime and static. See: http://code.activestate.com/recipes/578528-type-checking-using-python-3x-ann... It would be a lot more work to do everything the typing.py module and PEP 484 describes (algebraic typing, basically), but it's all just code that is compatible back to Python 3.0. On Fri, Jul 8, 2016 at 3:34 AM, Paul Moore <p.f.moore@gmail.com> wrote:
On 8 July 2016 at 10:35, Chris Angelico <rosuav@gmail.com> wrote:
Anyway, I just throw an idea to here. Greet to all of you for bring Python to me.
Keep on throwing ideas this way! Ideas are great. Well-thought-out ideas make for interesting discussions, even if they don't get implemented. Often the answer ends up being "here's a way to do this with the current version", but it doesn't carry with it a tag of "and you're an idiot for even suggesting it". We like ideas here - otherwise we wouldn't subscribe to this list :)
Agreed! I'd have never even thought of Chris' decorator approach before this discussion occurred, so thanks for raising the subject. I may never use it, but knowing how to do something like this is great.
Paul _______________________________________________ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
-- Keeping medicines from the bloodstreams of the sick; food from the bellies of the hungry; books from the hands of the uneducated; technology from the underdeveloped; and putting advocates of freedom in prisons. Intellectual property is to the 21st century what the slave trade was to the 16th.
pycontracts supports runtime type and bound checking on Python 2 & 3 There is a global toggle: https://andreacensi.github.io/contracts/overhead.html#overhead Unfortunately, the type specifiers precede and are different (e.g lowercase) from those used for static analysis in mypy. There are orobably additional differences? On Jul 8, 2016 11:04 AM, "David Mertz" <mertz@gnosis.cx> wrote:
I provided a decorator recipe to do just what Chris Angelico describes a year or two before PEP 484 described static typechecking. I think what I do is a consistent subset of PEP 484 still, modulo the difference between runtime and static.
See: http://code.activestate.com/recipes/578528-type-checking-using-python-3x-ann...
It would be a lot more work to do everything the typing.py module and PEP 484 describes (algebraic typing, basically), but it's all just code that is compatible back to Python 3.0.
On Fri, Jul 8, 2016 at 3:34 AM, Paul Moore <p.f.moore@gmail.com> wrote:
On 8 July 2016 at 10:35, Chris Angelico <rosuav@gmail.com> wrote:
Anyway, I just throw an idea to here. Greet to all of you for bring Python to me.
Keep on throwing ideas this way! Ideas are great. Well-thought-out ideas make for interesting discussions, even if they don't get implemented. Often the answer ends up being "here's a way to do this with the current version", but it doesn't carry with it a tag of "and you're an idiot for even suggesting it". We like ideas here - otherwise we wouldn't subscribe to this list :)
Agreed! I'd have never even thought of Chris' decorator approach before this discussion occurred, so thanks for raising the subject. I may never use it, but knowing how to do something like this is great.
Paul _______________________________________________ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
-- Keeping medicines from the bloodstreams of the sick; food from the bellies of the hungry; books from the hands of the uneducated; technology from the underdeveloped; and putting advocates of freedom in prisons. Intellectual property is to the 21st century what the slave trade was to the 16th.
_______________________________________________ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
On 7/8/16, Chris Angelico <rosuav@gmail.com> wrote:
On Fri, Jul 8, 2016 at 7:17 PM, Ma Lin <animalize81@hotmail.com> wrote:
I mean add an runtime option, not a build option, it can be only enabled in development/debugging: python -tc a.py Without the option, the interpreter just ignores Type Hints: python a.py
Right, but this means that the code to call a function has to check "was the --tc option specified?" before actually calling the function.
Could not interpreter just internally use "decorator trick" too?
participants (7)
-
Chris Angelico -
David Mertz -
Ma Lin -
Nick Coghlan -
Paul Moore -
Pavol Lisy -
Wes Turner