Possible enhancement to typing
The discussions on moving typing to a standalone library prompted me to take a look at the documentation for typing which in turn lead to an idea that should be reasonably simple to implement but greatly add value to typing for some applications. If a group of iterators were to be added to the typing module it would be reasonably simple to automatically add and assert to any decorated modules to ensure that such modules were always called with the documented types. I am thinking of decorators such as: - @typing.assert_params_mismatch - this would provide a wrapper that had auto-generated asserts that all the parameters were of designated types. - @typing.debug_assert_params_mismatch - this would provide a wrapper that had auto-generated asserts that all the parameters were of designated types only if a DEBUG environmental variable was set or similar. - @typing.warn_params_mismatch - this would provide a wrapper that had auto-generated warnings issued to stderr if any of the parameters were of not of the designated types. - @typing.coerce_params - this would add a wrapper to attempt to convert any mismatched parameters to the specified type(s). I also think that this might increase the uptake of typing by giving some clear benefits outside of documentation and static type checking. Pushing this out for comment before considering a more formal, (PEP), proposal. -- Steve (Gadget) Barnes Any opinions in this message are my personal opinions and do not reflect those of my employer. --- This email has been checked for viruses by AVG. http://www.avg.com
On Sun, Nov 05, 2017 at 07:18:30PM +0000, Steve Barnes wrote:
"Iterators"?
That's what assert does: assert only runs when __DEBUG__ is true. That's not controlled by an environment variable, but by the -O flag to the interpreter. So your assert_params_mismatch and debug_assert_params_mismatch are effectively the same thing. But using assert to check to perform argument checks is often an abuse of assert. To be more specific, using assert to check the value of public arguments in library code (where the arguments come from outside the library) is wrong, since you (the library author) cannot guarantee that your type tests will even run. Using asserts for argument checking inside application code is more of a grey area, with arguments for and against using assert. But in my opinion, the deciding factor is nearly always that an AssertionError is the wrong sort of exception. Outside of some fairly limited circumstances, most of which don't involve type-checking function arguments, using assert robs the caller of some useful information: the *kind* of error. (TypeError, ValueError, etc.) See here for further discussion: https://import-that.dreamwidth.org/676.html In general, I don't think we want to encourage such runtime type testing. Obviously there are exceptions -- library code should probably type check arguments, applications perhaps not -- and we're not exactly discouraging it either. There are already a number of third-party libraries that provide argument type tests at runtime, and I think that's probably the right place for them. [...]
I also think that this might increase the uptake of typing by giving some clear benefits outside of documentation and static type checking.
Problem is, the benefits of runtime type checking aren't clear. But the costs certainly are: if you want slow code, do lots and lots of runtime type checks. -- Steve
On 06/11/2017 07:13, Steven D'Aprano wrote:
Good point.
I see your point here.
I'll have to look out for them.
Too much time spent writing safety critical code on my part then! I'll drop the idea. -- Steve (Gadget) Barnes Any opinions in this message are my personal opinions and do not reflect those of my employer. --- This email has been checked for viruses by AVG. http://www.avg.com
A few random remarks: 1) This seems similar to the Typeguard project https://github.com/agronholm/typeguard I personally never used it. I'd rather have some machinery that would run the checks "magically" using the type annotations, without the need for explicit decoration. 2) I fully agree that there is a important difference between using runtime checks to verify your code (for instance, as part of an automated test suite, or even when manually testing an application), and for data validation. 3) In practice, however, I have never disabled assertions in production, but I agree if can be problematic if developers confuse AssertionErrors with TypeErros. 4) 10 years ago, when I was working on the EDOS project ( http://cordis.europa.eu/pub/ist/docs/directorate_d/st-ds/edos-project-story_... ), I ran a small experiment where I used, IIRC, the profile hook to intercept all function / method calls, and log information about arguments and return value types to a gigantic log file. Then the log file could be parsed and these information used to suggest type annotations. Except there were no type annotations at the time in Python. I know PyCharm can do a similar thing now: you run your program or your tests under the debugger, it logs runtime type information somewhere, and then can use it to suggest autocompletion or maybe type annotations. Now I believe something could be done along the lines: a) record runtime type information from test or regular runs b) massage these information and use them to annotate Python code with additional type information (up to the developer to then accept or not the proposed changes) c) also run a test suite or an app under some magical machinery, and either raise a TypeError or log warnings when discrepancies are detected between type annotation and runtime behaviour. (c) could be done independently from (a) and (b), (a) and (b) would use similar machinery, and (a), (b) and (c) would be probably a useful way to introduce type annotations to an existing code base without too much risk. (a) and (b) could also provide data for an interesting SE research project. S. On Mon, Nov 6, 2017 at 8:39 AM, Steve Barnes <gadgetsteve@live.co.uk> wrote:
-- Stefane Fermigier - http://fermigier.com/ - http://twitter.com/sfermigier - http://linkedin.com/in/sfermigier Founder & CEO, Abilian - Enterprise Social Software - http://www.abilian.com/ Chairman, Free&OSS Group / Systematic Cluster - http://www.gt-logiciel-libre.org/ Co-Chairman, National Council for Free & Open Source Software (CNLL) - http://cnll.fr/ Founder & Organiser, PyData Paris - http://pydata.fr/ --- “You never change things by fighting the existing reality. To change something, build a new model that makes the existing model obsolete.” — R. Buckminster Fuller
On Mon, Nov 6, 2017 at 11:34 AM, Stéfane Fermigier <sf@fermigier.com> wrote:
[...]
I mean, (a) and (c)
Similar to "Measuring Polymorphism in Python Programs", by Beatrice Akerblom and Tobias Wrigstad: https://people.dsv.su.se/~beatrice/python/dls15_large_images.pdf S. -- Stefane Fermigier - http://fermigier.com/ - http://twitter.com/sfermigier - http://linkedin.com/in/sfermigier Founder & CEO, Abilian - Enterprise Social Software - http://www.abilian.com/ Chairman, Free&OSS Group / Systematic Cluster - http://www.gt-logiciel-libre.org/ Co-Chairman, National Council for Free & Open Source Software (CNLL) - http://cnll.fr/ Founder & Organiser, PyData Paris - http://pydata.fr/ --- “You never change things by fighting the existing reality. To change something, build a new model that makes the existing model obsolete.” — R. Buckminster Fuller
On Mon, Nov 6, 2017 at 2:34 AM, Stéfane Fermigier <sf@fermigier.com> wrote:
I didn't know this. Do you know where there are docs for this feature?
We have an early version of a tool that does this at Dropbox; I am planning to open-source it by the end of this year. So far the experience is that the annotations require a fair amount of manual cleanup though. [snip]
<https://people.dsv.su.se/~beatrice/python/dls15_large_images.pdf> Thanks for the link! -- --Guido van Rossum (python.org/~guido)
On Tue, Nov 7, 2017 at 2:58 AM, Guido van Rossum <guido@python.org> wrote:
This was described in this blog post when first introduced: https://blog.jetbrains.com/pycharm/2013/02/dynamic-runtime-type-inference-in... And more tersely, in the documentation: https://www.jetbrains.com/help/pycharm/python-debugger.html a) record runtime type information from test or regular runs
That would be great ! S. -- Stefane Fermigier - http://fermigier.com/ - http://twitter.com/sfermigier - http://linkedin.com/in/sfermigier Founder & CEO, Abilian - Enterprise Social Software - http://www.abilian.com/ Chairman, Free&OSS Group / Systematic Cluster - http://www.gt-logiciel-libre.org/ Co-Chairman, National Council for Free & Open Source Software (CNLL) - http://cnll.fr/ Founder & Organiser, PyData Paris - http://pydata.fr/ --- “You never change things by fighting the existing reality. To change something, build a new model that makes the existing model obsolete.” — R. Buckminster Fuller
On Tue, Nov 7, 2017 at 9:27 PM, Guido van Rossum <guido@python.org> wrote:
The blog post was written in 2013, PEP 484 was not yet adopted at the time, and so they targeted their own typechecker and typing annotation conventions, which were at the time based on docstrings conventions. I have just checked with the current PyCharm version, and when using the "add annotation" intention, they're using now either the "# type: (...) -> ...", or the PEP484 conventions, depending on the version of Python you're using for your particular workspace. Also, I found the the call signature collection mechanism during debug run seems to live in the PyDev.Debugger project ( https://github.com/fabioz/PyDev.Debugger/ which is common to PyDev and PyCharm), more specifically here: https://github.com/fabioz/PyDev.Debugger/blob/master/_pydevd_bundle/pydevd_s... S. -- Stefane Fermigier - http://fermigier.com/ - http://twitter.com/sfermigier - http://linkedin.com/in/sfermigier Founder & CEO, Abilian - Enterprise Social Software - http://www.abilian.com/ Chairman, Free&OSS Group / Systematic Cluster - http://www.gt-logiciel-libre.org/ Co-Chairman, National Council for Free & Open Source Software (CNLL) - http://cnll.fr/ Founder & Organiser, PyData Paris - http://pydata.fr/ --- “You never change things by fighting the existing reality. To change something, build a new model that makes the existing model obsolete.” — R. Buckminster Fuller
On Sun, Nov 05, 2017 at 07:18:30PM +0000, Steve Barnes wrote:
"Iterators"?
That's what assert does: assert only runs when __DEBUG__ is true. That's not controlled by an environment variable, but by the -O flag to the interpreter. So your assert_params_mismatch and debug_assert_params_mismatch are effectively the same thing. But using assert to check to perform argument checks is often an abuse of assert. To be more specific, using assert to check the value of public arguments in library code (where the arguments come from outside the library) is wrong, since you (the library author) cannot guarantee that your type tests will even run. Using asserts for argument checking inside application code is more of a grey area, with arguments for and against using assert. But in my opinion, the deciding factor is nearly always that an AssertionError is the wrong sort of exception. Outside of some fairly limited circumstances, most of which don't involve type-checking function arguments, using assert robs the caller of some useful information: the *kind* of error. (TypeError, ValueError, etc.) See here for further discussion: https://import-that.dreamwidth.org/676.html In general, I don't think we want to encourage such runtime type testing. Obviously there are exceptions -- library code should probably type check arguments, applications perhaps not -- and we're not exactly discouraging it either. There are already a number of third-party libraries that provide argument type tests at runtime, and I think that's probably the right place for them. [...]
I also think that this might increase the uptake of typing by giving some clear benefits outside of documentation and static type checking.
Problem is, the benefits of runtime type checking aren't clear. But the costs certainly are: if you want slow code, do lots and lots of runtime type checks. -- Steve
On 06/11/2017 07:13, Steven D'Aprano wrote:
Good point.
I see your point here.
I'll have to look out for them.
Too much time spent writing safety critical code on my part then! I'll drop the idea. -- Steve (Gadget) Barnes Any opinions in this message are my personal opinions and do not reflect those of my employer. --- This email has been checked for viruses by AVG. http://www.avg.com
A few random remarks: 1) This seems similar to the Typeguard project https://github.com/agronholm/typeguard I personally never used it. I'd rather have some machinery that would run the checks "magically" using the type annotations, without the need for explicit decoration. 2) I fully agree that there is a important difference between using runtime checks to verify your code (for instance, as part of an automated test suite, or even when manually testing an application), and for data validation. 3) In practice, however, I have never disabled assertions in production, but I agree if can be problematic if developers confuse AssertionErrors with TypeErros. 4) 10 years ago, when I was working on the EDOS project ( http://cordis.europa.eu/pub/ist/docs/directorate_d/st-ds/edos-project-story_... ), I ran a small experiment where I used, IIRC, the profile hook to intercept all function / method calls, and log information about arguments and return value types to a gigantic log file. Then the log file could be parsed and these information used to suggest type annotations. Except there were no type annotations at the time in Python. I know PyCharm can do a similar thing now: you run your program or your tests under the debugger, it logs runtime type information somewhere, and then can use it to suggest autocompletion or maybe type annotations. Now I believe something could be done along the lines: a) record runtime type information from test or regular runs b) massage these information and use them to annotate Python code with additional type information (up to the developer to then accept or not the proposed changes) c) also run a test suite or an app under some magical machinery, and either raise a TypeError or log warnings when discrepancies are detected between type annotation and runtime behaviour. (c) could be done independently from (a) and (b), (a) and (b) would use similar machinery, and (a), (b) and (c) would be probably a useful way to introduce type annotations to an existing code base without too much risk. (a) and (b) could also provide data for an interesting SE research project. S. On Mon, Nov 6, 2017 at 8:39 AM, Steve Barnes <gadgetsteve@live.co.uk> wrote:
-- Stefane Fermigier - http://fermigier.com/ - http://twitter.com/sfermigier - http://linkedin.com/in/sfermigier Founder & CEO, Abilian - Enterprise Social Software - http://www.abilian.com/ Chairman, Free&OSS Group / Systematic Cluster - http://www.gt-logiciel-libre.org/ Co-Chairman, National Council for Free & Open Source Software (CNLL) - http://cnll.fr/ Founder & Organiser, PyData Paris - http://pydata.fr/ --- “You never change things by fighting the existing reality. To change something, build a new model that makes the existing model obsolete.” — R. Buckminster Fuller
On Mon, Nov 6, 2017 at 11:34 AM, Stéfane Fermigier <sf@fermigier.com> wrote:
[...]
I mean, (a) and (c)
Similar to "Measuring Polymorphism in Python Programs", by Beatrice Akerblom and Tobias Wrigstad: https://people.dsv.su.se/~beatrice/python/dls15_large_images.pdf S. -- Stefane Fermigier - http://fermigier.com/ - http://twitter.com/sfermigier - http://linkedin.com/in/sfermigier Founder & CEO, Abilian - Enterprise Social Software - http://www.abilian.com/ Chairman, Free&OSS Group / Systematic Cluster - http://www.gt-logiciel-libre.org/ Co-Chairman, National Council for Free & Open Source Software (CNLL) - http://cnll.fr/ Founder & Organiser, PyData Paris - http://pydata.fr/ --- “You never change things by fighting the existing reality. To change something, build a new model that makes the existing model obsolete.” — R. Buckminster Fuller
On Mon, Nov 6, 2017 at 2:34 AM, Stéfane Fermigier <sf@fermigier.com> wrote:
I didn't know this. Do you know where there are docs for this feature?
We have an early version of a tool that does this at Dropbox; I am planning to open-source it by the end of this year. So far the experience is that the annotations require a fair amount of manual cleanup though. [snip]
<https://people.dsv.su.se/~beatrice/python/dls15_large_images.pdf> Thanks for the link! -- --Guido van Rossum (python.org/~guido)
On Tue, Nov 7, 2017 at 2:58 AM, Guido van Rossum <guido@python.org> wrote:
This was described in this blog post when first introduced: https://blog.jetbrains.com/pycharm/2013/02/dynamic-runtime-type-inference-in... And more tersely, in the documentation: https://www.jetbrains.com/help/pycharm/python-debugger.html a) record runtime type information from test or regular runs
That would be great ! S. -- Stefane Fermigier - http://fermigier.com/ - http://twitter.com/sfermigier - http://linkedin.com/in/sfermigier Founder & CEO, Abilian - Enterprise Social Software - http://www.abilian.com/ Chairman, Free&OSS Group / Systematic Cluster - http://www.gt-logiciel-libre.org/ Co-Chairman, National Council for Free & Open Source Software (CNLL) - http://cnll.fr/ Founder & Organiser, PyData Paris - http://pydata.fr/ --- “You never change things by fighting the existing reality. To change something, build a new model that makes the existing model obsolete.” — R. Buckminster Fuller
On Tue, Nov 7, 2017 at 9:27 PM, Guido van Rossum <guido@python.org> wrote:
The blog post was written in 2013, PEP 484 was not yet adopted at the time, and so they targeted their own typechecker and typing annotation conventions, which were at the time based on docstrings conventions. I have just checked with the current PyCharm version, and when using the "add annotation" intention, they're using now either the "# type: (...) -> ...", or the PEP484 conventions, depending on the version of Python you're using for your particular workspace. Also, I found the the call signature collection mechanism during debug run seems to live in the PyDev.Debugger project ( https://github.com/fabioz/PyDev.Debugger/ which is common to PyDev and PyCharm), more specifically here: https://github.com/fabioz/PyDev.Debugger/blob/master/_pydevd_bundle/pydevd_s... S. -- Stefane Fermigier - http://fermigier.com/ - http://twitter.com/sfermigier - http://linkedin.com/in/sfermigier Founder & CEO, Abilian - Enterprise Social Software - http://www.abilian.com/ Chairman, Free&OSS Group / Systematic Cluster - http://www.gt-logiciel-libre.org/ Co-Chairman, National Council for Free & Open Source Software (CNLL) - http://cnll.fr/ Founder & Organiser, PyData Paris - http://pydata.fr/ --- “You never change things by fighting the existing reality. To change something, build a new model that makes the existing model obsolete.” — R. Buckminster Fuller
participants (4)
-
Guido van Rossum -
Steve Barnes -
Steven D'Aprano -
Stéfane Fermigier