A few random remarks: 1) This seems similar to the Typeguard project https://github.com/agronholm/typeguard I personally never used it. I'd rather have some machinery that would run the checks "magically" using the type annotations, without the need for explicit decoration. 2) I fully agree that there is a important difference between using runtime checks to verify your code (for instance, as part of an automated test suite, or even when manually testing an application), and for data validation. 3) In practice, however, I have never disabled assertions in production, but I agree if can be problematic if developers confuse AssertionErrors with TypeErros. 4) 10 years ago, when I was working on the EDOS project ( http://cordis.europa.eu/pub/ist/docs/directorate_d/st-ds/edos-project-story_... ), I ran a small experiment where I used, IIRC, the profile hook to intercept all function / method calls, and log information about arguments and return value types to a gigantic log file. Then the log file could be parsed and these information used to suggest type annotations. Except there were no type annotations at the time in Python. I know PyCharm can do a similar thing now: you run your program or your tests under the debugger, it logs runtime type information somewhere, and then can use it to suggest autocompletion or maybe type annotations. Now I believe something could be done along the lines: a) record runtime type information from test or regular runs b) massage these information and use them to annotate Python code with additional type information (up to the developer to then accept or not the proposed changes) c) also run a test suite or an app under some magical machinery, and either raise a TypeError or log warnings when discrepancies are detected between type annotation and runtime behaviour. (c) could be done independently from (a) and (b), (a) and (b) would use similar machinery, and (a), (b) and (c) would be probably a useful way to introduce type annotations to an existing code base without too much risk. (a) and (b) could also provide data for an interesting SE research project. S. On Mon, Nov 6, 2017 at 8:39 AM, Steve Barnes wrote:

On 06/11/2017 07:13, Steven D'Aprano wrote:

...

On Sun, Nov 05, 2017 at 07:18:30PM +0000, Steve Barnes wrote:

...

If a group of iterators were to be added to the typing module it would be reasonably simple to automatically add and assert to any decorated modules to ensure that such modules were always called with the documented types.

"Iterators"?

...

I am thinking of decorators such as:

- @typing.assert_params_mismatch - this would provide a wrapper that had auto-generated asserts that all the parameters were of designated types. - @typing.debug_assert_params_mismatch - this would provide a wrapper that had auto-generated asserts that all the parameters were of designated types only if a DEBUG environmental variable was set or similar.

That's what assert does: assert only runs when __DEBUG__ is true. That's not controlled by an environment variable, but by the -O flag to the interpreter.

...

So your assert_params_mismatch and debug_assert_params_mismatch are effectively the same thing.

But using assert to check to perform argument checks is often an abuse of assert. To be more specific, using assert to check the value of public arguments in library code (where the arguments come from outside

Good point. the

...

library) is wrong, since you (the library author) cannot guarantee that your type tests will even run.

Using asserts for argument checking inside application code is more of a grey area, with arguments for and against using assert.

But in my opinion, the deciding factor is nearly always that an AssertionError is the wrong sort of exception. Outside of some fairly limited circumstances, most of which don't involve type-checking function arguments, using assert robs the caller of some useful information: the *kind* of error. (TypeError, ValueError, etc.)

I see your point here.

...

See here for further discussion:

https://nam06.safelinks.protection.outlook.com/?url= https%3A%2F%2Fimport-that.dreamwidth.org%2F676.html& data=02%7C01%7Cgadgetsteve%40live.co.uk%7C5e419db490b84018f2d208d524e6 12b4%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0% 7C636455492928665911&sdata=EFZIvxGxO18vk5s97RnSHH9kAuLS3s XBxDNS9VbLWlw%3D&reserved=0

In general, I don't think we want to encourage such runtime type testing. Obviously there are exceptions -- library code should probably type check arguments, applications perhaps not -- and we're not exactly discouraging it either. There are already a number of third-party libraries that provide argument type tests at runtime, and I think that's probably the right place for them.

I'll have to look out for them.

...

[...]

...

I also think that this might increase the uptake of typing by giving some clear benefits outside of documentation and static type checking.

Problem is, the benefits of runtime type checking aren't clear. But the costs certainly are: if you want slow code, do lots and lots of runtime type checks.

Too much time spent writing safety critical code on my part then! I'll drop the idea. -- Steve (Gadget) Barnes Any opinions in this message are my personal opinions and do not reflect those of my employer.

--- This email has been checked for viruses by AVG. http://www.avg.com

_______________________________________________ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/

-- Stefane Fermigier - http://fermigier.com/ - http://twitter.com/sfermigier - http://linkedin.com/in/sfermigier Founder & CEO, Abilian - Enterprise Social Software - http://www.abilian.com/ Chairman, Free&OSS Group / Systematic Cluster - http://www.gt-logiciel-libre.org/ Co-Chairman, National Council for Free & Open Source Software (CNLL) - http://cnll.fr/ Founder & Organiser, PyData Paris - http://pydata.fr/ --- “You never change things by fighting the existing reality. To change something, build a new model that makes the existing model obsolete.” — R. Buckminster Fuller

On Mon, Nov 6, 2017 at 2:34 AM, Stéfane Fermigier wrote:

4) 10 years ago, when I was working on the EDOS project ( http://cordis.europa.eu/pub/ist/docs/directorate_d/st-ds/ edos-project-story_en.pdf ), I ran a small experiment where I used, IIRC, the profile hook to intercept all function / method calls, and log information about arguments and return value types to a gigantic log file. Then the log file could be parsed and these information used to suggest type annotations. Except there were no type annotations at the time in Python.

I know PyCharm can do a similar thing now: you run your program or your tests under the debugger, it logs runtime type information somewhere, and then can use it to suggest autocompletion or maybe type annotations.

I didn't know this. Do you know where there are docs for this feature?

Now I believe something could be done along the lines:

a) record runtime type information from test or regular runs b) massage these information and use them to annotate Python code with additional type information (up to the developer to then accept or not the proposed changes)

We have an early version of a tool that does this at Dropbox; I am planning to open-source it by the end of this year. So far the experience is that the annotations require a fair amount of manual cleanup though. [snip]

Similar to "Measuring Polymorphism in Python Programs", by Beatrice Akerblom and Tobias Wrigstad: https://people.dsv.su.se/~beatrice/python/dls15_large_images.pdf https://people.dsv.su.se/~beatrice/python/dls15_large_images.pdf

https://people.dsv.su.se/~beatrice/python/dls15_large_images.pdf Thanks for the link! -- --Guido van Rossum (python.org/~guido)

On Mon, Nov 6, 2017 at 10:36 PM, Stéfane Fermigier wrote:

On Tue, Nov 7, 2017 at 2:58 AM, Guido van Rossum wrote:

...

On Mon, Nov 6, 2017 at 2:34 AM, Stéfane Fermigier wrote:

...

4) 10 years ago, when I was working on the EDOS project ( http://cordis.europa.eu/pub/ist/docs/directorate_d/st-ds/edo s-project-story_en.pdf ), I ran a small experiment where I used, IIRC, the profile hook to intercept all function / method calls, and log information about arguments and return value types to a gigantic log file. Then the log file could be parsed and these information used to suggest type annotations. Except there were no type annotations at the time in Python.

I know PyCharm can do a similar thing now: you run your program or your tests under the debugger, it logs runtime type information somewhere, and then can use it to suggest autocompletion or maybe type annotations.

I didn't know this. Do you know where there are docs for this feature?

This was described in this blog post when first introduced:

https://blog.jetbrains.com/pycharm/2013/02/dynamic- runtime-type-inference-in-pycharm-2-7/

And more tersely, in the documentation: https://www. jetbrains.com/help/pycharm/python-debugger.html

Oh. It sounds like it doesn't generate stubs or PEP 484 annotations. -- --Guido van Rossum (python.org/~guido)