[Python-ideas] Possible enhancement to typing

Stéfane Fermigier sf at fermigier.com
Mon Nov 6 05:34:28 EST 2017

A few random remarks:

1) This seems similar to the Typeguard project

I personally never used it. I'd rather have some machinery that would run
the checks "magically" using the type annotations, without the need for
explicit decoration.

2) I fully agree that there is a important difference between using runtime
checks to verify your code (for instance, as part of an automated test
suite, or even when manually testing an application), and for data

3) In practice, however, I have never disabled assertions in production,
but I agree if can be problematic if developers confuse AssertionErrors
with TypeErros.

4) 10 years ago, when I was working on the EDOS project (
), I ran a small experiment where I used, IIRC, the profile hook to
intercept all function / method calls, and log information about arguments
and return value types to a gigantic log file. Then the log file could be
parsed and these information used to suggest type annotations. Except there
were no type annotations at the time in Python.

I know PyCharm can do a similar thing now: you run your program or your
tests under the debugger, it logs runtime type information somewhere, and
then can use it to suggest autocompletion or maybe type annotations.

Now I believe something could be done along the lines:

a) record runtime type information from test or regular runs
b) massage these information and use them to annotate Python code with
additional type information (up to the developer to then accept or not the
proposed changes)
c) also run a test suite or an app under some magical machinery, and either
raise a TypeError or log warnings when discrepancies are detected between
type annotation and runtime behaviour.

(c) could be done independently from (a) and (b), (a) and (b) would use
similar machinery, and (a), (b) and (c) would be probably a useful way to
introduce type annotations to an existing code base without too much risk.

(a) and (b) could also provide data for an interesting SE research project.


On Mon, Nov 6, 2017 at 8:39 AM, Steve Barnes <gadgetsteve at live.co.uk> wrote:

> On 06/11/2017 07:13, Steven D'Aprano wrote:
> > On Sun, Nov 05, 2017 at 07:18:30PM +0000, Steve Barnes wrote:
> >
> >> If a group of iterators were to be added to the typing module it would
> >> be reasonably simple to automatically add and assert to any decorated
> >> modules to ensure that such modules were always called with the
> >> documented types.
> >
> > "Iterators"?
> >
> >
> >> I am thinking of decorators such as:
> >>
> >>    - @typing.assert_params_mismatch - this would provide a wrapper that
> >> had auto-generated asserts that all the parameters were of designated
> types.
> >>    - @typing.debug_assert_params_mismatch - this would provide a
> wrapper
> >> that had auto-generated asserts that all the parameters were of
> >> designated types only if a DEBUG environmental variable was set or
> similar.
> >
> > That's what assert does: assert only runs when __DEBUG__ is true. That's
> > not controlled by an environment variable, but by the -O flag to the
> > interpreter.
> >
> Good point.
> > So your assert_params_mismatch and debug_assert_params_mismatch are
> > effectively the same thing.
> >
> > But using assert to check to perform argument checks is often an abuse
> > of assert. To be more specific, using assert to check the value of
> > public arguments in library code (where the arguments come from outside
> the
> > library) is wrong, since you (the library author) cannot guarantee
> > that your type tests will even run.
> >
> > Using asserts for argument checking inside application code is more of a
> > grey area, with arguments for and against using assert.
> >
> > But in my opinion, the deciding factor is nearly always that an
> > AssertionError is the wrong sort of exception. Outside of some fairly
> > limited circumstances, most of which don't involve type-checking
> > function arguments, using assert robs the caller of some useful
> > information: the *kind* of error. (TypeError, ValueError, etc.)
> >
> I see your point here.
> > See here for further discussion:
> >
> > https://nam06.safelinks.protection.outlook.com/?url=
> https%3A%2F%2Fimport-that.dreamwidth.org%2F676.html&
> data=02%7C01%7Cgadgetsteve%40live.co.uk%7C5e419db490b84018f2d208d524e6
> 12b4%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%
> 7C636455492928665911&sdata=EFZIvxGxO18vk5s97RnSHH9kAuLS3s
> XBxDNS9VbLWlw%3D&reserved=0
> >
> > In general, I don't think we want to encourage such runtime type
> > testing. Obviously there are exceptions -- library code should
> > probably type check arguments, applications perhaps not -- and
> > we're not exactly discouraging it either. There are already a number of
> > third-party libraries that provide argument type tests at runtime, and
> > I think that's probably the right place for them.
> >
> I'll have to look out for them.
> >
> > [...]
> >> I also think that this might increase the uptake of typing by giving
> >> some clear benefits outside of documentation and static type checking.
> >
> > Problem is, the benefits of runtime type checking aren't clear. But the
> > costs certainly are: if you want slow code, do lots and lots of runtime
> > type checks.
> >
> >
> Too much time spent writing safety critical code on my part then! I'll
> drop the idea.
> --
> Steve (Gadget) Barnes
> Any opinions in this message are my personal opinions and do not reflect
> those of my employer.
> ---
> This email has been checked for viruses by AVG.
> http://www.avg.com
> _______________________________________________
> Python-ideas mailing list
> Python-ideas at python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/

Stefane Fermigier - http://fermigier.com/ - http://twitter.com/sfermigier -
Founder & CEO, Abilian - Enterprise Social Software -
Chairman, Free&OSS Group / Systematic Cluster -
Co-Chairman, National Council for Free & Open Source Software (CNLL) -
Founder & Organiser, PyData Paris - http://pydata.fr/
“You never change things by fighting the existing reality. To change
something, build a new model that makes the existing model obsolete.” — R.
Buckminster Fuller
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20171106/ae41aad0/attachment-0001.html>

More information about the Python-ideas mailing list