Typing syntax and ecosystem, take 2
In any Python 3.6 or later, type >>> x : float = 1 >>> isinstance(x, float) or replace the second line with >>> type(x) As someone who has programmed in FORTRAN, Pascal, C/C++, Java, and Go this is not at all what I consider reasonable. I do not believe other programmers with experience in typed languages would expect this behaviour either. Having a type checker run before the Python interpreter in our current day continuous build/integration environment adds a second step and therefore the chance for it to be forgotten, for version mismatches, for warning/error reports to be misdirected. From an environmental point of view we're also doing the read source, lexical scan, syntax parse twice rather than once. If typing is the future of Python, and the number of PEPs being worked on suggests that it is, then type checking should be integrated into CPython itself. An alternative is the TypeScript/JavaScript model, where typed Python becomes a distinct programming language that cannot be executed directly by the Python interpreter. -- cheers, Hugh Fisher
On Tue, 2021-04-13 at 09:43 +1000, Hugh Fisher wrote:
Python is not a typed language, it is a language with optional typing.
This is simply not a good point in itself, and on top of that it seems to completely ignore that Python could be untyped, which is something I am almost certain won't likely change ever. At the very least you would have to pass a flag to the interpreter to run the type checker, which completely invalidates your "people will forget it if they have to explicitly run it" point.
Why? There is a type checker living under the same umbrella organization. What is the reasoning for the type checker to be included with the interpreter, especially given that it is entirely optional? All I can think of are reasons *not to* include it with the interpreter, mainly that it would be bound to the same super slow release cycle. Please also consider that a type checker is something that no user will have to run, only developers.
Why?? This reads like "let's break stuff that is absolutely working" for no real discerning reason. Why would you do this? -- Why would we break workflows that work? Why would we add a code generating step to typed Python code? I am really struggling here to understand what would be the benefit here, it seems you want to make it slightly harder for people to forget to run it, by either making it always run or forcing a new code generation step. Overall, I find that you are trying to apply concepts from other languages and environments to Python, without really taking into account what they would mean for the Python ecosystem. While it is a great idea to look at other languages and try to align ourselves with some of their designs, as a way to make the user experience easier for some, it is also okay to not design things in the same way. We need to take into account the requirements and current state of our own environment, and try to understand what is the best option for us. Benefits and drawbacks will vary, so it is absolutely valid to come up with different answers. Please do not take these comments personally, in a verbal setting I would try to use social cues to try to pass the feedback more lightly. This sort of communication medium can make things easily pass as harsh, but that is not my goal. I just want to make sure we are on the same page, if you have answers for any of my questions, please let me know, but try to make sure you are taking into account you what real impact your proposals would have on the Python ecosystem. Cheers, Filipe Laíns
On Tue, 13 Apr 2021 03:09:28 +0100 Filipe Laíns <lains@archlinux.org> wrote:
This is simply not a good point in itself, and on top of that it seems to completely ignore that Python could be untyped,
Python is definitely not untyped. It's strongly typed. Its typing is just dynamic and isn't explicitly spelled in most syntax. Regards Antoine.
On Tue, 13 Apr 2021 at 12:09, Filipe Laíns <lains@archlinux.org> wrote:
[ munch ]
Python is not a typed language, it is a language with optional typing.
Well, it's a dynamically typed language. And yes I agree, I want typing to remain optional. I am happy that I can write x = 1 x = [ "hello", "world" ] x = ExtremelyComplicatedObject() without even having to qualify it as x : object
This is my second (and somewhat preferred) proposal where type hints are recognised by the CPython interpreter, but they would remain optional. So no flag needed. If there aren't any type hints, that's fine. If there are type hints, assume they are there on purpose and parse and check.
Getting slightly off topic, I have always thought that one of the strengths of Python is that every use is a potential developer. If you have a Python program and a text editor, you can change it. No special tools required.
This is the model used by TypeScript (and various other dialects of JavaScript) and it works for them. The big advantage is that the Python interpreter does not need to know about typing, so no PEP 543 and no PEP 649, which is currently generating a lot of esoteric (to me) discussion on python-dev.
Please do not take these comments personally, in a verbal setting I would try to use social cues to try to pass the feedback more lightly.
No complain from me Filipe, you're doing better at this than I am. -- cheers, Hugh Fisher
On Tue, Apr 13, 2021 at 1:46 AM Hugh Fisher <hugo.fisher@gmail.com> wrote:
I have 30 years experience in other programming languages (C, C++, Pascal, Java...) and I'm OK with this behaviour :) Python is, historically, a dynamically typed language, and gradual typing (see https://en.wikipedia.org/wiki/Gradual_typing) has been progressively added to it in the last decade or so. This is a legitimate approach, both from an academic and industrial point of view, and similar (with some variants) to what has been done also for JavaScript, Ruby, PHP, Lua, Perl, Scheme, etc. over the same period. See for instance https://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.683.8854&rep=rep1&type=pdf for more references.
No need to integrate in CPython, one can already provide runtime type checking using the current language, see e.g. https://pypi.org/project/typeguard/
There are a few compile-to-Python languages out there (See a list here: https://github.com/sfermigier/awesome-functional-python#languages ), some with desirable (for some people at least) characteristics (homoiconic syntax, language-level support for actors and functional programming, pattern matching, etc.). They have very limited success in terms of adoption which is probably a lesson to keep in mind. S. -- Stefane Fermigier - http://fermigier.com/ - http://twitter.com/sfermigier - http://linkedin.com/in/sfermigier Founder & CEO, Abilian - Enterprise Social Software - http://www.abilian.com/ Chairman, National Council for Free & Open Source Software (CNLL) - http://cnll.fr/ Founder & Organiser, PyParis & PyData Paris - http://pyparis.org/ & http://pydata.fr/
On Tue, 13 Apr 2021 at 18:43, Stéfane Fermigier <sf@fermigier.com> wrote:
[ munch ]
Python is, historically, a dynamically typed language, and gradual typing (see https://en.wikipedia.org/wiki/Gradual_typing) has been progressively added to it in the last decade or so. This is a legitimate approach, both from an academic and industrial point of view, and similar (with some variants) to what has been done also for JavaScript, Ruby, PHP, Lua, Perl, Scheme, etc. over the same period.
I'm aware of the difference between untyped and dynamic typing, and want to emphasise that I *like* the dynamic typing in Python. My complaint is not with gradual typing, but the current way it is done. [ munch ]
No need to integrate in CPython, one can already provide runtime type checking using the current language, see e.g. https://pypi.org/project/typeguard/
The current runtime type checking approach is causing significant performance problems, hence PEP 563, PEP 649, and the current discussion on python-dev about how to reduce them. In the TypeScript approach, "typed Python" is a distinct programming language that is translated into dynamic Python, those problems go away because the type notation would not be valid Python code. In the integrated approach, type checking built in, there would still be overhead but it would be *useful* overhead doing the actual type checking. -- cheers, Hugh Fisher
Hugh Fisher writes:
From the point of view of typing, you've programmed in one other language. ;-) (Maybe Go makes two, I haven't used it.) They all demand that variables be declared with types, and then proceed to fill them with bit patterns that can't be introspected (unless you build it in to class definitions yourself). Python's different approach is a compromise. And as usual Guido's time machine (or somebody borrowed it) is in evidence: if that example bothers you, request that people around you who type things like that use stub files instead so you don't have to deal with it. If, on the other hand, you want explicitly typed variables (and not just "if we must have types, I want them integrated", I think you'll have a serious fight on your hands. It's often quite useful to me, for one, that names are not typed while objects are.
This is a good point, but having an explicitly typed language would mean I can't really use Python at work (that is, I can't recommend it to my business and economics students, who don't program, really, they record scripts).
From an environmental point of view we're also doing the read source, lexical scan, syntax parse twice rather than once.
This is not true. Most programs, *especially* programs that are complex enough to want type-checking, are run far more often than they are type-checked.
I don't see it as *the* future of Python. And I don't really see why it should be integrated into CPython. To me it seems to be a language facility, not an implementation facility. Integrating it with CPython would likely result in disintegrating it from PyPy and other implementations. I don't think that is desirable at all -- the version skew you mentioned would likely occur here. I like to think of it as "Python typing has some warts, but that's because it will turn out to be a prince(ss) when you kiss it." Steve
On Tue, 13 Apr 2021 at 20:30, Stephen J. Turnbull <turnbull.stephen.fw@u.tsukuba.ac.jp> wrote: [ munch ]
There are a lot of programmers like me. Those languages I listed are widely used, and therefore we assume that if a Python language construct looks like something we've used before, it will work in the same way.
And that would be the TypeScript style approach, "typed Python" is a language that gets translated into non-type hinted dynamic Python. No-one who works in JavaScript needs to deal with TypeScript language features unless they want to. This is not true of the current typing in Python. [ munch ]
I don't want Python to be explicitly typed either. I'm happy with dynamic typing, and do not want to have to write even x : object
Among the software devs I work with, testing and checking is something you do *every* time you make a change. If I'm adding type checks to my program, that's because I want them to be checked. Historical example would be lint for C programs. Running lint wasn't required before compiling your C code, but it was so useful that a number of projects made it compulsory, and the functionality of lint was eventually incorporated into the C compiler itself. [ munch ]
I like to think of it as "Python typing has some warts, but that's because it will turn out to be a prince(ss) when you kiss it."
Yes, perfect is the enemy of good enough. I'm reading (OK, skimming) the emails about implementing PEP 649 on this list and I'm seeing significant problems in the current tools, not just cosmetic. -- cheers, Hugh Fisher
On Wed, Apr 14, 2021 at 9:45 AM Hugh Fisher <hugo.fisher@gmail.com> wrote:
You don't. That's not the proposal. The proposals have ALL been about gradual typing and inferred typing, so you don't have to annotate anything that doesn't need it. Annotating function parameters/return values is about the extent that I'd ever do for most programs, and even then, not all functions. ChrisA
Hugh Fisher writes:
That's not very persuasive. When in London, England, you speak English, but you probably shouldn't assume that the words others use mean the same thing that they do in London, Ohio. Anyway, in which of those languages is "def foo(x:float = 1) -> int" legal syntax? What do you mean it "looks like" something you've used before? It looks to me like what you mean is "To me, declared types (no matter what they look like) aren't hints, types should be enforced." I agree, that's what you learn if your experience is restricted to common explicitly typed languages, but reality is more flexible than that. That's why I read manuals before using a new language or a feature newly added to a familiar language. :-)
I can't agree. To me it's the same Python with typed objects and untyped identifiers it's always been, and that's a feature. The question is whether you see the hints in the implementation source, or in an auxiliary file. "Typed Python" as currently implemented is not a different language. It's the same language with extensive use of a previously little-used feature (function annotations).
You *need* to deal with typing only to the extent of ignoring it. OK, one of the real selling points of Python is that its syntax is fairly economical, and type hints do have a cost in that sense. But the answer to that is "just don't", and go ahead and ask colleagues you share code with to "just don't" too.
And that's the last time anybody ever runs that program? That's certainly not true of any of the programs I would consider using type hinting for! Even the test suite will typically run mypy once, and the program itself many times since that's the easiest way to guarantee de novo initialization of the program's state for each test.
I think the analogy is valid, but it doesn't support your position. See Brett's reply for why, I'm just +1-ing here. Steve
On 4/12/21 7:43 PM, Hugh Fisher wrote:
This is very similar to statically typed languages. They also have two steps: * There is the first step that checks the types but does not run the program. In C/C++, this is the compiler, in Python it is "mypy". * There is the second step that runs the program but does not check the types. In C/C++, this is running the executable, in Python it is "python". In Python, we can skip the first step, and perhaps that is the source of your concern. I doubt anyone that has adopted type hinting in Python will forget to run mypy, especially in a CI environment like you consider. --Ned.
On Tue, 2021-04-13 at 09:43 +1000, Hugh Fisher wrote:
Python is not a typed language, it is a language with optional typing.
This is simply not a good point in itself, and on top of that it seems to completely ignore that Python could be untyped, which is something I am almost certain won't likely change ever. At the very least you would have to pass a flag to the interpreter to run the type checker, which completely invalidates your "people will forget it if they have to explicitly run it" point.
Why? There is a type checker living under the same umbrella organization. What is the reasoning for the type checker to be included with the interpreter, especially given that it is entirely optional? All I can think of are reasons *not to* include it with the interpreter, mainly that it would be bound to the same super slow release cycle. Please also consider that a type checker is something that no user will have to run, only developers.
Why?? This reads like "let's break stuff that is absolutely working" for no real discerning reason. Why would you do this? -- Why would we break workflows that work? Why would we add a code generating step to typed Python code? I am really struggling here to understand what would be the benefit here, it seems you want to make it slightly harder for people to forget to run it, by either making it always run or forcing a new code generation step. Overall, I find that you are trying to apply concepts from other languages and environments to Python, without really taking into account what they would mean for the Python ecosystem. While it is a great idea to look at other languages and try to align ourselves with some of their designs, as a way to make the user experience easier for some, it is also okay to not design things in the same way. We need to take into account the requirements and current state of our own environment, and try to understand what is the best option for us. Benefits and drawbacks will vary, so it is absolutely valid to come up with different answers. Please do not take these comments personally, in a verbal setting I would try to use social cues to try to pass the feedback more lightly. This sort of communication medium can make things easily pass as harsh, but that is not my goal. I just want to make sure we are on the same page, if you have answers for any of my questions, please let me know, but try to make sure you are taking into account you what real impact your proposals would have on the Python ecosystem. Cheers, Filipe Laíns
On Tue, 13 Apr 2021 03:09:28 +0100 Filipe Laíns <lains@archlinux.org> wrote:
This is simply not a good point in itself, and on top of that it seems to completely ignore that Python could be untyped,
Python is definitely not untyped. It's strongly typed. Its typing is just dynamic and isn't explicitly spelled in most syntax. Regards Antoine.
On Tue, 13 Apr 2021 at 12:09, Filipe Laíns <lains@archlinux.org> wrote:
[ munch ]
Python is not a typed language, it is a language with optional typing.
Well, it's a dynamically typed language. And yes I agree, I want typing to remain optional. I am happy that I can write x = 1 x = [ "hello", "world" ] x = ExtremelyComplicatedObject() without even having to qualify it as x : object
This is my second (and somewhat preferred) proposal where type hints are recognised by the CPython interpreter, but they would remain optional. So no flag needed. If there aren't any type hints, that's fine. If there are type hints, assume they are there on purpose and parse and check.
Getting slightly off topic, I have always thought that one of the strengths of Python is that every use is a potential developer. If you have a Python program and a text editor, you can change it. No special tools required.
This is the model used by TypeScript (and various other dialects of JavaScript) and it works for them. The big advantage is that the Python interpreter does not need to know about typing, so no PEP 543 and no PEP 649, which is currently generating a lot of esoteric (to me) discussion on python-dev.
Please do not take these comments personally, in a verbal setting I would try to use social cues to try to pass the feedback more lightly.
No complain from me Filipe, you're doing better at this than I am. -- cheers, Hugh Fisher
On Tue, Apr 13, 2021 at 1:46 AM Hugh Fisher <hugo.fisher@gmail.com> wrote:
I have 30 years experience in other programming languages (C, C++, Pascal, Java...) and I'm OK with this behaviour :) Python is, historically, a dynamically typed language, and gradual typing (see https://en.wikipedia.org/wiki/Gradual_typing) has been progressively added to it in the last decade or so. This is a legitimate approach, both from an academic and industrial point of view, and similar (with some variants) to what has been done also for JavaScript, Ruby, PHP, Lua, Perl, Scheme, etc. over the same period. See for instance https://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.683.8854&rep=rep1&type=pdf for more references.
No need to integrate in CPython, one can already provide runtime type checking using the current language, see e.g. https://pypi.org/project/typeguard/
There are a few compile-to-Python languages out there (See a list here: https://github.com/sfermigier/awesome-functional-python#languages ), some with desirable (for some people at least) characteristics (homoiconic syntax, language-level support for actors and functional programming, pattern matching, etc.). They have very limited success in terms of adoption which is probably a lesson to keep in mind. S. -- Stefane Fermigier - http://fermigier.com/ - http://twitter.com/sfermigier - http://linkedin.com/in/sfermigier Founder & CEO, Abilian - Enterprise Social Software - http://www.abilian.com/ Chairman, National Council for Free & Open Source Software (CNLL) - http://cnll.fr/ Founder & Organiser, PyParis & PyData Paris - http://pyparis.org/ & http://pydata.fr/
On Tue, 13 Apr 2021 at 18:43, Stéfane Fermigier <sf@fermigier.com> wrote:
[ munch ]
Python is, historically, a dynamically typed language, and gradual typing (see https://en.wikipedia.org/wiki/Gradual_typing) has been progressively added to it in the last decade or so. This is a legitimate approach, both from an academic and industrial point of view, and similar (with some variants) to what has been done also for JavaScript, Ruby, PHP, Lua, Perl, Scheme, etc. over the same period.
I'm aware of the difference between untyped and dynamic typing, and want to emphasise that I *like* the dynamic typing in Python. My complaint is not with gradual typing, but the current way it is done. [ munch ]
No need to integrate in CPython, one can already provide runtime type checking using the current language, see e.g. https://pypi.org/project/typeguard/
The current runtime type checking approach is causing significant performance problems, hence PEP 563, PEP 649, and the current discussion on python-dev about how to reduce them. In the TypeScript approach, "typed Python" is a distinct programming language that is translated into dynamic Python, those problems go away because the type notation would not be valid Python code. In the integrated approach, type checking built in, there would still be overhead but it would be *useful* overhead doing the actual type checking. -- cheers, Hugh Fisher
Hugh Fisher writes:
From the point of view of typing, you've programmed in one other language. ;-) (Maybe Go makes two, I haven't used it.) They all demand that variables be declared with types, and then proceed to fill them with bit patterns that can't be introspected (unless you build it in to class definitions yourself). Python's different approach is a compromise. And as usual Guido's time machine (or somebody borrowed it) is in evidence: if that example bothers you, request that people around you who type things like that use stub files instead so you don't have to deal with it. If, on the other hand, you want explicitly typed variables (and not just "if we must have types, I want them integrated", I think you'll have a serious fight on your hands. It's often quite useful to me, for one, that names are not typed while objects are.
This is a good point, but having an explicitly typed language would mean I can't really use Python at work (that is, I can't recommend it to my business and economics students, who don't program, really, they record scripts).
From an environmental point of view we're also doing the read source, lexical scan, syntax parse twice rather than once.
This is not true. Most programs, *especially* programs that are complex enough to want type-checking, are run far more often than they are type-checked.
I don't see it as *the* future of Python. And I don't really see why it should be integrated into CPython. To me it seems to be a language facility, not an implementation facility. Integrating it with CPython would likely result in disintegrating it from PyPy and other implementations. I don't think that is desirable at all -- the version skew you mentioned would likely occur here. I like to think of it as "Python typing has some warts, but that's because it will turn out to be a prince(ss) when you kiss it." Steve
On Tue, 13 Apr 2021 at 20:30, Stephen J. Turnbull <turnbull.stephen.fw@u.tsukuba.ac.jp> wrote: [ munch ]
There are a lot of programmers like me. Those languages I listed are widely used, and therefore we assume that if a Python language construct looks like something we've used before, it will work in the same way.
And that would be the TypeScript style approach, "typed Python" is a language that gets translated into non-type hinted dynamic Python. No-one who works in JavaScript needs to deal with TypeScript language features unless they want to. This is not true of the current typing in Python. [ munch ]
I don't want Python to be explicitly typed either. I'm happy with dynamic typing, and do not want to have to write even x : object
Among the software devs I work with, testing and checking is something you do *every* time you make a change. If I'm adding type checks to my program, that's because I want them to be checked. Historical example would be lint for C programs. Running lint wasn't required before compiling your C code, but it was so useful that a number of projects made it compulsory, and the functionality of lint was eventually incorporated into the C compiler itself. [ munch ]
I like to think of it as "Python typing has some warts, but that's because it will turn out to be a prince(ss) when you kiss it."
Yes, perfect is the enemy of good enough. I'm reading (OK, skimming) the emails about implementing PEP 649 on this list and I'm seeing significant problems in the current tools, not just cosmetic. -- cheers, Hugh Fisher
On Wed, Apr 14, 2021 at 9:45 AM Hugh Fisher <hugo.fisher@gmail.com> wrote:
You don't. That's not the proposal. The proposals have ALL been about gradual typing and inferred typing, so you don't have to annotate anything that doesn't need it. Annotating function parameters/return values is about the extent that I'd ever do for most programs, and even then, not all functions. ChrisA
Hugh Fisher writes:
That's not very persuasive. When in London, England, you speak English, but you probably shouldn't assume that the words others use mean the same thing that they do in London, Ohio. Anyway, in which of those languages is "def foo(x:float = 1) -> int" legal syntax? What do you mean it "looks like" something you've used before? It looks to me like what you mean is "To me, declared types (no matter what they look like) aren't hints, types should be enforced." I agree, that's what you learn if your experience is restricted to common explicitly typed languages, but reality is more flexible than that. That's why I read manuals before using a new language or a feature newly added to a familiar language. :-)
I can't agree. To me it's the same Python with typed objects and untyped identifiers it's always been, and that's a feature. The question is whether you see the hints in the implementation source, or in an auxiliary file. "Typed Python" as currently implemented is not a different language. It's the same language with extensive use of a previously little-used feature (function annotations).
You *need* to deal with typing only to the extent of ignoring it. OK, one of the real selling points of Python is that its syntax is fairly economical, and type hints do have a cost in that sense. But the answer to that is "just don't", and go ahead and ask colleagues you share code with to "just don't" too.
And that's the last time anybody ever runs that program? That's certainly not true of any of the programs I would consider using type hinting for! Even the test suite will typically run mypy once, and the program itself many times since that's the easiest way to guarantee de novo initialization of the program's state for each test.
I think the analogy is valid, but it doesn't support your position. See Brett's reply for why, I'm just +1-ing here. Steve
On 4/12/21 7:43 PM, Hugh Fisher wrote:
This is very similar to statically typed languages. They also have two steps: * There is the first step that checks the types but does not run the program. In C/C++, this is the compiler, in Python it is "mypy". * There is the second step that runs the program but does not check the types. In C/C++, this is running the executable, in Python it is "python". In Python, we can skip the first step, and perhaps that is the source of your concern. I doubt anyone that has adopted type hinting in Python will forget to run mypy, especially in a CI environment like you consider. --Ned.
participants (7)
-
Antoine Pitrou -
Chris Angelico -
Filipe Laíns -
Hugh Fisher -
Ned Batchelder -
Stephen J. Turnbull -
Stéfane Fermigier