Re: Should Python enforce Type-checking in the future?
On Fri, Dec 10, 2021 at 08:32:27AM -0800, Christopher Barker wrote:
That seems to be close to the opinion of Robert C Martin: http://blog.cleancoder.com/uncle-bob/2016/05/01/TypeWars.html He also has some comments on languages like Koitlin and Swift that have gone down that path of mandatory static typing: http://blog.cleancoder.com/uncle-bob/2017/01/11/TheDarkPath.html -- Steve
Steven D'Aprano wrote:
Yet another example of "Uncle Bob" writing stuff that's "not even wrong". Of course, typing doesn't catch all bugs; but neither does Uncle Bob's testing-testing-testing. And never will. Programmers need all the help they can in catching bugs and also in understanding other people's code - and types are really helpful for that ... I can't count the number of hours I've spent at Google, figuring out how something works so that I can add some functionality - often by adding stuff like logging.info("Type of qqsv: %s", qqsv.__class_). (On Mondays, Wednesdays, and Fridays, I think that mandatory typing would be fabulous (to paraphrase Brian Reid: "most people won't use types even when you threaten them"). On Tuesdays, Thursdays, and Saturdays, I meditate on how difficult it is to write types in Haskell, which is far easier for typing than Python. On Sundays, I go for a head-clearing walk in the park.) PS: For another example of "Uncle Bob" nonsense: https://groups.google.com/g/software-design-book/c/Kb5K3YcjIXw/m/qN8txMeOCAA...
Well, this has a lot of strong opinions already! I didn't expect less :-) I'm not sure how to answer so many thoughts without making this super-long. I'll try to reply in general. I agree that a compile-time python makes no sense, it would be a different language and we would lose 90% of what Python actually is. This is not what I was suggesting (sorry, I might not have been clear on my first email). I'm talking about something similar to what Typescript is. (mypy even when strict, is closer to what Typescript does than to C++/Rust) But there must be a way to skip typing altogether, both in programs and libraries. (It could also be by file extension... just suggesting more stuff here) When I said earlier in "compile time", I was just trying to refer to the parsing step, or pre-parsing step. Or maybe means that "python a.py" => "mypy a.py && python a.py". I'm being very ambiguous here on purpose because there are tons of ways to do this and "how to do it" or the cons of a particular method aren't something that I wanted to discuss, I just wanted to get a general idea of where the community stands. A lot of you are referring to scripting, learning, and other ways of using Python that would be badly impacted. I already acknowledged that these exist (or at least I didn't notice here any other that I wasn't aware), and that's why I mentioned that there should be a way to avoid type-enforcing for these scenarios. Maybe by flag, by Python version, file extension... The method doesn't matter much at this moment. If it were "easy" to continue writing the old programs without types, what's the problem in trying to make types more popular? And yes, types cannot cover everything. I guess this goes for any language, but especially in Python it is really impossible to type everything. But as Typescript has demonstrated (I think), having a bit of types is better than no types at all. MyPy is there for those that want to use it, of course; but needs to be promoted (this one or different tool) and encouraged. Otherwise most python libraries are just impossible to use with MyPy; and using Python without libraries is quite limiting.
This is not true unless your testing covers every branch in every possible state input. This also means that your unit testing in dynamic languages needs to be way more extensive than in static languages; because static checking gives you certain proofs of correctness that you don't need to test. More importantly, it removes the burden of remembering to do those test cases for those tricky scenarios. Overall, your tests will be shorter and easier to maintain. But I would say that even with extensive testing still will not cover all bugs that static type checks can catch: static type checks also can cover future bugs, not only present ones. A type check will prevent someone from calling a function/method in unexpected ways. Because type checks serve to define contracts, and 6 months later you might have forgot that the function assumed certain constraints for the input.
There you have it. I'm amazed that this is already a thing and the discussion of "enforcing types" didn't happen up to now. If MyPyC can compile to C and get a 4x performance boost, it means that Python is leaving a 4x boost on the table. When creating the bytecode (old *.pyc files) Python could create for some functions a second one that internally doesn't use Python objects, for those cases where the types actually match. And the GIL could be released (sometimes) during the execution of those. (It might look that the above statement is far too simplistic. But it should be roughly true with lots of caveats.) Anyway, I see that people are generally against what I'm suggesting. But on the other hand I also see that there are projects that are working towards faster python implementations via type-checking. I now noticed that PyPI includes filters for typed packages; I didn't recall this from 3 years ago. I wish they promoted typed ones a bit more, but well, at least we can filter. El vie, 10 dic 2021 a las 17:49, Peter Ludemann (<peter.ludemann@gmail.com>) escribió:
On Sat, Dec 11, 2021 at 7:44 AM deavid <deavidsedice@gmail.com> wrote:
When I said earlier in "compile time", I was just trying to refer to the parsing step, or pre-parsing step. Or maybe means that "python a.py" => "mypy a.py && python a.py". I'm being very ambiguous here on purpose because there are tons of ways to do this and "how to do it" or the cons of a particular method aren't something that I wanted to discuss, I just wanted to get a general idea of where the community stands.
You could make a shell alias that does that, if that's what you want.
A lot of you are referring to scripting, learning, and other ways of using Python that would be badly impacted. I already acknowledged that these exist (or at least I didn't notice here any other that I wasn't aware), and that's why I mentioned that there should be a way to avoid type-enforcing for these scenarios. Maybe by flag, by Python version, file extension... The method doesn't matter much at this moment. If it were "easy" to continue writing the old programs without types, what's the problem in trying to make types more popular?
Or by simply running Python without first running MyPy. That's a method.
From the MyPyC docs: """These are our near-term focus areas for improving mypyc: * Improved compatibility with Python""" The 4x speed boost is only if you restrict yourself to the things it can handle. That's fine if a project specifically chooses to do that, but not for general work. And it is definitely NOT a reason to push type checking into the core language. Side question: What if someone wants to use a different type checker than MyPy? Why should that one single checker be blessed with language support? Type annotations are intended to be there for *every* type checker to use. ChrisA
On 2021-12-10 at 20:20:05 +0000, deavid <deavidsedice@gmail.com> wrote:
I concede that certain programmers believe that type declarations and static type checking are unilaterally and universally beneficial, and that's why I think that there should be a way to request type-enforcing for these scenarios. Maybe by flag, by Python version, file extension... The method doesn't matter, because in all my decades of writing software, in a dozen or more languages, getting the types right at coding time has never been an issue. It's very easy to get type checking if you want it, what's the problem in trying to make types more popular? [insert your own perfect-joking-serious-balance emoticon (no emoji, please!) here]
Steven D'Aprano wrote:
Yet another example of "Uncle Bob" writing stuff that's "not even wrong". Of course, typing doesn't catch all bugs; but neither does Uncle Bob's testing-testing-testing. And never will. Programmers need all the help they can in catching bugs and also in understanding other people's code - and types are really helpful for that ... I can't count the number of hours I've spent at Google, figuring out how something works so that I can add some functionality - often by adding stuff like logging.info("Type of qqsv: %s", qqsv.__class_). (On Mondays, Wednesdays, and Fridays, I think that mandatory typing would be fabulous (to paraphrase Brian Reid: "most people won't use types even when you threaten them"). On Tuesdays, Thursdays, and Saturdays, I meditate on how difficult it is to write types in Haskell, which is far easier for typing than Python. On Sundays, I go for a head-clearing walk in the park.) PS: For another example of "Uncle Bob" nonsense: https://groups.google.com/g/software-design-book/c/Kb5K3YcjIXw/m/qN8txMeOCAA...
Well, this has a lot of strong opinions already! I didn't expect less :-) I'm not sure how to answer so many thoughts without making this super-long. I'll try to reply in general. I agree that a compile-time python makes no sense, it would be a different language and we would lose 90% of what Python actually is. This is not what I was suggesting (sorry, I might not have been clear on my first email). I'm talking about something similar to what Typescript is. (mypy even when strict, is closer to what Typescript does than to C++/Rust) But there must be a way to skip typing altogether, both in programs and libraries. (It could also be by file extension... just suggesting more stuff here) When I said earlier in "compile time", I was just trying to refer to the parsing step, or pre-parsing step. Or maybe means that "python a.py" => "mypy a.py && python a.py". I'm being very ambiguous here on purpose because there are tons of ways to do this and "how to do it" or the cons of a particular method aren't something that I wanted to discuss, I just wanted to get a general idea of where the community stands. A lot of you are referring to scripting, learning, and other ways of using Python that would be badly impacted. I already acknowledged that these exist (or at least I didn't notice here any other that I wasn't aware), and that's why I mentioned that there should be a way to avoid type-enforcing for these scenarios. Maybe by flag, by Python version, file extension... The method doesn't matter much at this moment. If it were "easy" to continue writing the old programs without types, what's the problem in trying to make types more popular? And yes, types cannot cover everything. I guess this goes for any language, but especially in Python it is really impossible to type everything. But as Typescript has demonstrated (I think), having a bit of types is better than no types at all. MyPy is there for those that want to use it, of course; but needs to be promoted (this one or different tool) and encouraged. Otherwise most python libraries are just impossible to use with MyPy; and using Python without libraries is quite limiting.
This is not true unless your testing covers every branch in every possible state input. This also means that your unit testing in dynamic languages needs to be way more extensive than in static languages; because static checking gives you certain proofs of correctness that you don't need to test. More importantly, it removes the burden of remembering to do those test cases for those tricky scenarios. Overall, your tests will be shorter and easier to maintain. But I would say that even with extensive testing still will not cover all bugs that static type checks can catch: static type checks also can cover future bugs, not only present ones. A type check will prevent someone from calling a function/method in unexpected ways. Because type checks serve to define contracts, and 6 months later you might have forgot that the function assumed certain constraints for the input.
There you have it. I'm amazed that this is already a thing and the discussion of "enforcing types" didn't happen up to now. If MyPyC can compile to C and get a 4x performance boost, it means that Python is leaving a 4x boost on the table. When creating the bytecode (old *.pyc files) Python could create for some functions a second one that internally doesn't use Python objects, for those cases where the types actually match. And the GIL could be released (sometimes) during the execution of those. (It might look that the above statement is far too simplistic. But it should be roughly true with lots of caveats.) Anyway, I see that people are generally against what I'm suggesting. But on the other hand I also see that there are projects that are working towards faster python implementations via type-checking. I now noticed that PyPI includes filters for typed packages; I didn't recall this from 3 years ago. I wish they promoted typed ones a bit more, but well, at least we can filter. El vie, 10 dic 2021 a las 17:49, Peter Ludemann (<peter.ludemann@gmail.com>) escribió:
On Sat, Dec 11, 2021 at 7:44 AM deavid <deavidsedice@gmail.com> wrote:
When I said earlier in "compile time", I was just trying to refer to the parsing step, or pre-parsing step. Or maybe means that "python a.py" => "mypy a.py && python a.py". I'm being very ambiguous here on purpose because there are tons of ways to do this and "how to do it" or the cons of a particular method aren't something that I wanted to discuss, I just wanted to get a general idea of where the community stands.
You could make a shell alias that does that, if that's what you want.
A lot of you are referring to scripting, learning, and other ways of using Python that would be badly impacted. I already acknowledged that these exist (or at least I didn't notice here any other that I wasn't aware), and that's why I mentioned that there should be a way to avoid type-enforcing for these scenarios. Maybe by flag, by Python version, file extension... The method doesn't matter much at this moment. If it were "easy" to continue writing the old programs without types, what's the problem in trying to make types more popular?
Or by simply running Python without first running MyPy. That's a method.
From the MyPyC docs: """These are our near-term focus areas for improving mypyc: * Improved compatibility with Python""" The 4x speed boost is only if you restrict yourself to the things it can handle. That's fine if a project specifically chooses to do that, but not for general work. And it is definitely NOT a reason to push type checking into the core language. Side question: What if someone wants to use a different type checker than MyPy? Why should that one single checker be blessed with language support? Type annotations are intended to be there for *every* type checker to use. ChrisA
On 2021-12-10 at 20:20:05 +0000, deavid <deavidsedice@gmail.com> wrote:
I concede that certain programmers believe that type declarations and static type checking are unilaterally and universally beneficial, and that's why I think that there should be a way to request type-enforcing for these scenarios. Maybe by flag, by Python version, file extension... The method doesn't matter, because in all my decades of writing software, in a dozen or more languages, getting the types right at coding time has never been an issue. It's very easy to get type checking if you want it, what's the problem in trying to make types more popular? [insert your own perfect-joking-serious-balance emoticon (no emoji, please!) here]
participants (5)
-
2QdxY4RzWzUUiLuE＠potatochowder.com -
Chris Angelico -
deavid -
Peter Ludemann -
Steven D'Aprano