Hi, I would like to hear the opinion of Python's community on enforcing types in the future for the language. I've been using Python as my main language for everything for around 10 years, until I started moving to Rust 2 years ago; one of the main factors was types.

Just before moving to Rust I started to use mypy heavily, which I liked a lot and uncovered tons of potential problems. Now (2 years later), it seems the situation hasn't changed much; I might be wrong, so let me know what improvements you think landed in this area in the last 2-3 years.

I feel it's possible this topic might cause a lot of passionate answers, but I just want to hear honest opinions on this.

I firmly believe that Python's future would be better if types were enforced by default at "compile time" This can't be done, in view of Python's dynamic nature.  As I understand it, it is impossible to check at compile time the type of an argument

Please, no, No, NO! It has always been the policy that typing is, and will remain, optional. On 09/12/2021 20:32, deavid wrote: that is about to be passed to a function, unless it is a constant (e.g. a literal int, float, str etc.) Best wishes Rob Cliffe

(whatever this means in Python), with an option/flag to disable this, and integrate MyPy or similar into the interpreter. I'm fully aware that a transition like this would be very hard and long, but I don't think it's impossible.

Here's a list of my reasons to think that Python is better if it was typed:

1) On really big codebases and complex projects, it's very easy to lose track of what things do. Types help detecting bugs early. (Ask anyone that has used Rust + Clippy, the amount of errors that are catched is amazing, programs tend to work on the first try) 2) Libraries are currently the top bottleneck for any team to start using MyPy/Pytype. Making types mandatory would ensure all libraries have type support. (If anyone has any other proposal to enforce this, I would like to hear) 3) IDE integration is way simpler and better with types. 4) The interpreter could take further optimizations if it can prove that a function or piece of code is guaranteed to have a limited set of types. This could be used by libraries to have great speed ups that currently are not possible. 5) Static analysis tools could also benefit from types to gain more insight on what the code is trying to do.

Of course, types have their own set of drawbacks; for example it could make Python look harder to code for newcomers, or it might get in the way for things like Jupyter notebooks, ML, and similar stuff. Because of this, an escape hatch must always exist. (maybe there are even more problems I am not aware about, I'd love to hear)

If it were for me, I would like to have a Python 4 that is exactly a Python 3 but with mypy bundled and strictly enforced by default; with a flag to convert errors into warnings or disable entirely. Then every release, say a Py3.11, would also get a Py4.11-beta (the beta would be to avoid people migrating until it's ready).

In this way, for a library to say it has Py4 compatibility it would need to be type-ready. Jupyter notebooks and such would be stuck at Py3, but of course, getting all the releases; and enterprises would be trying to use Py4 whenever it were ready.

Typescript is also basically Javascript with types (well, not only that, but anyway) and the result is quite good. In this fashion, another alternative is having a second binary called TPython or MPython, and include it on the regular Python distribution; this would cause less push to go for types, but it could do the trick too.

So well, my question here is: why is this not a thing? Has anyone proposed something like this before? I feel I must have missed something important.

Thanks, David

_______________________________________________ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-leave@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/QBWYBX... Code of Conduct: http://python.org/psf/codeofconduct/

First off I love types. They, as you point out, are a great tool for finding bugs. However, there are reasons why requiring types would either require the use of meaningless types e.g. marking everything as the Any type or require python to solve the halting problem. Consider the following code example: ``` class ArrayMeta(type): On Thu, Dec 9, 2021 at 12:52 PM deavid <deavidsedice@gmail.com> wrote:

Hi, I would like to hear the opinion of Python's community on enforcing types in the future for the language. I've been using Python as my main language for everything for around 10 years, until I started moving to Rust 2 years ago; one of the main factors was types.

Just before moving to Rust I started to use mypy heavily, which I liked a lot and uncovered tons of potential problems. Now (2 years later), it seems the situation hasn't changed much; I might be wrong, so let me know what improvements you think landed in this area in the last 2-3 years.

I feel it's possible this topic might cause a lot of passionate answers, but I just want to hear honest opinions on this.

I firmly believe that Python's future would be better if types were enforced by default at "compile time" (whatever this means in Python), with an option/flag to disable this, and integrate MyPy or similar into the interpreter. I'm fully aware that a transition like this would be very hard and long, but I don't think it's impossible.

Here's a list of my reasons to think that Python is better if it was typed:

1) On really big codebases and complex projects, it's very easy to lose track of what things do. Types help detecting bugs early. (Ask anyone that has used Rust + Clippy, the amount of errors that are catched is amazing, programs tend to work on the first try) 2) Libraries are currently the top bottleneck for any team to start using MyPy/Pytype. Making types mandatory would ensure all libraries have type support. (If anyone has any other proposal to enforce this, I would like to hear) 3) IDE integration is way simpler and better with types. 4) The interpreter could take further optimizations if it can prove that a function or piece of code is guaranteed to have a limited set of types. This could be used by libraries to have great speed ups that currently are not possible. 5) Static analysis tools could also benefit from types to gain more insight on what the code is trying to do.

Of course, types have their own set of drawbacks; for example it could make Python look harder to code for newcomers, or it might get in the way for things like Jupyter notebooks, ML, and similar stuff. Because of this, an escape hatch must always exist. (maybe there are even more problems I am not aware about, I'd love to hear)

If it were for me, I would like to have a Python 4 that is exactly a Python 3 but with mypy bundled and strictly enforced by default; with a flag to convert errors into warnings or disable entirely. Then every release, say a Py3.11, would also get a Py4.11-beta (the beta would be to avoid people migrating until it's ready).

In this way, for a library to say it has Py4 compatibility it would need to be type-ready. Jupyter notebooks and such would be stuck at Py3, but of course, getting all the releases; and enterprises would be trying to use Py4 whenever it were ready.

Typescript is also basically Javascript with types (well, not only that, but anyway) and the result is quite good. In this fashion, another alternative is having a second binary called TPython or MPython, and include it on the regular Python distribution; this would cause less push to go for types, but it could do the trick too.

So well, my question here is: why is this not a thing? Has anyone proposed something like this before? I feel I must have missed something important.

Thanks, David

_______________________________________________ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-leave@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/QBWYBX... Code of Conduct: http://python.org/psf/codeofconduct/

"Should Python enforce Type-checking in the future?" Python already enforces type-checking. It is *runtime* time checking. https://cdsmith.wordpress.com/2011/01/09/an-old-article-i-wrote/ As far as *static* type checking goes, the answer is No. (Obviously all of the following is my opinion, not holy writ.) (1) Moving mypy into CPython would be bad for both mypy and CPython. The release schedules and stability guarantees of each are different. It would require the influx of a huge amount of complexity into CPython to integrate mypy. And to save what? mypy is free, and easy to install if you have an internet connection. The benefits of integration are tiny compared to the costs. (2) Requiring static type checking would be bad for alternative implementations. Python is dominated by CPython, and we should be mildly concerned about that: languages dominated by a single implementation (like Ruby and Perl) are far more likely to lose popularity and fade away than those with many implementations (like C). Anything that makes it harder for new implementations to get started is a negative. Right now, we do have PyPy and MicroPython, both reasonably popular, but the other "big" implementations Jython and IronPython are stuck in the Python 2 world. Making it harder for them to catch up would be a negative. How hard would it be to migrate PyPy to enforced static typing? What would that do to the smaller Python implementations? How would it affect Cython and Numba and other projects? MicroPython needs to run on tiny machines with limited memory resources. It will probably never support mandatory static type checking. Running Python on tiny machines like Raspberry Pi and Arduino is a big part of the Python ecosystem, so mandatory static checking would break the language and ecosystem into what is effectively two languages: classic Python 3 and new "Python 4" with mandatory static checking. (3) Have we learned nothing from the 2-to-3 transition of the pain and extra work and disruption that a huge backward-compatibility breaking change to the language makes? Faced with a major migration from 3-to-4 that would be as big and painful as 2-to-3, many projects and libraries would just abandon Python altogether, in favour or rewriting their project in a "real" statically typed language with proper stability guarantees. We should not be talking about Python 4 here, we should be thinking Python 5000 or even Python 6000. More seriously, incremental changes to the language is fine. Massive breakage all at once, let's not repeat that. (4) Mandatory static types would be bad for students and beginners, bad for scientists, bad for casual users, and bad for system administrators who still use Python as a scripting language. It would make the language harder to use and harder to learn. Python is, not unique, but very unusual in the broad range of uses people put it to. It is a *teaching language* that can be used for professional projects. It is a *glue language* and a *scripting language* that is written by people who just want something that works and don't want to fight the compiler's type checking errors to run it. That's why dynamic type checking was invented in the first place. (5) Requiring static type checking would be bad for mypy. mypy is designed for *gradual typing*, that is, type checking when only parts of your code have static type hints. To move to a world where type hints are no longer hints but mandatory strictly enforced types would likely make a lot of mypy obsolete. It would likely take a lot of work to rip out the code that assumes gradual typing and replace it with code that strictly enforces types. (6) Requiring formal static types goes against modern language design for applications languages. Python is not a systems language like Rust. Modern applications languages tend to converge to the middle of the typing landscape: - languages with static types tend to develop ways to escape the type system and allow some dynamic typing features; e.g. Java has generics and reflection. - languages with dynamic typing tend to develop optional ways to enforce those types statically. This is, of course, exactly where Python is now. A fundamentally dynamically typed language with optional static typing features. And that's a good thing. -- Steve

Hi David, I think there's an important distinction to be made between "enforcing types" as in "type annotations that are present are checked for consistency at compile time and enforced at runtime, instead of only being checked by a linter and possibly totally wrong at runtime" vs "enforcing types" as in "everything must always be fully typed." It seems that you are proposing both together, and so far readers are mostly reacting against the second. But I would like to observe that the one does not have to imply the other. I think requiring types everywhere is pretty much a non-starter for the Python language and community; the resulting language would no longer be Python, it would be a very different language with Python's syntax. But I think integrating optional types more closely into the language, so that users can trust that the type annotations that are present are not well-intentioned lies, and the compiler can make use of them to optimize some operations, increases the value of type annotations and the experience of using them, and can still fully allow for untyped and partially-typed code in the places where static types don't make sense (because the code is highly dynamic or because its just a quick script or prototype.) I currently work on a project we call "Static Python" (not a great name, since really it's fully dynamic Python with optional static types) which does exactly this. Types remain fully optional, the type system is gradual, but where you do specify annotations of types that are built-ins or are defined in modules opted into the Static Python compiler, you get compile-time type checking and runtime checks that the values used match the annotated types. This is currently only available if you use the fork of CPython in which we develop it [1]. We hope that at some point it may be possible to instead make it available as a pip-installable extension to normal CPython, but this is probably a long ways out yet. We are building this primarily for the performance benefits. The benefit of being able to use statically known and trusted type information to elide some costly dynamic operations more than makes up for the cost of the added runtime type checks, partly because our compiler is able to intelligently elide those runtime type checks in static-to-static calls, only checking when values first are passed in from untyped sources. This is early experimental work, and very far from anything that might be considered for mainstream CPython, but if you're interested in this area, you might find it worth taking a look at this early stage! Carl [1] https://github.com/facebookincubator/cinder

Hi, (NB: I'm just a lurker here, posting for almost the first time. I may say silly things.) Le 09/12/2021 à 21:32, deavid a écrit :

Hi, I would like to hear the opinion of Python's community on enforcing types in the future for the language. I've been using Python as my main language for everything for around 10 years, until I started moving to Rust 2 years ago; one of the main factors was types.

Just before moving to Rust I started to use mypy heavily, which I liked a lot and uncovered tons of potential problems. Now (2 years later), it seems the situation hasn't changed much; I might be wrong, so let me know what improvements you think landed in this area in the last 2-3 years.

I don't follow typing closely at all, but I do nevertheless see quite a number of typing PEPs getting accepted. Here are a few recent ones gathered just by reading the index in PEP 0: https://www.python.org/dev/peps/pep-0589/ https://www.python.org/dev/peps/pep-0586/ https://www.python.org/dev/peps/pep-0585/ https://www.python.org/dev/peps/pep-0591/ https://www.python.org/dev/peps/pep-0593/ https://www.python.org/dev/peps/pep-0604/ https://www.python.org/dev/peps/pep-0612/ https://www.python.org/dev/peps/pep-0613/ https://www.python.org/dev/peps/pep-0647/

I feel it's possible this topic might cause a lot of passionate answers, but I just want to hear honest opinions on this.

I firmly believe that Python's future would be better if types were enforced by default at "compile time" (whatever this means in Python), with an option/flag to disable this, and integrate MyPy or similar into the interpreter. I'm fully aware that a transition like this would be very hard and long, but I don't think it's impossible.

Here's a list of my reasons to think that Python is better if it was typed:

1) On really big codebases and complex projects, it's very easy to lose track of what things do. Types help detecting bugs early. (Ask anyone that has used Rust + Clippy, the amount of errors that are catched is amazing, programs tend to work on the first try) 2) Libraries are currently the top bottleneck for any team to start using MyPy/Pytype. Making types mandatory would ensure all libraries have type support. (If anyone has any other proposal to enforce this, I would like to hear) 3) IDE integration is way simpler and better with types. 4) The interpreter could take further optimizations if it can prove that a function or piece of code is guaranteed to have a limited set of types. This could be used by libraries to have great speed ups that currently are not possible. 5) Static analysis tools could also benefit from types to gain more insight on what the code is trying to do.

These are benefits of types in general, but I don't see clearly how having checks built as part of the standard CPython interpreter will increase them. Performance perhaps, but the projects using annotations to eliminate runtime overhead are in experimental stage. Asking a genuine question here: is it actually possible to generate optimized code that supports all of Python? What about things like int subclasses? MyPyC doesn't support them as far as I can see, and I don't see how it could when trying to generate optimized code. It might only optimize int operations when it can prove that all callers don't pass int subclasses, but this means recompiling when a new caller does pass a subclass -- which turns Python into far more of a compiled language. Or am I missing something? On the other hand, with the specializing adaptive that is the result of the Faster CPython project, it might become eventually possible to specialize based on types, and have a cheaper code path when the annotation is int that will still fall back to a slower path when an int subclass instance is passed. Also, you seem to imply that having typing more tied to the core language will bring it more popularity. How will this happen exactly? People don't like it when you break their code. If Python 4 causes trouble for users who don't need/want typing, these users will not suddenly start using typing, but simply stay with Python 3. You'd apparently want to require libraries to be annotated for them to support Python 4, which means pain that the ecosystem simply cannot afford, I think. -- And if the annotations remain optional, then I guess I don't really understand how it's going to push people to add them, especially on large libraries, where this is a significant endeavor. All the best, Jean