Since Python's function annotations precede formalized type hinting, users can't just assume that any forms of annotations are automatically type annotations. It's a consequence of historical baggage that annotations as type hints are opt-in and not opt-out. ---- ORIGINAL MESSAGE ---- Date: 2023-07-08 14:56:18 UTC+0200 From:p.f.moore@gmail.com To:carl@oddbird.net CC:typing-sig@python.org Subject: [Typing-sig] Re: What is the effect of py.typed?

Huh? Putting that the other way around, what you are saying is that type checkers (in general, and basically by definition) *don't* respect type annotations on installed libraries by default? Why on earth would I want that behaviour? I can't even think why I'd want to have an *option* to say "please ignore my type annotations", much less have it be the default.

Paul

On Sat, 8 Jul 2023 at 13:48, Carl Meyer <carl@oddbird.net> wrote:

Hi Paul,

On Sat, Jul 8, 2023 at 6:38 AM Paul Moore <p.f.moore@gmail.com> wrote:

What do I lose by *not* having a `py.typed` file? I've read (skimmed, to be fair) the PEP and the linked mypy documentation, and I can't find an answer to that question.

It makes your type annotations also “externally facing.” If someone else has a project depending on your library, and they are using mypy, the py.typed file will allow their mypy runs to consider the annotations on your functions and classes that they import and use.

Carl

_______________________________________________ Typing-sig mailing list --typing-sig@python.org To unsubscribe send an email totyping-sig-leave@python.org https://mail.python.org/mailman3/lists/typing-sig.python.org/ Member address:dev@reggx.eu

It's only vaguely related, but apparently there's also a "Typing :: Typed" trove classifier. What's the point of that? What benefit do people get from me adding that? Again, it's not like it's hard for me to add it, but I don't see the point in doing so, and if there's no good reason to have it, I prefer to keep things minimal. Paul On Sat, 8 Jul 2023 at 14:33, Paul Moore <p.f.moore@gmail.com> wrote:

So you're saying that unless I add the py.typed file, users of my library simply won't see the type annotations I added, and will have no way of fixing that? And type checkers won't even try to interpret the annotations as types?

Well that sucks. It's not like I can't add the `py.typed` file, and I don't want to claim that it's a massive overhead, but it does feel like annoying boilerplate. Plus, I just did a release that didn't include the file, so it'll be missing until the next release. I'm pretty disappointed that nothing warned me about this - surely mypy (which I did run over my code) could have mentioned it? Or linters could spot that I have type annotations but no `py.typed` file?

Paul

On Sat, 8 Jul 2023 at 14:01, ReggX <dev@reggx.eu> wrote:

...

Since Python's function annotations precede formalized type hinting, users can't just assume that any forms of annotations are automatically type annotations. It's a consequence of historical baggage that annotations as type hints are opt-in and not opt-out.

---- ORIGINAL MESSAGE ---- Date: 2023-07-08 14:56:18 UTC+0200 From: p.f.moore@gmail.com To: carl@oddbird.net CC: typing-sig@python.org Subject: [Typing-sig] Re: What is the effect of py.typed?

Huh? Putting that the other way around, what you are saying is that type checkers (in general, and basically by definition) *don't* respect type annotations on installed libraries by default? Why on earth would I want that behaviour? I can't even think why I'd want to have an *option* to say "please ignore my type annotations", much less have it be the default.

Paul

On Sat, 8 Jul 2023 at 13:48, Carl Meyer <carl@oddbird.net> wrote:

...

Hi Paul,

On Sat, Jul 8, 2023 at 6:38 AM Paul Moore <p.f.moore@gmail.com> wrote:

...

What do I lose by *not* having a `py.typed` file? I've read (skimmed, to be fair) the PEP and the linked mypy documentation, and I can't find an answer to that question.

It makes your type annotations also “externally facing.” If someone else has a project depending on your library, and they are using mypy, the py.typed file will allow their mypy runs to consider the annotations on your functions and classes that they import and use.

Carl

_______________________________________________ Typing-sig mailing list -- typing-sig@python.org To unsubscribe send an email to typing-sig-leave@python.orghttps://mail.python.org/mailman3/lists/typing-sig.python.org/ Member address: dev@reggx.eu

Well, yes, I understand you can use classifiers to search, but what I'm asking is why anyone would search for typed libraries. That seems a bizarre thing to base a choice of library on. Paul On Sat, 8 Jul 2023 at 14:45, ReggX <dev@reggx.eu> wrote:

The classifiers are very useful when searching for relevant projects on PyPI, since they work as search filters.

Example: https://pypi.org/search/?q=example&o=&c=Typing+%3A%3A+Typed

---- ORIGINAL MESSAGE ---- Date: 2023-07-08 15:39:12 UTC+0200 From: p.f.moore@gmail.com To: dev@reggx.eu CC: carl@oddbird.net, typing-sig@python.org Subject: Re: [Typing-sig] Re: What is the effect of py.typed?

It's only vaguely related, but apparently there's also a "Typing :: Typed" trove classifier. What's the point of that? What benefit do people get from me adding that? Again, it's not like it's hard for me to add it, but I don't see the point in doing so, and if there's no good reason to have it, I prefer to keep things minimal.

Paul

On Sat, 8 Jul 2023 at 14:33, Paul Moore <p.f.moore@gmail.com> wrote:

...

So you're saying that unless I add the py.typed file, users of my library simply won't see the type annotations I added, and will have no way of fixing that? And type checkers won't even try to interpret the annotations as types?

Well that sucks. It's not like I can't add the `py.typed` file, and I don't want to claim that it's a massive overhead, but it does feel like annoying boilerplate. Plus, I just did a release that didn't include the file, so it'll be missing until the next release. I'm pretty disappointed that nothing warned me about this - surely mypy (which I did run over my code) could have mentioned it? Or linters could spot that I have type annotations but no `py.typed` file?

Paul

On Sat, 8 Jul 2023 at 14:01, ReggX <dev@reggx.eu> wrote:

...

Since Python's function annotations precede formalized type hinting, users can't just assume that any forms of annotations are automatically type annotations. It's a consequence of historical baggage that annotations as type hints are opt-in and not opt-out.

---- ORIGINAL MESSAGE ---- Date: 2023-07-08 14:56:18 UTC+0200 From: p.f.moore@gmail.com To: carl@oddbird.net CC: typing-sig@python.org Subject: [Typing-sig] Re: What is the effect of py.typed?

Huh? Putting that the other way around, what you are saying is that type checkers (in general, and basically by definition) *don't* respect type annotations on installed libraries by default? Why on earth would I want that behaviour? I can't even think why I'd want to have an *option* to say "please ignore my type annotations", much less have it be the default.

Paul

On Sat, 8 Jul 2023 at 13:48, Carl Meyer <carl@oddbird.net> wrote:

...

Hi Paul,

On Sat, Jul 8, 2023 at 6:38 AM Paul Moore <p.f.moore@gmail.com> wrote:

...

What do I lose by *not* having a `py.typed` file? I've read (skimmed, to be fair) the PEP and the linked mypy documentation, and I can't find an answer to that question.

It makes your type annotations also “externally facing.” If someone else has a project depending on your library, and they are using mypy, the py.typed file will allow their mypy runs to consider the annotations on your functions and classes that they import and use.

Carl

_______________________________________________ Typing-sig mailing list -- typing-sig@python.org To unsubscribe send an email to typing-sig-leave@python.orghttps://mail.python.org/mailman3/lists/typing-sig.python.org/ Member address: dev@reggx.eu

Python is my go-to language these days, but I agree that the legacy of non-typed code is a drag. Opting in is a little price to pay, though, and the landscape does get better noticeably year by year. I recently bumped onto a talk that puts this in perspective: https://youtu.be/Tml94je2edk On Sat, 8 Jul 2023, 16:31 ReggX, <dev@reggx.eu> wrote:

Given two libraries that implement the same functionality (like an API wrapper), I'll pick the typed library every time since it saves me the trouble of inspecting the library and creating partial type stubs for my own project.

For people like me, who prefer to fully type their projects to get all of the benefits of type hints, being able to find compatible libraries is a godsend.

Thanks. That's an interesting perspective. Not one that will change what I do, admittedly ;-) I added type hints because it gave *me* better checking that the project's code was correct. I assumed others would get the benefit of those type hints, but given that they didn't, I've added a `py.typed` file to fix that. But I have no intention of reviewing, much less maintaining, the type hints from the perspective of being a contract for end users. Certainly I'm writing the type hints to match what I view as the "expected" inputs, not just what the library code needs, but I'm 100% fine with people overriding the type hints or simply ignoring them, if they want to. If that means I'm not using type hints "the way they were intended", I guess I'll just have to deal with that :-) Paul On Sat, 8 Jul 2023 at 16:42, Mehdi2277 <med2277@gmail.com> wrote:

There's another reason why current behavior is to not use your type hints. It's common for existing library that introduces type hints to only have partial type coverage for some time and that using type hints (especially in --strict mode) would lead to many false positive errors for user. Only when library has enough type coverage that maintainers think type checking it's usage will generally be helpful is py.typed good to add. A rough rule are the most common public API's type hinted? Internal apis do not need to be type hinted when considering py.typed/publicly exposing your libraries types, but if the main documented APIs are still type incomplete it becomes unclear whether it's good choice to use the library's types.

Given the gradual nature of adding types to existing libraries, the other default of respect types even though they may be in progress state would have caused large amount of false positives and I think current choice to be opt-in still makes sense today. I think today somewhere roughly around 25-40% of popular pypi libraries are typed. Many libraries still now having py.typed would be a negative user experience. _______________________________________________ Typing-sig mailing list -- typing-sig@python.org To unsubscribe send an email to typing-sig-leave@python.org https://mail.python.org/mailman3/lists/typing-sig.python.org/ Member address: p.f.moore@gmail.com

Thanks - I do use VS Code, so I'm aware of the advantages of getting tooltips/completion etc. from type checkers. These days, I feel awfully lost when I don't get a tooltip while I code. That's one of the main reasons I want to expose the hints, so that people can get that information. I don't mean to overstate my reservations and sound like I have a big issue with exposing types. I'm mostly just concerned that I might get people wanting me to make the type hints more complex to handle weird edge cases - you know the sort of thing, 3 protocols, 2 overrides and a generic type because my hints don't cater for a user-defined string type that is also registered as providing the `Path` API. Things like that :-) I find the sort of "practicality vs purity" debates that get triggered by these sorts of situations very draining, and I'd like to set expectations. But it seems hard to be nuanced here - you're either a type annotation supporter, or you "don't use types". Trying to explain that you like types, but aren't willing to let them rule your life, seems to cause an awful lot of confusion[^1]... Anyway, this is rather off-topic at this point. Thanks to everyone for their help and insights. I've added the marker file and classifier to my project, hopefully that's the end of it. Paul [^1]: "But your types are *wrong*" - "They are just hints, not absolute truth" - "But they are *wrong*, they don't allow for (some obscure case)"[^2]. [^2]: And to be clear, I've done this myself - the tools are very bad at dealing with things that are fine at runtime but violate the static types, so it's very tempting to view types as "right" or "wrong" with no in-between. On Mon, 10 Jul 2023 at 14:30, Oscar Benjamin <oscar.j.benjamin@gmail.com> wrote:

On Sat, 8 Jul 2023 at 17:21, Paul Moore <p.f.moore@gmail.com> wrote:

...

Thanks. That's an interesting perspective. Not one that will change what

I do, admittedly ;-) I added type hints because it gave *me* better checking that the project's code was correct. I assumed others would get the benefit of those type hints, but given that they didn't, I've added a `py.typed` file to fix that. But I have no intention of reviewing, much less maintaining, the type hints from the perspective of being a contract for end users. Certainly I'm writing the type hints to match what I view as the "expected" inputs, not just what the library code needs, but I'm 100% fine with people overriding the type hints or simply ignoring them, if they want to. If that means I'm not using type hints "the way they were intended", I guess I'll just have to deal with that :-)

The purpose of the py.typed file is precisely to expose those hints to users. There can be good reasons for wanting not to do that e.g. if the codebase does not have many hints or the hints are not well developed. It sounds like you didn't really want to expose them to users or at least do not see what benefit that could provide.

It is worth remembering that the hints are not just used for type checking in the conventional sense. Possibly the main way that your hints will get used downstream is for completion, inference and auto-suggestion in editors. See the example images here showing pyright being used in neovim: https://github.com/sympy/sympy/pull/25103 As you can see from those examples pyright will report errors and more to users even if you *don't* add hints: adding hints can *reduce* the number of "errors" seen by downstream users. Also those users can still derive benefit from your type hints even if they are not using type hints in their own code.

I found these pyright editor features to be useful when I was using it and I expect that many newer Python users probably expect features like that from e.g. pylance in vscode. Unfortunately I have since uninstalled pyright from my editor because just opening a Python file from within a sympy VCS checkout would lock up one core at 100% for 5 minutes while pyright inspects the sympy codebase. I've just checked that this is still the case with the latest version of pyright (and then uninstalled it again).

The last point is potentially one reason not to add a py.typed file. It might be fixed now but in previous versions of pylance (i.e. vscode) any sympy user who added code like "import sympy" would see 100% CPU usage for several minutes. I think that problem was initially triggered by sympy adding a py.typed file somehow but is maybe "fixed" in some version of pylance (not sure how exactly).

If you are not big into typing and don't use the latest trendy editors etc then you might not have any idea what impact the py.typed file is going to have. Adding the file to sympy seemed harmless since there were a few type hints and I presumed that it would not impact users who were not themselves using type hints but that turned out not to be the case.

-- Oscar _______________________________________________ Typing-sig mailing list -- typing-sig@python.org To unsubscribe send an email to typing-sig-leave@python.org https://mail.python.org/mailman3/lists/typing-sig.python.org/ Member address: p.f.moore@gmail.com

Eric, Thanks for the detailed reply. My personal use of type checking is very basic - I run mypy on my code as one of the "lint" checks, and I use VS Code as my editor (which uses pyrite "under the hood" I believe?) This seems a common sort of usage, though, if my experience with other projects is indicative. It never occurred to me that different tools might do different things here - I assumed that standardisation meant well-defined behaviour in all tools. Maybe that's my experience with packaging standards misleading me. I don't think it makes a big deal to me, personally, I can add `py.typed` so that mypy picks up the types and that's fine. I think that as a library developer, my main concern is that it seems quite hard to correctly annotate the sorts of usage that come under the heading of "duck typing". Things like protocols, overloads and generic types seem to be aimed at that sort of use case. I very much do intend my code to be usable in a "duck typed" manner - I'm willing (in principle) to take the view that if you supply an object with the right interface, I'll use it correctly. But I *don't* want to have to learn complex typing to express that. So I take a middle ground - annotate my functions with their expected types (and type check my code because those are the types I use internally), but leave broader usage supported. I fear that some people may see the type annotations and think duck typing is prohibited, but there's not much I can do about that, and I have no wish to include a big philosophical tract on my position over typing in my docs. Conversely, though, I worry that some people might ask me to "support" duck typing, and not be willing to accept "it's supported, just ignore the type checking errors" as an answer. That's the expanded version of my statement "I have no intention of reviewing, much less maintaining, the type hints from the perspective of being a contract for end users". It doesn't mean I don't want to make type hints available for users, or that I don't want them to be useful. Just that I consider "str" to be a much more useful annotation than some construct that says "Container of characters that supports the `partition`, `endswith` and `replace` methods" (for example). Even if passing (certain instances of) the latter is technically a violation of the `str` type hint. Paul On Wed, 12 Jul 2023 at 10:23, Eric Traut <eric@traut.com> wrote:

Paul, you're asking a good question here. I also found PEP 561 to be lacking in describing what "py.typed" implies and how it should be treated by type checkers and language servers. In an attempt to clarify, I wrote the following documentation for pyright: https://microsoft.github.io/pyright/#/typed-libraries. The current version of this documentation incorporates feedback from the community and has been stable for the past two years. A stripped-down version of this documentation was also incorporated into the python/typing site: https://github.com/python/typing/blob/master/docs/source/libraries.rst.

Here is how pyright (and pylance — the VS Code language server built on pyright) interpret a "py.typed" file.

* By default, pyright makes use of type annotations within a library regardless of whether a "py.typed" file is present. (This differs from mypy's default.) * If the user sets the "useLibaryCodeForTypes" configuration option to false, pyright will treat all symbols imported from a non-py.typed library as "Unknown". (This is effectively what mypy does by default.) * For py.typed libraries, pyright applies stricter rules about which symbols are considered "public". For details, refer to the documentation. Pyright will allow users to access non-public symbols in a py.typed library, but it will flag such use as an error by default.

I think that most (all?) Python language servers honor type annotations in your library even if you don't have a "py.typed" file. Some (but not all) Python type checkers also honor type annotations in non-py.typed files by default.

Keep in mind that the percentage of Python developers who use a language server is much larger than the percentage who use a type checker. I estimate that 70% of all active Python developers use a language server or smart editor of some sort (pylance, jedi, PyCharm) but only 5-10% use static type checkers (pyright, mypy, pyre).

I've incorporated a tool into pyright that allows you to validate the type information in your "py.typed" library. If you use `pyright --verifytypes <your lib>`, pyright will analyze the library for symbols whose types are not annotated. Increasingly, library authors are incorporating this into their CI pipelines to keep their libraries "type clean".

Eric Traut Primary author of pyright _______________________________________________ Typing-sig mailing list -- typing-sig@python.org To unsubscribe send an email to typing-sig-leave@python.org https://mail.python.org/mailman3/lists/typing-sig.python.org/ Member address: p.f.moore@gmail.com

On Wed, 12 Jul 2023 at 11:03, Paul Moore <p.f.moore@gmail.com> wrote:

Thanks for the detailed reply. My personal use of type checking is very basic - I run mypy on my code as one of the "lint" checks, and I use VS Code as my editor (which uses pyrite "under the hood" I believe?) This seems a common sort of usage, though, if my experience with other projects is indicative. It never occurred to me that different tools might do different things here - I assumed that standardisation meant well-defined behaviour in all tools. Maybe that's my experience with packaging standards misleading me.

Does your experience of packaging standards honestly look like "well defined behaviour in all tools" :) The PEPs for typing leave a lot of things undefined. The py.typed PEP says that "Package maintainers who wish to support type checking of their code MUST add a marker file named py.typed to their package supporting typing". The PEP does not say anything about how the presence or absence of the file influences what checkers or other tools will or will not do though. I guess there is no way as a package author to know what the effect of including or not including the file is because the tools can choose to do anything. Among type checkers the tools don't even agree on how to interpret the hints and differ wildly in how they analyse both hinted and unhinted code. If you try to apply type hints to a large previously untyped codebase that uses lots of dynamic typing features (e.g. sympy) then you will find a *lot* of differences between the tools. Currently running `mypy sympy` passes without error (I previously made a lot of changes including currently 200 type:ignore to make that happen) but `pyright sympy` reports: 42405 errors, 926 warnings, 0 informations Analysing those carefully reveals a lot of differences between mypy and pyright. In retrospect I would have preferred for sympy to use pyright because when I have found differences in fully hinted code I have always preferred pyright's opinion over mypy's. Also pyright is used in the most popular language servers and I now realise that those are a bigger application of type hints in Python than CI checks. Note that it is probably impossible to get *both* mypy and pyright to accept or at least understand a fully hinted sympy codebase without major rewriting or a lot of type: ignore. The tools also have different options so e.g.: $ mypy --strict sympy ... Found 179467 errors in 1261 files (checked 1452 source files) That's approximately one error per line of actual code! Both mypy and pyright have many, many more options than this and different incompatible ways of configuring them. It is impossible to know what combinations of those might be used by any users downstream and likewise your configuration might differ from any upstream codebases that you use. Maybe usually those differences don't matter but again the tools can decide to do anything. Someone even made a tool that monkeypatches typing.TYPE_CHECKING=True at runtime. That seems obviously invalid but it doesn't stop sympy getting a bug report about not being compatible with the tool: https://github.com/sympy/sympy/issues/23216 What that shows is that "tools" really can just do *anything*. You can choose to use them or not. Your users might choose to use different tools. As a package author there is probably nothing that you can do that would prevent users sometimes requesting that you support something that you don't want to support. -- Oscar

On Wed, 12 Jul 2023 at 15:53, Paul Moore <p.f.moore@gmail.com> wrote:

On Wed, 12 Jul 2023 at 13:51, Oscar Benjamin <oscar.j.benjamin@gmail.com> wrote:

...

On Wed, 12 Jul 2023 at 11:03, Paul Moore <p.f.moore@gmail.com> wrote:

Among type checkers the tools don't even agree on how to interpret the hints and differ wildly in how they analyse both hinted and unhinted code. If you try to apply type hints to a large previously untyped codebase that uses lots of dynamic typing features (e.g. sympy) then you will find a *lot* of differences between the tools. Currently running `mypy sympy` passes without error (I previously made a lot of changes including currently 200 type:ignore to make that happen) but `pyright sympy` reports:

42405 errors, 926 warnings, 0 informations

Analysing those carefully reveals a lot of differences between mypy and pyright.

That sounds pretty crazy for tools that are supposed to indicate whether your code is "correct" (whatever correct means in this context...)

The parenthetical remark is spot on: what does "correct" mean but also in *which* context? I think that mypy and pyright have different fundamental goals which is why they differ in their analysis of code. The basic goal of pyright is to do static analysis that predicts what would happen at runtime. That enables pyright to offer code completion etc to vscode users in relation to the libraries they use regardless of whether those libraries have hints or happen to look odd from pyright's perspective. The goal of mypy is to be a complicated code linter i.e. how you are already using it. It uses type hints and static analysis but its goal is just to reject the code that it analyses if the code does not meet its own possibly opinionated demands. Validation/rejection (especially if opinionated) is a very different task from analysis/inference: there might be overlap between these but this is like the is/ought distinction. For an example of the difference between them see here: https://github.com/python/mypy/issues/15182 Let's adapt that example removing inheritance and type hints to demonstrate this point: ``` class A: def __new__(cls): return super().__new__(cls) class B: def __new__(cls): return A() reveal_type(B()) ``` The two checkers show: $ pyright q.py /home/oscar/current/active/sympy/q.py /home/oscar/current/active/sympy/q.py:9:13 - information: Type of "B()" is "A" 0 errors, 0 warnings, 1 information $ mypy q.py q.py:9: note: Revealed type is "q.B" Success: no issues found in 1 source file Here pyright is correct in inferring what *actually* happens at runtime: the result of the expression B() will be an instance of the class A. That is the information that is needed to offer accurate suggestions in vscode etc. In that sense mypy is incorrect because it infers that B() will be of type B but the difference is that if I add type hints (just `-> A` return type on both methods) then mypy would reject this code altogether: $ mypy q.py q.py:8: error: Incompatible return type for "__new__" (returns "A", but must return a subtype of "B") [misc] q.py:11: note: Revealed type is "q.B" Found 1 error in 1 file (checked 1 source file) The difference here is that mypy has decided that B.__new__ *should* return an instance of B whereas pyright has inferred that B.__new__ *does* in fact return an instance of A. Mypy only allowed the code to begin with because by default it is permissive of code that lacks type hints but with --strict it would have never allowed B.__new__ to return A(). From mypy's perspective this is non-conforming code and should be rejected although AFAIK there is no standard for why this is rejected. Mypy does not feel as strong a need to analyse the runtime behaviour of this code accurately because any code that could ever do this is a priori invalid and would already be rejected: you should fix the code first and only then ask mypy what the type of B() is. This __new__ problem puts SymPy in a bind. SymPy makes heavy use of __new__: grep counts 445 __new__ methods in the codebase and almost all the objects used by users are returned from those methods. The reason for using __new__ is *precisely* to be able to return an object of a different type (although always an instance of a common superclass). These methods are more complicated than the examples above though and without type hints pyright cannot offer accurate inference/autocompletion etc to users. To some extent it can be okay to have different tools do different things. There can only be one version of the type hints in the SymPy codebase though and all the tools that anyone chooses to use will need to share those hints. Currently SymPy is faced with choosing between hints that make pyright work for downstream users and their editors or hints that make mypy happy for downstream library authors and their CI. SymPy's internal correctness is unlikely to gain much either way.

PS Thanks for all your work on sympy. I use it regularly - mostly just for casual curiosity and "toy" problems, but it's an awesome library and really useful.

Thanks and thanks for all your work on pip (and packaging more generally) which I use literally every day! -- Oscar

I think what I’m seeing here, to summarize, is simply that type checking differs from some of the philosophies that many developers expect. I don’t mean to put words in anyone’s mouth though, so I apologize if this is not an accurate representation. As a developer of python, I expect the zen of python to, generally speaking, be true. Specifically: There should be one-- and preferably only one --obvious way to do it. Type checkers though are fundamentally different than this as there is no one way to do it because of the goals. As a developer who works in other languages, whether typescript, go, or rust, typing is a singular concept to me the user. A way to begin to address this may be to more clearly articulate the motivation for each project in their respective repositories. Having just looked at both Mypy and Pyright’s README, they both do a good job of explaining the “what” they are and their characteristics (e.g., high performance or designed for gradual typing). None of that is wrong, but it skips over what’s being highlighted here about motivation for each project and why there is a need for different implementations and when a user may want to choose one over another. I think that this disconnect, for lack of a better word, between the tooling and the expectations many bring to them is at the heart of much of this discussion. As I said, I apologize if anyone feels I misrepresented what they said, just what my takeaway was from watching this conversation. Sincerely, Kevin Kirsche ----- Please excuse the brevity; this email was sent from a mobile device. On Jul 13, 2023, at 03:34, Eric Traut <eric@traut.com> wrote: I like to think of a static type checker as "a tool that helps you to write _robust code_ if that is your goal". A quick-and-dirty script doesn't need to be robust, but production code does. To me, "robust" not only means that the code is "correct" (in that it does what it's intended to do), but that it also continues to works correctly as it is modified (perhaps by many different contributors) over time — and as new versions of libraries and Python runtimes are released. A static type checker makes it easier and cheaper to create and maintain robust code. Yes, there are differences between Python type checkers. These differences are generally manifest in the analysis of _implementations_ rather than in the interpretation of _interfaces_. In other words, if you annotate the _interface_ to your library, all Python static type checkers _should_ interpret these annotations the same. This is important for both library authors and consumers, we (the maintainers of static type checkers) place a lot of emphasis on consistency in how type checkers treat interface annotations. This is where we have spent the most effort on standardization. If you or the consumers of your library see differences in how pyright and mypy interpret the annotations in your library's _interface_, please report that to us. You will see some differences between type checkers when analyzing the _implementation_ of your code, but in "basic type checking mode" (i.e. not "strict mode") the differences tend to be pretty minimal. Oscar, you mentioned that when you run pyright on sympy you receive 42K errors. This is mostly because mypy skips functions that do not have parameter annotations whereas pyright analyzes all code by default. If I run `pyright --skipunannotated` on the sympy code base, the error count drops to under 1K. Most of those fall into a couple of patterns that could be easily ignored or fixed. A few of them (at least on first inspection) look like real bugs that pyright is catching.

On Thu, 13 Jul 2023 at 17:19, Kevin Kirsche <kev.kirsche@gmail.com> wrote:

I think what I’m seeing here, to summarize, is simply that type checking differs from some of the philosophies that many developers expect. I don’t mean to put words in anyone’s mouth though, so I apologize if this is not an accurate representation. As a developer of python, I expect the zen of python to, generally speaking, be true. Specifically:

There should be one-- and preferably only one --obvious way to do it.

Type checkers though are fundamentally different than this as there is no one way to do it because of the goals. As a developer who works in other languages, whether typescript, go, or rust, typing is a singular concept to me the user.

I don't know typescript or go very well but I would say that both typescript and typed python are in a separate category from languages like go and rust (and C etc.). The difference is fundamental: in python and typescript the static type system exists in parallel to a runtime type system. There is nothing "singular" about this because there are fundamentally two type systems that do not interact directly. The static type system loosely attempts to mimic a stricter form of the runtime type system but it does not determine what happens at runtime. In go, rust, C etc the compiler needs the type system in order to generate the code: without fully resolving the static types there will be no runtime. The fact that python has two type systems in parallel and only one is responsible for actually making the code run means that the two systems can contradict each other. I think I have seen references to examples of type contradictions in typescript but I don't know typescript well enough to remember what they were or list them here. Contradictions between python's runtime and static type systems were introduced from the outset with e.g. PEP 484 declaring that int is a subtype of float: https://peps.python.org/pep-0484/#the-numeric-tower That standardised behaviour means that both mypy and pyright will agree that this code is fine even in strict mode despite the fact that it fails at runtime and regardless of the fact that it could easily be rejected by a static checker: ``` import math def mysqrt(x: float) -> float: return math.sqrt(x) print(mysqrt(1e1000)) # inf print(mysqrt(10 ** 1000)) # OverflowError ``` The tools can be consistent but that will never mean that there is a singular concept of types as long as the static type system only mimics the *real* runtime one and does not even *aim* to imitate it precisely. -- Oscar

Is there an option to tell pyright not to print out the "informations" that things are being skipped because they are unannotated?

There's currently no way to suppress that diagnostic. You could easily filter these diagnostics with a script if they bother you. But we're getting pretty far off topic at this point. If you have questions about pyright or want to suggest new features, please post to the pyright issue tracker or discussion forum.