Re: Typing syntax and ecosystem
Message: 1 Date: Sun, 11 Apr 2021 13:31:12 -0700 From: Barry Warsaw <barry@python.org> Subject: [Python-Dev] Re: PEP 647 Accepted
This is something the SC has been musing about, but as it’s not a fully formed idea, I’m a little hesitant to bring it up. That said, it’s somewhat relevant: We wonder if it may be time to in a sense separate the typing syntax from Python’s regular syntax. TypeGuards are a case where if typing had more flexibility to adopt syntax that wasn’t strictly legal “normal” Python, maybe something more intuitive could have been proposed. I wonder if the typing-sig has discussed this possibility (in the future, of course)?
[ munch ]
Agreed. It’s interesting that PEP 593 proposes a different approach to enriching the typing system. Typing itself is becoming a little ecosystem of its own, and given that many Python users are still not fully embracing typing, maybe continuing to tie the typing syntax to Python syntax is starting to strain.
I would really like to see either "Typed Python" become a different programming language, or progress to building type checking into the CPython implementation itself. (Python 4 seems to me the obvious release.) The current halfway approach is confusing and slightly ridiculous. The first, a separate programming language, would be like RATFOR and CFront in the past and TypeScript today. Typed Python can have whatever syntax the designers want because it doesn't have to be compatible with Python, just as TypeScript is not constrained by JavaScript. A type checker translates the original Typed Python source into "dynamic" or "classic" Python for execution. (Maybe into .pyc instead of .py?) This would mean no overhead for type checking in CPython itself. No need to contort the parser into ignoring bits of code that are, in effect, syntax checked comments. And for the typing in Python enthusiasts, you won't have to listen to people like me complaining. The second approach is to assume that type checking in Python is useful and popular. Not with me, but I'm willing to accept that I'm in the minority and can be ignored - after all, I can still write my Python code without type annotations. If so running a type checker as a separate step, as we do at the moment, is like asking C programmers to run the preprocessor by hand. In today's world of continuous build and integration, it seems silly to me to have a type checker read the source, scan into lexical tokens, build an abstract syntax tree, perform semantic analysis with type checking, and then throw it away before running an interpreter which reads the same source, scans into lexical tokens, builds an abstract syntax tree, and executes. On the purely pragmatic level there is an extra chance for mismatches and things to go wrong; and from an environmental viewpoint it isn't a great use of resources. -- cheers, Hugh Fisher
On Mon, Apr 12, 2021 at 3:01 AM Hugh Fisher <hugo.fisher@gmail.com> wrote:
Message: 1 Date: Sun, 11 Apr 2021 13:31:12 -0700 From: Barry Warsaw <barry@python.org> Subject: [Python-Dev] Re: PEP 647 Accepted
This is something the SC has been musing about, but as it’s not a fully
formed idea, I’m a little hesitant to bring it up. That said, it’s somewhat relevant: We wonder if it may be time to in a sense separate the typing syntax from Python’s regular syntax. TypeGuards are a case where if typing had more flexibility to adopt syntax that wasn’t strictly legal “normal” Python, maybe something more intuitive could have been proposed. I wonder if the typing-sig has discussed this possibility (in the future, of course)?
[ munch ]
Agreed. It’s interesting that PEP 593 proposes a different approach to
enriching the typing system. Typing itself is becoming a little ecosystem of its own, and given that many Python users are still not fully embracing typing, maybe continuing to tie the typing syntax to Python syntax is starting to strain.
I would really like to see either "Typed Python" become a different programming language, or progress to building type checking into the CPython implementation itself. (Python 4 seems to me the obvious release.) The current halfway approach is confusing and slightly ridiculous.
Please don't denigrate the hard work people have put in to even bring forward the idea of typing in Python by saying it's "slightly ridiculous". -Brett
The first, a separate programming language, would be like RATFOR and CFront in the past and TypeScript today. Typed Python can have whatever syntax the designers want because it doesn't have to be compatible with Python, just as TypeScript is not constrained by JavaScript. A type checker translates the original Typed Python source into "dynamic" or "classic" Python for execution. (Maybe into .pyc instead of .py?)
This would mean no overhead for type checking in CPython itself. No need to contort the parser into ignoring bits of code that are, in effect, syntax checked comments. And for the typing in Python enthusiasts, you won't have to listen to people like me complaining.
The second approach is to assume that type checking in Python is useful and popular. Not with me, but I'm willing to accept that I'm in the minority and can be ignored - after all, I can still write my Python code without type annotations. If so running a type checker as a separate step, as we do at the moment, is like asking C programmers to run the preprocessor by hand.
In today's world of continuous build and integration, it seems silly to me to have a type checker read the source, scan into lexical tokens, build an abstract syntax tree, perform semantic analysis with type checking, and then throw it away before running an interpreter which reads the same source, scans into lexical tokens, builds an abstract syntax tree, and executes. On the purely pragmatic level there is an extra chance for mismatches and things to go wrong; and from an environmental viewpoint it isn't a great use of resources.
--
cheers, Hugh Fisher _______________________________________________ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-leave@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/2DMJPVE4... Code of Conduct: http://python.org/psf/codeofconduct/
April 12, 2021 4:59 PM, "Brett Cannon" <brett@python.org> wrote:
On Mon, Apr 12, 2021 at 3:01 AM Hugh Fisher <hugo.fisher@gmail.com> wrote:
Message: 1 Date: Sun, 11 Apr 2021 13:31:12 -0700 From: Barry Warsaw <barry@python.org> Subject: [Python-Dev] Re: PEP 647 Accepted
This is something the SC has been musing about, but as it’s not a fully formed idea, I’m a little
hesitant to bring it up. That said, it’s somewhat relevant: We wonder if it may be time to in a sense separate the typing syntax from Python’s regular syntax. TypeGuards are a case where if typing had more flexibility to adopt syntax that wasn’t strictly legal “normal” Python, maybe something more intuitive could have been proposed. I wonder if the typing-sig has discussed this possibility (in the future, of course)?
[ munch ]
Agreed. It’s interesting that PEP 593 proposes a different approach to enriching the typing
system. Typing itself is becoming a little ecosystem of its own, and given that many Python users are still not fully embracing typing, maybe continuing to tie the typing syntax to Python syntax is starting to strain.
I would really like to see either "Typed Python" become a different programming language, or progress to building type checking into the CPython implementation itself. (Python 4 seems to me the obvious release.) The current halfway approach is confusing and slightly ridiculous.
Please don't denigrate the hard work people have put in to even bring forward the idea of typing in Python by saying it's "slightly ridiculous".
-Brett
Aren't people allowed to have their own opinions? Please, I hate to see this list descend further and further into such knee-jerk reactions. If criticism of any current implementation of any construct becomes off-limits is automatically classed as "denigrating", there is no reason for this list to exist. You might not agreed with the criticism, but you should at least be open to discussion.
On Mon, Apr 12, 2021 at 2:19 PM <edwin@211mainstreet.net> wrote:
April 12, 2021 4:59 PM, "Brett Cannon" <brett@python.org> wrote:
On Mon, Apr 12, 2021 at 3:01 AM Hugh Fisher <hugo.fisher@gmail.com> wrote:
Message: 1 Date: Sun, 11 Apr 2021 13:31:12 -0700 From: Barry Warsaw <barry@python.org> Subject: [Python-Dev] Re: PEP 647 Accepted
This is something the SC has been musing about, but as it’s not a
fully formed idea, I’m a little hesitant to bring it up. That said, it’s somewhat relevant: We wonder if it may be time to in a sense separate the typing syntax from Python’s regular syntax. TypeGuards are a case where if typing had more flexibility to adopt syntax that wasn’t strictly legal “normal” Python, maybe something more intuitive could have been proposed. I wonder if the typing-sig has discussed this possibility (in the future, of course)?
[ munch ]
Agreed. It’s interesting that PEP 593 proposes a different approach to
enriching the typing system. Typing itself is becoming a little ecosystem of its own, and given that many Python users are still not fully embracing typing, maybe continuing to tie the typing syntax to Python syntax is starting to strain.
I would really like to see either "Typed Python" become a different programming language, or progress to building type checking into the CPython implementation itself. (Python 4 seems to me the obvious release.) The current halfway approach is confusing and slightly ridiculous.
Please don't denigrate the hard work people have put in to even bring forward the idea of typing in Python by saying it's "slightly ridiculous".
-Brett
Aren't people allowed to have their own opinions?
Yes, of course.
Please, I hate to see this list descend further and further into such knee-jerk reactions.
It wasn't a knee-jerk reaction. I did take the time to think about replying.
If criticism of any current implementation of any construct becomes off-limits is automatically classed as "denigrating", there is no reason for this list to exist. You might not agreed with the criticism, but you should at least be open to discussion.
And I didn't try to shut down the discussion. My point was not about the intent of the message, but how that message was delivered. Being considerate and acknowledging people's time and effort is important and I don't think saying something is "ridiculous" does that. Had the sentences ended at "confusing" or said something like "I don't think it's as optimal as it could be" or "I think it could be better" are all fine. But saying that the current approach is "arousing or deserving ridicule : extremely silly or unreasonable : absurd, preposterous" as defined by Merriam-Webster <https://www.merriam-webster.com/dictionary/ridiculous> is not necessary to make the point; it could have been phrased in such a way as to be a bit more respectful to those who have put in the time and effort to get things to where they are.
On 4/12/2021 6:34 PM, Brett Cannon wrote:
Had the sentences ended at "confusing" or said something like "I don't think it's as optimal as it could be" or "I think it could be better" are all fine. But saying that the current approach is "arousing or deserving ridicule : extremely silly or unreasonable : absurd, preposterous" as defined by Merriam-Webster <https://www.merriam-webster.com/dictionary/ridiculous> is not necessary to make the point; it could have been phrased in such a way as to be a bit more respectful to those who have put in the time and effort to get things to where they are. My plea is very simple: that everyone would be a bit more gracious. Email by its very nature does not convey meaning as well as in-person conversation. As Hugh just remarked, the meaning you took from his comments was not the meaning he intended to convey. These misunderstandings seem to be happening more and more frequently. I read this list to understand the direction that Python will take in the future, but I have thought numerous times of unsubscribing due to all the curt "you said it wrong" responses like the one that triggered my first email.
Any way, in the interest of not starting a flame war, I will have nothing more to say.
The old rule is best: be strict in what you produce and liberal i what you accept. Kind regards, Steve On Tue, Apr 13, 2021 at 12:52 AM Edwin Zimmerman <edwin@211mainstreet.net> wrote:
On 4/12/2021 6:34 PM, Brett Cannon wrote:
Had the sentences ended at "confusing" or said something like "I don't think it's as optimal as it could be" or "I think it could be better" are all fine. But saying that the current approach is "arousing or deserving ridicule : extremely silly or unreasonable : absurd, preposterous" as defined by Merriam-Webster <https://www.merriam-webster.com/dictionary/ridiculous> is not necessary to make the point; it could have been phrased in such a way as to be a bit more respectful to those who have put in the time and effort to get things to where they are.
My plea is very simple: that everyone would be a bit more gracious. Email by its very nature does not convey meaning as well as in-person conversation. As Hugh just remarked, the meaning you took from his comments was not the meaning he intended to convey. These misunderstandings seem to be happening more and more frequently. I read this list to understand the direction that Python will take in the future, but I have thought numerous times of unsubscribing due to all the curt "you said it wrong" responses like the one that triggered my first email.
Any way, in the interest of not starting a flame war, I will have nothing more to say. _______________________________________________ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-leave@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/ETJCGR4U... Code of Conduct: http://python.org/psf/codeofconduct/
Hugh was unfortunate in presenting the problem, but I agree that we should commit all the way to supporting type hints, and that means bundling a type checker as part of the standard library and distribution. There is always a delay after a Python release before Mypy catches up to—and that's the type checker hosted in the python organization on github. I believe this is an unfortunate state of affairs for many users. I am not aware of any other optionally typed language that underwent core changes to support type annotations and yet does not bundle a type checker. Cheers, Luciano On Mon, Apr 12, 2021 at 7:01 AM Hugh Fisher <hugo.fisher@gmail.com> wrote:
Message: 1 Date: Sun, 11 Apr 2021 13:31:12 -0700 From: Barry Warsaw <barry@python.org> Subject: [Python-Dev] Re: PEP 647 Accepted
This is something the SC has been musing about, but as it’s not a fully formed idea, I’m a little hesitant to bring it up. That said, it’s somewhat relevant: We wonder if it may be time to in a sense separate the typing syntax from Python’s regular syntax. TypeGuards are a case where if typing had more flexibility to adopt syntax that wasn’t strictly legal “normal” Python, maybe something more intuitive could have been proposed. I wonder if the typing-sig has discussed this possibility (in the future, of course)?
[ munch ]
Agreed. It’s interesting that PEP 593 proposes a different approach to enriching the typing system. Typing itself is becoming a little ecosystem of its own, and given that many Python users are still not fully embracing typing, maybe continuing to tie the typing syntax to Python syntax is starting to strain.
I would really like to see either "Typed Python" become a different programming language, or progress to building type checking into the CPython implementation itself. (Python 4 seems to me the obvious release.) The current halfway approach is confusing and slightly ridiculous.
The first, a separate programming language, would be like RATFOR and CFront in the past and TypeScript today. Typed Python can have whatever syntax the designers want because it doesn't have to be compatible with Python, just as TypeScript is not constrained by JavaScript. A type checker translates the original Typed Python source into "dynamic" or "classic" Python for execution. (Maybe into .pyc instead of .py?)
This would mean no overhead for type checking in CPython itself. No need to contort the parser into ignoring bits of code that are, in effect, syntax checked comments. And for the typing in Python enthusiasts, you won't have to listen to people like me complaining.
The second approach is to assume that type checking in Python is useful and popular. Not with me, but I'm willing to accept that I'm in the minority and can be ignored - after all, I can still write my Python code without type annotations. If so running a type checker as a separate step, as we do at the moment, is like asking C programmers to run the preprocessor by hand.
In today's world of continuous build and integration, it seems silly to me to have a type checker read the source, scan into lexical tokens, build an abstract syntax tree, perform semantic analysis with type checking, and then throw it away before running an interpreter which reads the same source, scans into lexical tokens, builds an abstract syntax tree, and executes. On the purely pragmatic level there is an extra chance for mismatches and things to go wrong; and from an environmental viewpoint it isn't a great use of resources.
--
cheers, Hugh Fisher _______________________________________________ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-leave@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/2DMJPVE4... Code of Conduct: http://python.org/psf/codeofconduct/
-- Luciano Ramalho | Author of Fluent Python (O'Reilly, 2015) | http://shop.oreilly.com/product/0636920032519.do | Technical Principal at ThoughtWorks | Twitter: @ramalhoorg
On Apr 13, 2021, at 14:55, Luciano Ramalho <luciano@ramalho.org> wrote:
Hugh was unfortunate in presenting the problem, but I agree that we should commit all the way to supporting type hints, and that means bundling a type checker as part of the standard library and distribution.
I wouldn’t necessarily be opposed to bundling a type checker with the interpreter/stdlib, but I think there are some issues with it. Just off the top of my head (there are undoubtedly many more issues to resolve): * Which type checker would we adopt or adapt, if any? * Which parts of the typing system require more frequent release cycles? * Is there a core technology that could be put in the stdlib and still allow experimentation? * Would the type checker authors become core developers? * Do the same feature release / deprecation policies apply? I would still be opposed to requiring type hinting in Python. -Barry
On 13/04/2021 23:21, Barry Warsaw wrote:
I would still be opposed to requiring type hinting in Python.
-Barry
(Gasps in horror.) I can only hope I've misunderstood this sentence. Has it ever been even tentatively suggested that type hinting become mandatory? (What would that even mean: that the type of every function parameter or function return value must be specified?) Some of us don't use type hinting or annotations (I don't even pretend to understand what they are) and don't intend to. No offence to those who like them, carry on doing your thing. Please reassure me, someone. 😁 Rob Cliffe
_______________________________________________ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-leave@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/P7GMIJY3... Code of Conduct: http://python.org/psf/codeofconduct/
On Apr 13, 2021, at 16:35, Rob Cliffe via Python-Dev <python-dev@python.org> wrote:
(Gasps in horror.) I can only hope I've misunderstood this sentence. Has it ever been even tentatively suggested that type hinting become mandatory? (What would that even mean: that the type of every function parameter or function return value must be specified?) Some of us don't use type hinting or annotations (I don't even pretend to understand what they are) and don't intend to. No offence to those who like them, carry on doing your thing. Please reassure me, someone. 😁
Rest assured :D -Barry
On Wed, 14 Apr 2021 at 08:21, Barry Warsaw <barry@python.org> wrote:
I wouldn’t necessarily be opposed to bundling a type checker with the interpreter/stdlib, but I think there are some issues with it. Just off the top of my head (there are undoubtedly many more issues to resolve):
* Which type checker would we adopt or adapt, if any?
Mypy. This has become an implementation issue, not one of which type system to adopt. A lot of code, even in the stdlib, has been annotated but I'm not aware of multiple different annotations with different semantics or type systems being created. For example, type equivalence by name only is used in Ada (or was, it's been many years) and probably other languages. In equivalence by name, the following code would not pass the type checker. x : list[int] y : list[int] x = y # Type error But I'm not aware of anyone implementing type by name equivalence for Python, and the original PEP 483 seems to explicitly close off that possibility. Instead the assumption seems to be Java/C++ structural equivalence for types. Skimming a bunch of current type system related PEPs, I'm not seeing anything that a Java/C++ programmer would find unfamiliar. And this is probably a good thing.
* Which parts of the typing system require more frequent release cycles? * Is there a core technology that could be put in the stdlib and still allow experimentation? * Would the type checker authors become core developers? * Do the same feature release / deprecation policies apply?
No answers from me.
I would still be opposed to requiring type hinting in Python.
I'm opposed to requiring type hints on everything, I want to still be able to write x = 1 x = "hello" etc without declaring any kind of type for x. -- cheers, Hugh Fisher
On Wed, 14 Apr 2021 10:12:19 +1000 Hugh Fisher <hugo.fisher@gmail.com> wrote:
For example, type equivalence by name only is used in Ada (or was, it's been many years) and probably other languages. In equivalence by name, the following code would not pass the type checker. x : list[int] y : list[int] x = y # Type error
But I'm not aware of anyone implementing type by name equivalence for Python, and the original PEP 483 seems to explicitly close off that possibility. Instead the assumption seems to be Java/C++ structural equivalence for types.
Can you explain why you think C++ typing is based on structural equivalence? Regards Antoine.
On Tue, Apr 13, 2021 at 5:12 PM Hugh Fisher <hugo.fisher@gmail.com> wrote:
I wouldn’t necessarily be opposed to bundling a type checker with the interpreter/stdlib, but I think there are some issues with it. Just off
On Wed, 14 Apr 2021 at 08:21, Barry Warsaw <barry@python.org> wrote: the top of my head (there are undoubtedly many more issues to resolve):
* Which type checker would we adopt or adapt, if any?
Mypy. This has become an implementation issue, not one of which type system to adopt. A lot of code, even in the stdlib, has been annotated but I'm not aware of multiple different annotations with different semantics or type systems being created.
But there are feature concerns there as well, e.g. mypy and pytype offer different "extras", even if the type checkers all align on semantics (which I know they work on in the typing SIG). There's also variance in maintenance, code complexity, etc. To me, this is not a clear-cut "mypy wins" situation. And I know Luciano said mypy because it's "the type checker hosted in the python organization on github", but I don't know if the SC would approve that today (Guido brought mypy into the org during his BDFL days), and instead my guess is it would have ended up in the psf org like Black did.
For example, type equivalence by name only is used in Ada (or was, it's been many years) and probably other languages. In equivalence by name, the following code would not pass the type checker. x : list[int] y : list[int] x = y # Type error
But I'm not aware of anyone implementing type by name equivalence for Python, and the original PEP 483 seems to explicitly close off that possibility. Instead the assumption seems to be Java/C++ structural equivalence for types.
Skimming a bunch of current type system related PEPs, I'm not seeing anything that a Java/C++ programmer would find unfamiliar. And this is probably a good thing.
* Which parts of the typing system require more frequent release cycles? * Is there a core technology that could be put in the stdlib and still allow experimentation? * Would the type checker authors become core developers? * Do the same feature release / deprecation policies apply?
No answers from me.
My guess is the closest we would ever come is some ensuretypechecker situation like we have with ensurepip, but pip is done that way for bootstrapping reasons and we don't *need *a type checker for bootstrapping. -Brett
I would still be opposed to requiring type hinting in Python.
I'm opposed to requiring type hints on everything, I want to still be able to write x = 1 x = "hello" etc without declaring any kind of type for x.
--
cheers, Hugh Fisher _______________________________________________ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-leave@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/OABB53NT... Code of Conduct: http://python.org/psf/codeofconduct/
Today I stumbled upon another false positive on Mypy: https://github.com/python/mypy/issues/4660 . The issue is more than 3 years old, but it refers to a language feature (__init_subclass__) which was added in Python 3.6, released in late 2016. I understand that maintaining a type checker is a lot of work, and requires a particular set of skills, besides interest in the subject matter. So maybe the current state is the best we can offer. But I do think we should mention this situation somewhere in the official documentation. Anyone who uses a language with support for static typing expects the language distribution to include a type checker. Since that may be beyond our means, then the least we could do is have some official guidance on the matter, maybe in the form of a "Typing HOWTO" as part of the official docs, and linked from the "typing" module docs. Such a HOWTO could provide a gentle introduction to the topic, but also cover the tools—which the existing docs and PEPs avoid. It should also explain that the tools are external, as is typeshed, which means there is always a lag between new language features and libraries and the support provided by the tools. I volunteer to help with a "Typing HOWTO". For the next few months, I can offer to review if someone else writes it. In the second semester, I could write it myself, if the experts on typing and the type checkers would be willing to review it. Cheers, Luciano On Wed, Apr 14, 2021 at 2:19 PM Brett Cannon <brett@python.org> wrote:
On Tue, Apr 13, 2021 at 5:12 PM Hugh Fisher <hugo.fisher@gmail.com> wrote:
On Wed, 14 Apr 2021 at 08:21, Barry Warsaw <barry@python.org> wrote:
I wouldn’t necessarily be opposed to bundling a type checker with the interpreter/stdlib, but I think there are some issues with it. Just off the top of my head (there are undoubtedly many more issues to resolve):
* Which type checker would we adopt or adapt, if any?
Mypy. This has become an implementation issue, not one of which type system to adopt. A lot of code, even in the stdlib, has been annotated but I'm not aware of multiple different annotations with different semantics or type systems being created.
But there are feature concerns there as well, e.g. mypy and pytype offer different "extras", even if the type checkers all align on semantics (which I know they work on in the typing SIG). There's also variance in maintenance, code complexity, etc. To me, this is not a clear-cut "mypy wins" situation.
And I know Luciano said mypy because it's "the type checker hosted in the python organization on github", but I don't know if the SC would approve that today (Guido brought mypy into the org during his BDFL days), and instead my guess is it would have ended up in the psf org like Black did.
For example, type equivalence by name only is used in Ada (or was, it's been many years) and probably other languages. In equivalence by name, the following code would not pass the type checker. x : list[int] y : list[int] x = y # Type error
But I'm not aware of anyone implementing type by name equivalence for Python, and the original PEP 483 seems to explicitly close off that possibility. Instead the assumption seems to be Java/C++ structural equivalence for types.
Skimming a bunch of current type system related PEPs, I'm not seeing anything that a Java/C++ programmer would find unfamiliar. And this is probably a good thing.
* Which parts of the typing system require more frequent release cycles? * Is there a core technology that could be put in the stdlib and still allow experimentation? * Would the type checker authors become core developers? * Do the same feature release / deprecation policies apply?
No answers from me.
My guess is the closest we would ever come is some ensuretypechecker situation like we have with ensurepip, but pip is done that way for bootstrapping reasons and we don't need a type checker for bootstrapping.
-Brett
I would still be opposed to requiring type hinting in Python.
I'm opposed to requiring type hints on everything, I want to still be able to write x = 1 x = "hello" etc without declaring any kind of type for x.
--
cheers, Hugh Fisher _______________________________________________ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-leave@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/OABB53NT... Code of Conduct: http://python.org/psf/codeofconduct/
-- Luciano Ramalho | Author of Fluent Python (O'Reilly, 2015) | http://shop.oreilly.com/product/0636920032519.do | Technical Principal at ThoughtWorks | Twitter: @ramalhoorg
On Apr 15, 2021, at 17:49, Luciano Ramalho <luciano@ramalho.org> wrote:
Anyone who uses a language with support for static typing expects the language distribution to include a type checker. Since that may be beyond our means, then the least we could do is have some official guidance on the matter, maybe in the form of a "Typing HOWTO" as part of the official docs, and linked from the "typing" module docs.
Actually, I think it’s time for a comprehensive guide to type annotations. Just anecdotally, I was trying to annotate a library of mine and was having an impossible time of it, until a chat with Guido lead me to @typing.overload. That solved my problem intuitively and easily, but I just didn’t know about it. Right now, there’s information spread out all over the place, the stdlib documentation, tool documentation, StackOverflow :D etc. It’s a complicated topic that I think a comprehensive guide, a tutorial, etc. could really help with. One of my favorite frameworks for thinking about documentation on a topic such as this is: https://documentation.divio.com/ I really think that would help people get into Python type annotations, both casually and deeply.
I volunteer to help with a "Typing HOWTO". For the next few months, I can offer to review if someone else writes it. In the second semester, I could write it myself, if the experts on typing and the type checkers would be willing to review it.
I don’t know whether I’ll have time to *start* something any time soon, but I would also volunteer to be a reviewer and/or provide some content. -Barry
Am 16.04.21 um 03:21 schrieb Barry Warsaw:
Actually, I think it’s time for a comprehensive guide to type annotations. Just anecdotally, I was trying to annotate a library of mine and was having an impossible time of it, until a chat with Guido lead me to @typing.overload. That solved my problem intuitively and easily, but I just didn’t know about it. Right now, there’s information spread out all over the place, the stdlib documentation, tool documentation, StackOverflow :D etc. It’s a complicated topic that I think a comprehensive guide, a tutorial, etc. could really help with.
As a typeshed maintainer, I agree. Currently, the typing documentation is spread over various PEPs, the typing module documentation, and the mypy documentation. We also have a style guide for stub in the typeshed documentation. That said, at least for type stubs, we are working on a PEP (still looking for a sponsor) that is supposed to be a comprehensive document. [1][2] - Sebastian [1] https://mail.python.org/archives/list/typing-sig@python.org/message/VNMC3JPW... [2] https://github.com/srittau/type-stub-pep/blob/type-stub-pep/pep-9999.rst
Hi Barry & Luciano, Barry Warsaw wrote:
Actually, I think it’s time for a comprehensive guide to type annotations. Just anecdotally, I was trying to annotate a library of mine and was having an impossible time of it, until a chat with Guido lead me to @typing.overload. That solved my problem intuitively and easily, but I just didn’t know about it. Right now, there’s information spread out all over the place, the stdlib documentation, tool documentation, StackOverflow :D etc. It’s a complicated topic that I think a comprehensive guide, a tutorial, etc. could really help with. One of my favorite frameworks for thinking about documentation on a topic such as this is: https://documentation.divio.com/ I really think that would help people get into Python type annotations, both casually and deeply.
I volunteer to help with a "Typing HOWTO". For the next few months, I can offer to review if someone else writes it. In the second semester, I could write it myself, if the experts on typing and the type checkers would be willing to review it. I don’t know whether I’ll have time to *start* something any time soon, but I would also volunteer to be a reviewer and/or provide some content.
I'm also interested in helping with this. I think the first question to answer is, are the current mypy docs (https://mypy.readthedocs.io/en/stable/) insufficient for this purpose, and why? They do include both tutorial-style "getting started" paths as well as reference documentation. If they are not serving this purpose, why not? Is it due to their content or structure, or just because they are framed as "the mypy docs" and not "a typed Python HOWTO", so they don't find the right audience? If we do need to write something new, one resource I can offer is an "intro to typed Python" talk I gave at PyCon 2018: https://www.youtube.com/watch?v=pMgmKJyWKn8 I've received feedback from many people without previous experience with typing that this talk was useful to them in understanding both the why and the how. If it seems useful I'd potentially be willing to adapt and update the content and code examples from this talk into written form as a starting point. Carl
Hi Carl,
I think the first question to answer is, are the current mypy docs (https://mypy.readthedocs.io/en/stable/) insufficient for this purpose, and why?
There’s certainly lots of great documentation in the mypy docs; it’s often my first go-to.
They do include both tutorial-style "getting started" paths as well as reference documentation. If they are not serving this purpose, why not? Is it due to their content or structure, or just because they are framed as "the mypy docs" and not "a typed Python HOWTO", so they don't find the right audience?
Framing is an aspect. For example, the information on installing, configuring, and running mypy wouldn’t (IMHO) be appropriate for a comprehensive typing guide. There’s also the fact that it lives under the mypy banner rather than under docs.python.org for example. But I think mypy’s docs would make an excellent source for the guides I’m thinking about. You’d want to write the guide being tool agnostic as much as possible (though, pointing out where semantics or behavior may differ), and you’d want to have a section on “Type Checkers” to talk about the common tools, without diving into them too much.
If we do need to write something new, one resource I can offer is an "intro to typed Python" talk I gave at PyCon 2018: https://www.youtube.com/watch?v=pMgmKJyWKn8
I’ve become a huge fan of short videos as a way to help introduce and train developers on the tools that are available to them. I’ve been working on “the documentation problem” at work for the last year+ and there’s nothing like a good video to engage with developers, especially as they start their journey into a particular topic. A series of curated videos under the Python/PSF banner would be awesome. Cheers, -Barry
I see multiple problems with including a type checker as part of the standard library: First of all this would require defining precise type checking semantics and not making breaking changes to them. Currently some parts of type checking are not precisely defined and are done differently by different type checkers. Take this example: if condition: a = 1 else: a = "foo" reveal_type(a) Mypy raises an error at the second assignment and infers a as int. Pyright on the other hand doesn't report errors and infers a as Union[int, str]. Both approaches are equally valid and have their advantages and drawbacks. My second concern would be the development speed. Python type checking is still relatively young and is developing at a vastly different rate from the rest of the language. While cpython currently has a 1 year release cycle mypy had 5 releases (excluding minor releases) in the last year. This development speed difference can also be seen by the use of typing_extensions to back port typing features to older python versions. GitHub search shows 16.500 python results for "from typing_extensions" (excluding forks). Being tied to the cpython release cycle would probably significantly hinder the development of a type checker. I agree that a delay between a python release and mypy (and other type checkers) supporting it isn't optimal, but it can probably be solved much easier: By having more developers put in work to keep it up do date. This work would still need to be done, even for a type checker that's part of the standard library. The only difference would be that changes would then cause tests in cpython to fail instead of just in mypy. In the future, when development on type checkers has slowed, adding a type checker to the standard library might be useful, but in my opinion it would currently do more harm than good. Adrian Freund On April 13, 2021 11:55:05 PM GMT+02:00, Luciano Ramalho <luciano@ramalho.org> wrote:
Hugh was unfortunate in presenting the problem, but I agree that we should commit all the way to supporting type hints, and that means bundling a type checker as part of the standard library and distribution.
There is always a delay after a Python release before Mypy catches up to—and that's the type checker hosted in the python organization on github.
I believe this is an unfortunate state of affairs for many users. I am not aware of any other optionally typed language that underwent core changes to support type annotations and yet does not bundle a type checker.
Cheers,
Luciano
On Mon, Apr 12, 2021 at 7:01 AM Hugh Fisher <hugo.fisher@gmail.com> wrote:
Message: 1 Date: Sun, 11 Apr 2021 13:31:12 -0700 From: Barry Warsaw <barry@python.org> Subject: [Python-Dev] Re: PEP 647 Accepted
This is something the SC has been musing about, but as it’s not a fully formed idea, I’m a little hesitant to bring it up. That said, it’s somewhat relevant: We wonder if it may be time to in a sense separate the typing syntax from Python’s regular syntax. TypeGuards are a case where if typing had more flexibility to adopt syntax that wasn’t strictly legal “normal” Python, maybe something more intuitive could have been proposed. I wonder if the typing-sig has discussed this possibility (in the future, of course)?
[ munch ]
Agreed. It’s interesting that PEP 593 proposes a different approach to enriching the typing system. Typing itself is becoming a little ecosystem of its own, and given that many Python users are still not fully embracing typing, maybe continuing to tie the typing syntax to Python syntax is starting to strain.
I would really like to see either "Typed Python" become a different programming language, or progress to building type checking into the CPython implementation itself. (Python 4 seems to me the obvious release.) The current halfway approach is confusing and slightly ridiculous.
The first, a separate programming language, would be like RATFOR and CFront in the past and TypeScript today. Typed Python can have whatever syntax the designers want because it doesn't have to be compatible with Python, just as TypeScript is not constrained by JavaScript. A type checker translates the original Typed Python source into "dynamic" or "classic" Python for execution. (Maybe into .pyc instead of .py?)
This would mean no overhead for type checking in CPython itself. No need to contort the parser into ignoring bits of code that are, in effect, syntax checked comments. And for the typing in Python enthusiasts, you won't have to listen to people like me complaining.
The second approach is to assume that type checking in Python is useful and popular. Not with me, but I'm willing to accept that I'm in the minority and can be ignored - after all, I can still write my Python code without type annotations. If so running a type checker as a separate step, as we do at the moment, is like asking C programmers to run the preprocessor by hand.
In today's world of continuous build and integration, it seems silly to me to have a type checker read the source, scan into lexical tokens, build an abstract syntax tree, perform semantic analysis with type checking, and then throw it away before running an interpreter which reads the same source, scans into lexical tokens, builds an abstract syntax tree, and executes. On the purely pragmatic level there is an extra chance for mismatches and things to go wrong; and from an environmental viewpoint it isn't a great use of resources.
--
cheers, Hugh Fisher _______________________________________________ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-leave@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/2DMJPVE4... Code of Conduct: http://python.org/psf/codeofconduct/
-- Luciano Ramalho | Author of Fluent Python (O'Reilly, 2015) | http://shop.oreilly.com/product/0636920032519.do | Technical Principal at ThoughtWorks | Twitter: @ramalhoorg _______________________________________________ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-leave@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/FAYGAU72... Code of Conduct: http://python.org/psf/codeofconduct/
I agree that showing support for type hints is important, as long as this doesn't convey the counterproductive message that type hints have become almost mandatory. They are useful in some circumstances, and less so (specially when dealing with beginners or non-professional programmers) in others. I don't agree that mypy should become part of the standard lib. 1) It's already obvious that it's the prefered implementation (in Python - there are at least 3 other implementations that are not written in Python and/or not open source that have an impact on the market). 2) Mypy is already part of the Python organisation on GitHub, it's even one of the 6 featured repositories (which means it probably has a top-level status with the PSF). 3) But it has a release cycle that is shorter than Python. It hasn't even reached 1.0 ! 4) If we are to "commit all the way to supporting type hints", a first step IMHO would be to type hint all the standard library. AFAICT, type hints for the stdlib are currently supported in the typeshed project, which is also a top-level project. But I have not checked if it covers 100% of the stdlib. Also, one could argue that the stdlib type hints should live closer to the stdlib itself, and have a release cycle aligned with that of the stdlib, and that typeshed should be reserved for type hinting popular third-party libraries. Indeed, if there was a proposal to make, mine would be to do exactly this :) S. On Tue, Apr 13, 2021 at 11:58 PM Luciano Ramalho <luciano@ramalho.org> wrote:
Hugh was unfortunate in presenting the problem, but I agree that we should commit all the way to supporting type hints, and that means bundling a type checker as part of the standard library and distribution.
There is always a delay after a Python release before Mypy catches up to—and that's the type checker hosted in the python organization on github.
I believe this is an unfortunate state of affairs for many users. I am not aware of any other optionally typed language that underwent core changes to support type annotations and yet does not bundle a type checker.
Cheers,
Luciano
On Mon, Apr 12, 2021 at 7:01 AM Hugh Fisher <hugo.fisher@gmail.com> wrote:
Message: 1 Date: Sun, 11 Apr 2021 13:31:12 -0700 From: Barry Warsaw <barry@python.org> Subject: [Python-Dev] Re: PEP 647 Accepted
This is something the SC has been musing about, but as it’s not a
fully formed idea, I’m a little hesitant to bring it up. That said, it’s somewhat relevant: We wonder if it may be time to in a sense separate the typing syntax from Python’s regular syntax. TypeGuards are a case where if typing had more flexibility to adopt syntax that wasn’t strictly legal “normal” Python, maybe something more intuitive could have been proposed. I wonder if the typing-sig has discussed this possibility (in the future, of course)?
[ munch ]
Agreed. It’s interesting that PEP 593 proposes a different approach
to enriching the typing system. Typing itself is becoming a little ecosystem of its own, and given that many Python users are still not fully embracing typing, maybe continuing to tie the typing syntax to Python syntax is starting to strain.
I would really like to see either "Typed Python" become a different
language, or progress to building type checking into the CPython implementation itself. (Python 4 seems to me the obvious release.) The current halfway approach is confusing and slightly ridiculous.
The first, a separate programming language, would be like RATFOR and CFront in the past and TypeScript today. Typed Python can have whatever syntax
designers want because it doesn't have to be compatible with Python, just as TypeScript is not constrained by JavaScript. A type checker translates the original Typed Python source into "dynamic" or "classic" Python for execution. (Maybe into .pyc instead of .py?)
This would mean no overhead for type checking in CPython itself. No need to contort the parser into ignoring bits of code that are, in effect, syntax checked comments. And for the typing in Python enthusiasts, you won't have to
to people like me complaining.
The second approach is to assume that type checking in Python is useful and popular. Not with me, but I'm willing to accept that I'm in the minority and can be ignored - after all, I can still write my Python code without type annotations. If so running a type checker as a separate step, as we do at the moment, is like asking C programmers to run the preprocessor by hand.
In today's world of continuous build and integration, it seems silly to me to have a type checker read the source, scan into lexical tokens, build an abstract syntax tree, perform semantic analysis with type checking, and then throw it away before running an interpreter which reads the same source, scans into lexical tokens, builds an abstract syntax tree, and executes. On the purely
programming the listen pragmatic
level there is an extra chance for mismatches and things to go wrong; and from an environmental viewpoint it isn't a great use of resources.
--
cheers, Hugh Fisher _______________________________________________ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-leave@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/2DMJPVE4... Code of Conduct: http://python.org/psf/codeofconduct/
-- Luciano Ramalho | Author of Fluent Python (O'Reilly, 2015) | http://shop.oreilly.com/product/0636920032519.do | Technical Principal at ThoughtWorks | Twitter: @ramalhoorg _______________________________________________ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-leave@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/FAYGAU72... Code of Conduct: http://python.org/psf/codeofconduct/
-- Stefane Fermigier - http://fermigier.com/ - http://twitter.com/sfermigier - http://linkedin.com/in/sfermigier Founder & CEO, Abilian - Enterprise Social Software - http://www.abilian.com/ Chairman, National Council for Free & Open Source Software (CNLL) - http://cnll.fr/ Founder & Organiser, PyParis & PyData Paris - http://pyparis.org/ & http://pydata.fr/
participants (13)
-
Adrian Freund -
Antoine Pitrou -
Barry Warsaw -
Brett Cannon -
Carl Meyer -
Edwin Zimmerman -
edwin＠211mainstreet.net
-
Hugh Fisher -
Luciano Ramalho -
Rob Cliffe -
Sebastian Rittau -
Steve Holden -
Stéfane Fermigier