What are people's feelings about a DefinitelyTyped for Python?
Working at Microsoft means I get asked semi-regularly about why the Python community has not set up a [DefinitelyTyped](https://definitelytyped.org/) equivalent, and I have historically said 🤷♂️. I have now been asked this enough times that I'm asking all of you if there's a specific reason beyond lack of time and resources so I can give a more informed answer next time this question comes up? I found https://github.com/python/typeshed/issues/2491 which didn't suggest any particular issue with the idea other than time and resources. Or I guess another way to phrase the question is if someone created a site like DefinitelyTyped where things were highly automated and anyone could provide type stubs for any project on PyPI -- and maybe even automated initial type stubs -- would people be interested in such a thing or see any reasons why people would balk at it or not use it?
I would love it if something like DefinitelyTyped existed for Python! It would be a great step forward for usability. It's unclear to me right now how to find types for third party libraries other than by googling "<library> type stubs" or by generating them myself, which is what I imagine most people are doing. It would be nice if we could share our work more easily. On Mon, Oct 21, 2019 at 7:49 PM Brett Cannon <brett@python.org> wrote:
Working at Microsoft means I get asked semi-regularly about why the Python community has not set up a [DefinitelyTyped](https://definitelytyped.org/) equivalent, and I have historically said 🤷♂️. I have now been asked this enough times that I'm asking all of you if there's a specific reason beyond lack of time and resources so I can give a more informed answer next time this question comes up? I found https://github.com/python/typeshed/issues/2491 which didn't suggest any particular issue with the idea other than time and resources.
Or I guess another way to phrase the question is if someone created a site like DefinitelyTyped where things were highly automated and anyone could provide type stubs for any project on PyPI -- and maybe even automated initial type stubs -- would people be interested in such a thing or see any reasons why people would balk at it or not use it? _______________________________________________ Typing-sig mailing list -- typing-sig@python.org To unsubscribe send an email to typing-sig-leave@python.org https://mail.python.org/mailman3/lists/typing-sig.python.org/
Typeshed does exist and is kind of like DefinitelyTyped, although the coverage is of course a lot less. We've discussed DefinitelyTyped at a couple of recent typing meetups and there's a consensus that it would be good to make typeshed more like DefinitelyTyped, but as Brett said we haven't been able to find time to do the automation/tooling work to get that off the ground. Contributions are welcome! El lun., 21 oct. 2019 a las 17:25, Cohen Karnell (<cohen@trialspark.com>) escribió:
I would love it if something like DefinitelyTyped existed for Python! It would be a great step forward for usability. It's unclear to me right now how to find types for third party libraries other than by googling "<library> type stubs" or by generating them myself, which is what I imagine most people are doing. It would be nice if we could share our work more easily.
On Mon, Oct 21, 2019 at 7:49 PM Brett Cannon <brett@python.org> wrote:
Working at Microsoft means I get asked semi-regularly about why the Python community has not set up a [DefinitelyTyped]( https://definitelytyped.org/) equivalent, and I have historically said 🤷♂️. I have now been asked this enough times that I'm asking all of you if there's a specific reason beyond lack of time and resources so I can give a more informed answer next time this question comes up? I found https://github.com/python/typeshed/issues/2491 which didn't suggest any particular issue with the idea other than time and resources.
Or I guess another way to phrase the question is if someone created a site like DefinitelyTyped where things were highly automated and anyone could provide type stubs for any project on PyPI -- and maybe even automated initial type stubs -- would people be interested in such a thing or see any reasons why people would balk at it or not use it? _______________________________________________ Typing-sig mailing list -- typing-sig@python.org To unsubscribe send an email to typing-sig-leave@python.org https://mail.python.org/mailman3/lists/typing-sig.python.org/
_______________________________________________ Typing-sig mailing list -- typing-sig@python.org To unsubscribe send an email to typing-sig-leave@python.org https://mail.python.org/mailman3/lists/typing-sig.python.org/
I will say in my discussions with the DefinitelyTyped folks as I recall, I got the understanding that they still reviewed the code to a degree. I think it's best to do this, as we would want a certain degree of trust in the project. Otherwise I think it is essentially a task of making the automation happen and defining the workflow. On Mon, Oct 21, 2019, 6:38 PM Jelle Zijlstra <jelle.zijlstra@gmail.com> wrote:
Typeshed does exist and is kind of like DefinitelyTyped, although the coverage is of course a lot less. We've discussed DefinitelyTyped at a couple of recent typing meetups and there's a consensus that it would be good to make typeshed more like DefinitelyTyped, but as Brett said we haven't been able to find time to do the automation/tooling work to get that off the ground. Contributions are welcome!
El lun., 21 oct. 2019 a las 17:25, Cohen Karnell (<cohen@trialspark.com>) escribió:
I would love it if something like DefinitelyTyped existed for Python! It would be a great step forward for usability. It's unclear to me right now how to find types for third party libraries other than by googling "<library> type stubs" or by generating them myself, which is what I imagine most people are doing. It would be nice if we could share our work more easily.
On Mon, Oct 21, 2019 at 7:49 PM Brett Cannon <brett@python.org> wrote:
Working at Microsoft means I get asked semi-regularly about why the Python community has not set up a [DefinitelyTyped]( https://definitelytyped.org/) equivalent, and I have historically said 🤷♂️. I have now been asked this enough times that I'm asking all of you if there's a specific reason beyond lack of time and resources so I can give a more informed answer next time this question comes up? I found https://github.com/python/typeshed/issues/2491 which didn't suggest any particular issue with the idea other than time and resources.
Or I guess another way to phrase the question is if someone created a site like DefinitelyTyped where things were highly automated and anyone could provide type stubs for any project on PyPI -- and maybe even automated initial type stubs -- would people be interested in such a thing or see any reasons why people would balk at it or not use it? _______________________________________________ Typing-sig mailing list -- typing-sig@python.org To unsubscribe send an email to typing-sig-leave@python.org https://mail.python.org/mailman3/lists/typing-sig.python.org/
_______________________________________________ Typing-sig mailing list -- typing-sig@python.org To unsubscribe send an email to typing-sig-leave@python.org https://mail.python.org/mailman3/lists/typing-sig.python.org/
_______________________________________________ Typing-sig mailing list -- typing-sig@python.org To unsubscribe send an email to typing-sig-leave@python.org https://mail.python.org/mailman3/lists/typing-sig.python.org/
Am 22.10.19 um 01:48 schrieb Brett Cannon:
Working at Microsoft means I get asked semi-regularly about why the Python community has not set up a [DefinitelyTyped](https://definitelytyped.org/) equivalent, and I have historically said 🤷♂️. I have now been asked this enough times that I'm asking all of you if there's a specific reason beyond lack of time and resources so I can give a more informed answer next time this question comes up? I found https://github.com/python/typeshed/issues/2491 which didn't suggest any particular issue with the idea other than time and resources.
Or I guess another way to phrase the question is if someone created a site like DefinitelyTyped where things were highly automated and anyone could provide type stubs for any project on PyPI -- and maybe even automated initial type stubs -- would people be interested in such a thing or see any reasons why people would balk at it or not use it?
It's just time and resources. Work on this has already started in the third-party-dist branch in the typeshed repository, but the main blocker in my opinion is the lack of support for namespaces in pypi/warehouse. Without it we are opening a significant security hole, because people will expect to be able to just call "pip install types.whatever" and get the typings, so it's very easy to name-squat and run malicious code on a developer's machine. (https://github.com/pypa/warehouse/issues/4967) I looked into adding support myself, but failed even setting up a development environment. I don't think we should start a separate project. As Ethan points out, we still want a certain amount of review, especially since typing is fairly complex and there are many pitfalls. (https://github.com/srittau/type-stub-pep is aimed to provide better documentation for this when we actually get around to finish it.) But we could certainly use more people involved in typeshed and having a decent website for the project would also be quite useful. - Sebastian
On Tue, 22 Oct 2019 at 09:53, Sebastian Rittau <srittau@rittau.biz> wrote:
Working at Microsoft means I get asked semi-regularly about why the Python community has not set up a [DefinitelyTyped]( https://definitelytyped.org/) equivalent, and I have historically said 🤷♂️. I have now been asked this enough times that I'm asking all of you if there's a specific reason beyond lack of time and resources so I can give a more informed answer next time this question comes up? I found https://github.com/python/typeshed/issues/2491 which didn't suggest any
Am 22.10.19 um 01:48 schrieb Brett Cannon: particular issue with the idea other than time and resources.
Or I guess another way to phrase the question is if someone created a
site like DefinitelyTyped where things were highly automated and anyone could provide type stubs for any project on PyPI -- and maybe even automated initial type stubs -- would people be interested in such a thing or see any reasons why people would balk at it or not use it?
It's just time and resources. Work on this has already started in the third-party-dist branch in the typeshed repository, but the main blocker in my opinion is the lack of support for namespaces in pypi/warehouse.
Also IIUC Jukka (cc'ed) is going to spend a week or two early November working 100% on kick-starting the transformation of typeshed to a more modular model. -- Ivan
Is something going to change wrt accepting new stubs in TypeShed (see https://github.com/python/typeshed/issues/2440) and requiring approval from package owners ( https://github.com/python/typeshed/blob/master/CONTRIBUTING.md#the-contribut... ). On Tue, Oct 22, 2019 at 5:59 AM Ivan Levkivskyi <levkivskyi@gmail.com> wrote:
On Tue, 22 Oct 2019 at 09:53, Sebastian Rittau <srittau@rittau.biz> wrote:
Working at Microsoft means I get asked semi-regularly about why the Python community has not set up a [DefinitelyTyped]( https://definitelytyped.org/) equivalent, and I have historically said 🤷♂️. I have now been asked this enough times that I'm asking all of you if there's a specific reason beyond lack of time and resources so I can give a more informed answer next time this question comes up? I found https://github.com/python/typeshed/issues/2491 which didn't suggest any
Am 22.10.19 um 01:48 schrieb Brett Cannon: particular issue with the idea other than time and resources.
Or I guess another way to phrase the question is if someone created a
site like DefinitelyTyped where things were highly automated and anyone could provide type stubs for any project on PyPI -- and maybe even automated initial type stubs -- would people be interested in such a thing or see any reasons why people would balk at it or not use it?
It's just time and resources. Work on this has already started in the third-party-dist branch in the typeshed repository, but the main blocker in my opinion is the lack of support for namespaces in pypi/warehouse.
Also IIUC Jukka (cc'ed) is going to spend a week or two early November working 100% on kick-starting the transformation of typeshed to a more modular model.
-- Ivan
_______________________________________________ Typing-sig mailing list -- typing-sig@python.org To unsubscribe send an email to typing-sig-leave@python.org https://mail.python.org/mailman3/lists/typing-sig.python.org/
-- Sebastian Kreft
Am 22.10.19 um 15:06 schrieb Sebastian Kreft:
Is something going to change wrt accepting new stubs in TypeShed (see https://github.com/python/typeshed/issues/2440) and requiring approval from package owners (https://github.com/python/typeshed/blob/master/CONTRIBUTING.md#the-contribut...).
At the moment no changes are planned. We would still accept third-party stubs (after all, easier availability of those stubs is the goal of this project), but it still makes sense to contact the maintainer of the module first. Of course, maintainers are still encouraged to ship stubs with their own packages, if they are willing and able to support them. This is preferable to shipping stubs as third-party packages. - Sebastian
Sebastian Kreft wrote:
Is something going to change wrt accepting new stubs in TypeShed (see https://github.com/python/typeshed/issues/2440) and requiring approval from package owners ( https://github.com/python/typeshed/blob/master/CONTRIBUTING.md#the-contribut... ).
This is actually the biggest driver of people asking me about a DefinitelyTyped equivalent for Python since approval from the project receiving the type hints is not required on DefinitelyTyped which some around here believe has allowed it to flourish while being a detriment to contributions to typeshed (whether that worry is founded or not).
On Tue, Oct 22, 2019 at 5:59 AM Ivan Levkivskyi levkivskyi@gmail.com wrote:
On Tue, 22 Oct 2019 at 09:53, Sebastian Rittau srittau@rittau.biz wrote: Am 22.10.19 um 01:48 schrieb Brett Cannon: Working at Microsoft means I get asked semi-regularly about why the Python community has not set up a DefinitelyTyped equivalent, and I have historically said 🤷♂️. I have now been asked this enough times that I'm asking all of you if there's a specific reason beyond lack of time and resources so I can give a more informed answer next time this question comes up? I found https://github.com/python/typeshed/issues/2491 which didn't suggest any particular issue with the idea other than time and resources. Or I guess another way to phrase the question is if someone created a site like DefinitelyTyped where things were highly automated and anyone could provide type stubs for any project on PyPI -- and maybe even automated initial type stubs -- would people be interested in such a thing or see any reasons why people would balk at it or not use it? It's just time and resources. Work on this has already started in the third-party-dist branch in the typeshed repository, but the main blocker in my opinion is the lack of support for namespaces in pypi/warehouse. Also IIUC Jukka (cc'ed) is going to spend a week or two early November working 100% on kick-starting the transformation of typeshed to a more modular model. -- Ivan
Typing-sig mailing list -- typing-sig@python.org To unsubscribe send an email to typing-sig-leave@python.org https://mail.python.org/mailman3/lists/typing-sig.python.org/ -- Sebastian Kreft
Am 29.10.19 um 18:29 schrieb Brett Cannon:
Sebastian Kreft wrote:
Is something going to change wrt accepting new stubs in TypeShed (see https://github.com/python/typeshed/issues/2440) and requiring approval from package owners ( https://github.com/python/typeshed/blob/master/CONTRIBUTING.md#the-contribut... ). This is actually the biggest driver of people asking me about a DefinitelyTyped equivalent for Python since approval from the project receiving the type hints is not required on DefinitelyTyped which some around here believe has allowed it to flourish while being a detriment to contributions to typeshed (whether that worry is founded or not).
We have already relaxed the approval requirement a bit and also accept stubs when the submitter has not received a response within a month. That said, I would personally prefer to swap the approval requirement to an information requirement. This gives package maintainers the chance to react to a submission and give their feedback, but will lower the bar for inclusion. - Sebastian
El mar., 29 oct. 2019 a las 10:54, Sebastian Rittau (<srittau@rittau.biz>) escribió:
Am 29.10.19 um 18:29 schrieb Brett Cannon:
Sebastian Kreft wrote:
Is something going to change wrt accepting new stubs in TypeShed (see https://github.com/python/typeshed/issues/2440) and requiring approval from package owners (
https://github.com/python/typeshed/blob/master/CONTRIBUTING.md#the-contribut. ..
). This is actually the biggest driver of people asking me about a DefinitelyTyped equivalent for Python since approval from the project receiving the type hints is not required on DefinitelyTyped which some around here believe has allowed it to flourish while being a detriment to contributions to typeshed (whether that worry is founded or not).
We have already relaxed the approval requirement a bit and also accept stubs when the submitter has not received a response within a month. That said, I would personally prefer to swap the approval requirement to an information requirement. This gives package maintainers the chance to react to a submission and give their feedback, but will lower the bar for inclusion.
I'm supportive of making a change like this, so we can ensure the barrier for new contributors to typeshed is as low as possible.
- Sebastian _______________________________________________ Typing-sig mailing list -- typing-sig@python.org To unsubscribe send an email to typing-sig-leave@python.org https://mail.python.org/mailman3/lists/typing-sig.python.org/
On Tue, Oct 29, 2019 at 10:58 AM Jelle Zijlstra <jelle.zijlstra@gmail.com> wrote:
El mar., 29 oct. 2019 a las 10:54, Sebastian Rittau (<srittau@rittau.biz>) escribió:
Am 29.10.19 um 18:29 schrieb Brett Cannon:
Sebastian Kreft wrote:
Is something going to change wrt accepting new stubs in TypeShed (see https://github.com/python/typeshed/issues/2440) and requiring approval from package owners (
https://github.com/python/typeshed/blob/master/CONTRIBUTING.md#the-contribut. ..
). This is actually the biggest driver of people asking me about a DefinitelyTyped equivalent for Python since approval from the project receiving the type hints is not required on DefinitelyTyped which some around here believe has allowed it to flourish while being a detriment to contributions to typeshed (whether that worry is founded or not).
We have already relaxed the approval requirement a bit and also accept stubs when the submitter has not received a response within a month. That said, I would personally prefer to swap the approval requirement to an information requirement. This gives package maintainers the chance to react to a submission and give their feedback, but will lower the bar for inclusion.
I'm supportive of making a change like this, so we can ensure the barrier for new contributors to typeshed is as low as possible.
OK, make a PR that updates PEP 484 to change the requirement into information only, and I'll get the SC to approve it. -- --Guido van Rossum (python.org/~guido) *Pronouns: he/him **(why is my pronoun here?)* <http://feministing.com/2015/02/03/how-using-they-as-a-singular-pronoun-can-c...>
El mar., 29 oct. 2019 a las 11:13, Guido van Rossum (<guido@python.org>) escribió:
On Tue, Oct 29, 2019 at 10:58 AM Jelle Zijlstra <jelle.zijlstra@gmail.com> wrote:
El mar., 29 oct. 2019 a las 10:54, Sebastian Rittau (<srittau@rittau.biz>) escribió:
Am 29.10.19 um 18:29 schrieb Brett Cannon:
Sebastian Kreft wrote:
Is something going to change wrt accepting new stubs in TypeShed (see https://github.com/python/typeshed/issues/2440) and requiring approval from package owners (
https://github.com/python/typeshed/blob/master/CONTRIBUTING.md#the-contribut. ..
). This is actually the biggest driver of people asking me about a DefinitelyTyped equivalent for Python since approval from the project receiving the type hints is not required on DefinitelyTyped which some around here believe has allowed it to flourish while being a detriment to contributions to typeshed (whether that worry is founded or not).
We have already relaxed the approval requirement a bit and also accept stubs when the submitter has not received a response within a month. That said, I would personally prefer to swap the approval requirement to an information requirement. This gives package maintainers the chance to react to a submission and give their feedback, but will lower the bar for inclusion.
I'm supportive of making a change like this, so we can ensure the barrier for new contributors to typeshed is as low as possible.
OK, make a PR that updates PEP 484 to change the requirement into information only, and I'll get the SC to approve it.
-- --Guido van Rossum (python.org/~guido) *Pronouns: he/him **(why is my pronoun here?)* <http://feministing.com/2015/02/03/how-using-they-as-a-singular-pronoun-can-c...>
With Jelle's change in, do we want to potentially promote this change in policy to see if it causes more contributions to come in?
Well, to begin we should update typeshed's README.md, which still states NOTE: When you're contributing a new stub for a package that you did not develop, please obtain consent of the package owner (this is specified in [PEP 484](https://www.python.org/dev/peps/pep-0484/#the-typeshed-repo)). The best way to obtain consent is to file an issue in the third-party package's tracker and include the link to a positive response in your PR for typeshed. After that's fixed, we can start a social media campaign, e.g. the PSF and some popular core devs could tweet about this. Maybe we could produce a blog post explaining how easy it is to create stubs from an installed package and copy those into typeshed? (IIRC Jukka is a little hesitant until some issues with stubgen have been fixed; personally I think we shouldn't wait for perfection.) On Wed, Nov 6, 2019 at 4:54 PM Brett Cannon <brett@python.org> wrote:
With Jelle's change in, do we want to potentially promote this change in policy to see if it causes more contributions to come in? _______________________________________________ Typing-sig mailing list -- typing-sig@python.org To unsubscribe send an email to typing-sig-leave@python.org https://mail.python.org/mailman3/lists/typing-sig.python.org/
-- --Guido van Rossum (python.org/~guido) *Pronouns: he/him **(why is my pronoun here?)* <http://feministing.com/2015/02/03/how-using-they-as-a-singular-pronoun-can-c...>
El mié., 6 nov. 2019 a las 17:09, Guido van Rossum (<guido@python.org>) escribió:
Well, to begin we should update typeshed's README.md, which still states
NOTE: When you're contributing a new stub for a package that you did not develop, please obtain consent of the package owner (this is specified in [PEP 484](https://www.python.org/dev/peps/pep-0484/#the-typeshed-repo)). The best way to obtain consent is to file an issue in the third-party package's tracker and include the link to a positive response in your PR for typeshed.
After that's fixed, we can start a social media campaign, e.g. the PSF and some popular core devs could tweet about this. Maybe we could produce a blog post explaining how easy it is to create stubs from an installed package and copy those into typeshed? (IIRC Jukka is a little hesitant until some issues with stubgen have been fixed; personally I think we shouldn't wait for perfection.)
Sebastian has opened a PR to update the README: https://github.com/python/typeshed/pull/3443
On Wed, Nov 6, 2019 at 4:54 PM Brett Cannon <brett@python.org> wrote:
With Jelle's change in, do we want to potentially promote this change in policy to see if it causes more contributions to come in? _______________________________________________ Typing-sig mailing list -- typing-sig@python.org To unsubscribe send an email to typing-sig-leave@python.org https://mail.python.org/mailman3/lists/typing-sig.python.org/
-- --Guido van Rossum (python.org/~guido) *Pronouns: he/him **(why is my pronoun here?)* <http://feministing.com/2015/02/03/how-using-they-as-a-singular-pronoun-can-c...> _______________________________________________ Typing-sig mailing list -- typing-sig@python.org To unsubscribe send an email to typing-sig-leave@python.org https://mail.python.org/mailman3/lists/typing-sig.python.org/
On Thu, Nov 7, 2019 at 1:09 AM Guido van Rossum <guido@python.org> wrote:
After that's fixed, we can start a social media campaign, e.g. the PSF and some popular core devs could tweet about this. Maybe we could produce a blog post explaining how easy it is to create stubs from an installed package and copy those into typeshed? (IIRC Jukka is a little hesitant until some issues with stubgen have been fixed; personally I think we shouldn't wait for perfection.)
FWIW, the next mypy release (in two weeks maybe) should have big improvements to stubgen. I'm going to prepare PRs this week. I can do a quick experiment to estimate how much better stubgen will be after the improvements, and you can decide if it looks substantial enough to postpone the social media campaign. Jukka
On Tue, Oct 29, 2019 at 10:29 AM Brett Cannon <brett@python.org> wrote:
Sebastian Kreft wrote:
Is something going to change wrt accepting new stubs in TypeShed (see https://github.com/python/typeshed/issues/2440) and requiring approval from package owners (
https://github.com/python/typeshed/blob/master/CONTRIBUTING.md#the-contribut. ..
).
This is actually the biggest driver of people asking me about a DefinitelyTyped equivalent for Python since approval from the project receiving the type hints is not required on DefinitelyTyped which some around here believe has allowed it to flourish while being a detriment to contributions to typeshed (whether that worry is founded or not).
Then why didn't you say so? :-) That rule was proposed as a compromise to make package authors less worried about type annotations they weren't ready for. AFAIK we've never had a case where a package refused to allow stubs to be added to typeshed. And type annotations are much more accepted now than when PEP 484 and typeshed were first introduced. So maybe we can just drop this requirement? I believe we could ask the Steering Council to rule on this. It would require a PEP change, since PEP 484 has this on the topic: The Typeshed Repo ----------------- There is a shared repository where useful stubs are being collected [typeshed]_. Note that stubs for a given package will not be included here without the explicit consent of the package owner. Further policies regarding the stubs collected here will be decided at a later time, after discussion on python-dev, and reported in the typeshed repo's README. We could just propose to scratch the sentence "Note that stubs for a given package will not be included here without the explicit consent of the package owner." -- --Guido van Rossum (python.org/~guido) *Pronouns: he/him **(why is my pronoun here?)* <http://feministing.com/2015/02/03/how-using-they-as-a-singular-pronoun-can-c...>
Guido van Rossum wrote:
On Tue, Oct 29, 2019 at 10:29 AM Brett Cannon brett@python.org wrote:
Sebastian Kreft wrote: Is something going to change wrt accepting new stubs in TypeShed (see https://github.com/python/typeshed/issues/2440) and requiring approval from package owners ( https://github.com/python/typeshed/blob/master/CONTRIBUTING.md#the-contribut. .. ). This is actually the biggest driver of people asking me about a DefinitelyTyped equivalent for Python since approval from the project receiving the type hints is not required on DefinitelyTyped which some around here believe has allowed it to flourish while being a detriment to contributions to typeshed (whether that worry is founded or not). Then why didn't you say so? :-)
Because I don't know about the history of typeshed enough to be able to refute or validate it, plus not everyone who has an opinion has an informed one. ;) The other driver is simply people not understanding why the Python community hasn't embraced typing across the board like it's a required thing everywhere (as I said, not all opinions are informed).
That rule was proposed as a compromise to make package authors less worried about type annotations they weren't ready for. AFAIK we've never had a case where a package refused to allow stubs to be added to typeshed. And type annotations are much more accepted now than when PEP 484 and typeshed were first introduced. So maybe we can just drop this requirement? I believe we could ask the Steering Council to rule on this.
Fine by me. :) I'll add it to the agenda for today.
It would require a PEP change, since PEP 484 has this on the topic: The Typeshed Repo There is a shared repository where useful stubs are being collected [typeshed]_. Note that stubs for a given package will not be included here without the explicit consent of the package owner. Further policies regarding the stubs collected here will be decided at a later time, after discussion on python-dev, and reported in the typeshed repo's README. We could just propose to scratch the sentence "Note that stubs for a given package will not be included here without the explicit consent of the package owner."
👍
On Tue, Oct 22, 2019 at 9:59 AM Ivan Levkivskyi <levkivskyi@gmail.com> wrote:
Also IIUC Jukka (cc'ed) is going to spend a week or two early November working 100% on kick-starting the transformation of typeshed to a more modular model.
I'm probably going to work at least on improving stubgen so that it can more reliably generate draft stubs for most third-party packages. Another small project would be to add support for writing tests for stubs in typeshed. These should make it easier to create baseline stubs for many additional third-party packages and to ensure that they are not quite broken. I may also look at automated uploading of stub packages. Perhaps we can work around the lack of pypi namespaces by writing some additional tooling. A strawman proposal would be to embed a cryptographic signature in packages uploaded from typeshed and have a tool that downloads the stubs and verifies the signature in the stub package. This could be partially integrated to tools like mypy so that it could suggest how to safely download stubs if some stub seems to be missing. Jukka
Jukka Lehtosalo wrote:
On Tue, Oct 22, 2019 at 9:59 AM Ivan Levkivskyi levkivskyi@gmail.com wrote:
Also IIUC Jukka (cc'ed) is going to spend a week or two early November working 100% on kick-starting the transformation of typeshed to a more modular model.
That's great to hear! Is any of this outlined/planned somewhere? Or is this all in your head, Jukka? ;)
I'm probably going to work at least on improving stubgen so that it can more reliably generate draft stubs for most third-party packages. Another small project would be to add support for writing tests for stubs in typeshed. These should make it easier to create baseline stubs for many additional third-party packages and to ensure that they are not quite broken.
Yeah, when I thought about the dream scenario the one that came to mind would be to check projects for `py.typed` and if they lacked it and a `-stubs` project to then generate the stubs for the project as best effort. And then if someone came along and contributed a better, tighter set of stubs to then test that they would be valid to the best of everyone's knowledge somehow in an automated fashion. That way short of needing to step in for spam purposes, things could more-or-less be optimized.
I may also look at automated uploading of stub packages. Perhaps we can work around the lack of pypi namespaces by writing some additional tooling. A strawman proposal would be to embed a cryptographic signature in packages uploaded from typeshed and have a tool that downloads the stubs and verifies the signature in the stub package. This could be partially integrated to tools like mypy so that it could suggest how to safely download stubs if some stub seems to be missing.
Probably whatever PyPI has set up would work. One thing I have always wondered about is how typeshed handles package versions? It's currently a completely flat namespace with no inherent versioning, so when a major version for a package comes out that changes the types of things in a non-backwards-compatible fashion, how are projects supposed to express that for type stubs on typeshed (or in their `-stubs` packages on PyPI for that matter)?
On Tue, Oct 29, 2019 at 10:50 AM Brett Cannon <brett@python.org> wrote:
Jukka Lehtosalo wrote:
On Tue, Oct 22, 2019 at 9:59 AM Ivan Levkivskyi levkivskyi@gmail.com wrote:
Also IIUC Jukka (cc'ed) is going to spend a week or two early November working 100% on kick-starting the transformation of typeshed to a more modular model.
That's great to hear! Is any of this outlined/planned somewhere? Or is this all in your head, Jukka? ;)
I'm probably going to work at least on improving stubgen so that it can more reliably generate draft stubs for most third-party packages. Another small project would be to add support for writing tests for stubs in typeshed. These should make it easier to create baseline stubs for many additional third-party packages and to ensure that they are not quite broken.
Yeah, when I thought about the dream scenario the one that came to mind would be to check projects for `py.typed` and if they lacked it and a `-stubs` project to then generate the stubs for the project as best effort. And then if someone came along and contributed a better, tighter set of stubs to then test that they would be valid to the best of everyone's knowledge somehow in an automated fashion. That way short of needing to step in for spam purposes, things could more-or-less be optimized.
I may also look at automated uploading of stub packages. Perhaps we can work around the lack of pypi namespaces by writing some additional tooling. A strawman proposal would be to embed a cryptographic signature in packages uploaded from typeshed and have a tool that downloads the stubs and verifies the signature in the stub package. This could be partially integrated to tools like mypy so that it could suggest how to safely download stubs if some stub seems to be missing.
Probably whatever PyPI has set up would work.
One thing I have always wondered about is how typeshed handles package versions? It's currently a completely flat namespace with no inherent versioning, so when a major version for a package comes out that changes the types of things in a non-backwards-compatible fashion, how are projects supposed to express that for type stubs on typeshed (or in their `-stubs` packages on PyPI for that matter)?
PEP 561 has the following on this topic: In addition, stub-only distributions SHOULD indicate which version(s) of the runtime package are supported by indicating the runtime distribution's version(s) through normal dependency data. For example, the stub package ``flyingcircus-stubs`` can indicate the versions of the runtime ``flyingcircus`` distribution it supports through ``install_requires`` in distutils-based tools, or the equivalent in other packaging tools. [...] So if we start having stub-only packages in typeshed (like DefinitelyTyped, and per Jukka's plans) we should do it the same way. -- --Guido van Rossum (python.org/~guido) *Pronouns: he/him **(why is my pronoun here?)* <http://feministing.com/2015/02/03/how-using-they-as-a-singular-pronoun-can-c...>
On Tue, Oct 29, 2019 at 10:55 AM Guido van Rossum <guido@python.org> wrote:
On Tue, Oct 29, 2019 at 10:50 AM Brett Cannon <brett@python.org> wrote:
Jukka Lehtosalo wrote:
On Tue, Oct 22, 2019 at 9:59 AM Ivan Levkivskyi levkivskyi@gmail.com wrote:
Also IIUC Jukka (cc'ed) is going to spend a week or two early November working 100% on kick-starting the transformation of typeshed to a more modular model.
That's great to hear! Is any of this outlined/planned somewhere? Or is this all in your head, Jukka? ;)
I'm probably going to work at least on improving stubgen so that it can more reliably generate draft stubs for most third-party packages. Another small project would be to add support for writing tests for stubs in typeshed. These should make it easier to create baseline stubs for many additional third-party packages and to ensure that they are not quite broken.
Yeah, when I thought about the dream scenario the one that came to mind would be to check projects for `py.typed` and if they lacked it and a `-stubs` project to then generate the stubs for the project as best effort. And then if someone came along and contributed a better, tighter set of stubs to then test that they would be valid to the best of everyone's knowledge somehow in an automated fashion. That way short of needing to step in for spam purposes, things could more-or-less be optimized.
I may also look at automated uploading of stub packages. Perhaps we can work around the lack of pypi namespaces by writing some additional tooling. A strawman proposal would be to embed a cryptographic signature in packages uploaded from typeshed and have a tool that downloads the stubs and verifies the signature in the stub package. This could be partially integrated to tools like mypy so that it could suggest how to safely download stubs if some stub seems to be missing.
Probably whatever PyPI has set up would work.
One thing I have always wondered about is how typeshed handles package versions? It's currently a completely flat namespace with no inherent versioning, so when a major version for a package comes out that changes the types of things in a non-backwards-compatible fashion, how are projects supposed to express that for type stubs on typeshed (or in their `-stubs` packages on PyPI for that matter)?
PEP 561 has the following on this topic:
In addition, stub-only distributions SHOULD indicate which version(s) of the runtime package are supported by indicating the runtime distribution's version(s) through normal dependency data. For example, the stub package ``flyingcircus-stubs`` can indicate the versions of the runtime ``flyingcircus`` distribution it supports through ``install_requires`` in distutils-based tools, or the equivalent in other packaging tools. [...]
So if we start having stub-only packages in typeshed (like DefinitelyTyped, and per Jukka's plans) we should do it the same way.
So how would that work if you wanted to provide stubs for older project versions? Let's say for the deadparrot project there's types for version 40 (last version with Python 2 support) and version 42 (latest release), but those stubs are not compatible with each other (e.g. they went nuts with replacing string constants with enums in the latest release). How could stubs be provided for both versions simultaneously on typeshed without having to do a union solution of a single release that weakens the type details? On PyPI there's implicit version support so the stubs could be version-matched to the deadparrot project or be versions 1 and 2, but from what I can tell typeshed doesn't have a mechanism to provide two separate sets of stubs for the same project.
El mar., 29 oct. 2019 a las 11:46, Brett Cannon (<brett@python.org>) escribió:
On Tue, Oct 29, 2019 at 10:55 AM Guido van Rossum <guido@python.org> wrote:
On Tue, Oct 29, 2019 at 10:50 AM Brett Cannon <brett@python.org> wrote:
Jukka Lehtosalo wrote:
On Tue, Oct 22, 2019 at 9:59 AM Ivan Levkivskyi levkivskyi@gmail.com wrote:
Also IIUC Jukka (cc'ed) is going to spend a week or two early November working 100% on kick-starting the transformation of typeshed to a more modular model.
That's great to hear! Is any of this outlined/planned somewhere? Or is this all in your head, Jukka? ;)
I'm probably going to work at least on improving stubgen so that it can more reliably generate draft stubs for most third-party packages. Another small project would be to add support for writing tests for stubs in typeshed. These should make it easier to create baseline stubs for many additional third-party packages and to ensure that they are not quite broken.
Yeah, when I thought about the dream scenario the one that came to mind would be to check projects for `py.typed` and if they lacked it and a `-stubs` project to then generate the stubs for the project as best effort. And then if someone came along and contributed a better, tighter set of stubs to then test that they would be valid to the best of everyone's knowledge somehow in an automated fashion. That way short of needing to step in for spam purposes, things could more-or-less be optimized.
I may also look at automated uploading of stub packages. Perhaps we can work around the lack of pypi namespaces by writing some additional tooling. A strawman proposal would be to embed a cryptographic signature in packages uploaded from typeshed and have a tool that downloads the stubs and verifies the signature in the stub package. This could be partially integrated to tools like mypy so that it could suggest how to safely download stubs if some stub seems to be missing.
Probably whatever PyPI has set up would work.
One thing I have always wondered about is how typeshed handles package versions? It's currently a completely flat namespace with no inherent versioning, so when a major version for a package comes out that changes the types of things in a non-backwards-compatible fashion, how are projects supposed to express that for type stubs on typeshed (or in their `-stubs` packages on PyPI for that matter)?
PEP 561 has the following on this topic:
In addition, stub-only distributions SHOULD indicate which version(s) of the runtime package are supported by indicating the runtime distribution's version(s) through normal dependency data. For example, the stub package ``flyingcircus-stubs`` can indicate the versions of the runtime ``flyingcircus`` distribution it supports through ``install_requires`` in distutils-based tools, or the equivalent in other packaging tools. [...]
So if we start having stub-only packages in typeshed (like DefinitelyTyped, and per Jukka's plans) we should do it the same way.
So how would that work if you wanted to provide stubs for older project versions? Let's say for the deadparrot project there's types for version 40 (last version with Python 2 support) and version 42 (latest release), but those stubs are not compatible with each other (e.g. they went nuts with replacing string constants with enums in the latest release). How could stubs be provided for both versions simultaneously on typeshed without having to do a union solution of a single release that weakens the type details? On PyPI there's implicit version support so the stubs could be version-matched to the deadparrot project or be versions 1 and 2, but from what I can tell typeshed doesn't have a mechanism to provide two separate sets of stubs for the same project.
Currently, it doesn't work in typeshed. We could add a new system in typeshed to handle this sort of case, but it will require some amount of design and implementation effort. Also, in practice a scenario like what you describe isn't terribly common: usually new library versions end up just adding new functions and optional parameters. We haven't had a lot of bug reports on typeshed about this sort of issue, although we have had to enhance stubs for libraries like click repeatedly after new releases.
_______________________________________________ Typing-sig mailing list -- typing-sig@python.org To unsubscribe send an email to typing-sig-leave@python.org https://mail.python.org/mailman3/lists/typing-sig.python.org/
Am 29.10.19 um 19:45 schrieb Brett Cannon:
So how would that work if you wanted to provide stubs for older project versions? Let's say for the deadparrot project there's types for version 40 (last version with Python 2 support) and version 42 (latest release), but those stubs are not compatible with each other (e.g. they went nuts with replacing string constants with enums in the latest release). How could stubs be provided for both versions simultaneously on typeshed without having to do a union solution of a single release that weakens the type details? On PyPI there's implicit version support so the stubs could be version-matched to the deadparrot project or be versions 1 and 2, but from what I can tell typeshed doesn't have a mechanism to provide two separate sets of stubs for the same project.
I had ideas for this, but I don't think I have written them down anywhere. At least I can't find it at the moment. But it basically comes down to restructuring the third_party directory in typeshed a bit: Instead of maintaining the current directory split by Python version, we'd have a flat hierarchy and each package sub-directory would contain a metadata file. (A first draft is actually at https://github.com/python/typeshed/blob/third-party-dist/third_party/build/M....) This could contain a package name and supported version range for the package. This would allow us to have, for example, third-party/foo-2 and third-party/foo-3 directories for different versions of the same package. Main problem is how the packages would be named on pypy. Ideally we'd use consistent names, so that type hints for a package "foo" are only a "pip install types.foo" away. But this would not work with supporting multiple package versions. - Sebastian
On Tue, Oct 29, 2019 at 5:50 PM Brett Cannon <brett@python.org> wrote:
Jukka Lehtosalo wrote:
On Tue, Oct 22, 2019 at 9:59 AM Ivan Levkivskyi levkivskyi@gmail.com wrote:
Also IIUC Jukka (cc'ed) is going to spend a week or two early November working 100% on kick-starting the transformation of typeshed to a more modular model.
That's great to hear! Is any of this outlined/planned somewhere? Or is this all in your head, Jukka? ;)
I wrote something about this during PyCon, but I've lost track of my notes. I'll try to find them :-)
Yeah, when I thought about the dream scenario the one that came to mind would be to check projects for `py.typed` and if they lacked it and a `-stubs` project to then generate the stubs for the project as best effort. And then if someone came along and contributed a better, tighter set of stubs to then test that they would be valid to the best of everyone's knowledge somehow in an automated fashion. That way short of needing to step in for spam purposes, things could more-or-less be optimized.
I'd love if contributing auto-generated stubs to typeshed would be a very low-friction process. If we manage to do this, it should be quite feasible to have, say, hundreds of additional packages supported without a lot of effort. And once we have baseline stubs, I hope that we'll soon have many new contributors to gradually improve the annotation coverage of the stubs. In my experience getting started with a new set of stubs is the hard part, and incremental change is quite easy afterwards.
One thing I have always wondered about is how typeshed handles package versions? It's currently a completely flat namespace with no inherent versioning, so when a major version for a package comes out that changes the types of things in a non-backwards-compatible fashion, how are projects supposed to express that for type stubs on typeshed (or in their `-stubs` packages on PyPI for that matter)?
Currently there is no story for versioning, but for the modular typeshed idea we'd probably at least want to add some metadata about package versions. I think that there was some discussion about this some time ago, but I can't remember where. Jukka
On Tue, Oct 29, 2019 at 12:02 PM Jukka Lehtosalo <jlehtosalo@gmail.com> wrote:
On Tue, Oct 29, 2019 at 5:50 PM Brett Cannon <brett@python.org> wrote:
Jukka Lehtosalo wrote:
On Tue, Oct 22, 2019 at 9:59 AM Ivan Levkivskyi levkivskyi@gmail.com wrote:
Also IIUC Jukka (cc'ed) is going to spend a week or two early November working 100% on kick-starting the transformation of typeshed to a more modular model.
That's great to hear! Is any of this outlined/planned somewhere? Or is this all in your head, Jukka? ;)
I wrote something about this during PyCon, but I've lost track of my notes. I'll try to find them :-)
That would be great! I can't promise anything, but if there's a plan I can at least point people at that plan if they are motivated enough to help out.
Yeah, when I thought about the dream scenario the one that came to mind would be to check projects for `py.typed` and if they lacked it and a `-stubs` project to then generate the stubs for the project as best effort. And then if someone came along and contributed a better, tighter set of stubs to then test that they would be valid to the best of everyone's knowledge somehow in an automated fashion. That way short of needing to step in for spam purposes, things could more-or-less be optimized.
I'd love if contributing auto-generated stubs to typeshed would be a very low-friction process. If we manage to do this, it should be quite feasible to have, say, hundreds of additional packages supported without a lot of effort.
Well, in my dream it's **all** of PyPI, so a bit more than triple digits. :)
And once we have baseline stubs, I hope that we'll soon have many new contributors to gradually improve the annotation coverage of the stubs. In my experience getting started with a new set of stubs is the hard part, and incremental change is quite easy afterwards.
Yeah, that would be my hope.
One thing I have always wondered about is how typeshed handles package versions? It's currently a completely flat namespace with no inherent versioning, so when a major version for a package comes out that changes the types of things in a non-backwards-compatible fashion, how are projects supposed to express that for type stubs on typeshed (or in their `-stubs` packages on PyPI for that matter)?
Currently there is no story for versioning, but for the modular typeshed idea we'd probably at least want to add some metadata about package versions. I think that there was some discussion about this some time ago, but I can't remember where.
I guess it's more of a question of what's the overhead of pulling forward stubs when there is little to no change versus wider coverage. As Jelle said, there's probably minimal shift for the large, old projects that have type stubs on typeshed. But when I'm think of what it would take to type all of PyPI then that no longer holds. :) It seems like project version support is really critical if you go down the automation route to provide baseline support, and then you can rely on testing to either carry forward any handwritten types and then fall back to new auto-generated versions when things would break by blindly pulling something forward.
Am 29.10.19 um 20:02 schrieb Jukka Lehtosalo:
I'd love if contributing auto-generated stubs to typeshed would be a very low-friction process. If we manage to do this, it should be quite feasible to have, say, hundreds of additional packages supported without a lot of effort. And once we have baseline stubs, I hope that we'll soon have many new contributors to gradually improve the annotation coverage of the stubs. In my experience getting started with a new set of stubs is the hard part, and incremental change is quite easy afterwards.
That makes sense. From a review standpoint, we should probably just accept auto-generated stubs if they pass the tests and only check for red flags, not for correctness, like we usually do at the moment. Checking for correctness could be reserved to improving on stubs. - Sebastian
participants (9)
-
Brett Cannon -
Cohen Karnell -
Ethan Smith -
Guido van Rossum -
Ivan Levkivskyi -
Jelle Zijlstra -
Jukka Lehtosalo -
Sebastian Kreft -
Sebastian Rittau