Clarifications on PEP 561
Hi! Now that we (currently me and Jukka) are working on modular typeshed, we think there are two things underspecified in PEP 561 about stub-only packages. Namely: * What to do with distributions that have only modules, but not packages? Since it is not easy to add a data file (like .pyi) not in a package, we use "dummy" packages named module_name-stubs with a single __init__.pyi. * What to do with packages that have different stub files for Python 2 and 3 (for example six)? Currently we just add two packages to the distribution, one six-stubs (as specified by PEP 561) and another six-python2-stubs. Are there any opinions/comments on these two points? -- Ivan
El sáb., 27 jun. 2020 a las 8:51, Ivan Levkivskyi (<levkivskyi@gmail.com>) escribió:
Hi!
Now that we (currently me and Jukka) are working on modular typeshed, we think there are two things underspecified in PEP 561 about stub-only packages. Namely:
* What to do with distributions that have only modules, but not packages? Since it is not easy to add a data file (like .pyi) not in a package, we use "dummy" packages named module_name-stubs with a single __init__.pyi.
That seems like the right solution. I think we intentionally left this case unsupported in PEP 561.
* What to do with packages that have different stub files for Python 2 and 3 (for example six)? Currently we just add two packages to the distribution, one six-stubs (as specified by PEP 561) and another six-python2-stubs.
Does this come up often for packages other than six? Perhaps we could just `if sys.version_info` checks.
Are there any opinions/comments on these two points?
-- Ivan
_______________________________________________ Typing-sig mailing list -- typing-sig@python.org To unsubscribe send an email to typing-sig-leave@python.org https://mail.python.org/mailman3/lists/typing-sig.python.org/ Member address: jelle.zijlstra@gmail.com
On Sat, 27 Jun 2020 at 17:55, Jelle Zijlstra <jelle.zijlstra@gmail.com> wrote:
El sáb., 27 jun. 2020 a las 8:51, Ivan Levkivskyi (<levkivskyi@gmail.com>) escribió:
* What to do with packages that have different stub files for Python 2 and 3 (for example six)? Currently we just add two packages to the distribution, one six-stubs (as specified by PEP 561) and another six-python2-stubs.
Does this come up often for packages other than six? Perhaps we could just `if sys.version_info` checks.
It doesn't come up often but six itself is pretty big, so may be a pain to merge in a single package with version checks. -- Ivan
Hi! Exciting to see work on a modular typeshed happening!
* What to do with distributions that have only modules, but not packages? Since it is not easy to add a data file (like .pyi) not in a package, we use "dummy" packages named module_name-stubs with a single __init__.pyi.
PEP 561 actually says " This PEP does not support distributing typing information as part of module-only distributions. The code should be refactored into a package-based distribution and indicate that the package supports typing as described above." So your understanding is correct, stubs should be refactored into a package for module-only distributions.
* What to do with packages that have different stub files for Python 2 and 3 (for example six)? Currently we just add two packages to the distribution, one six-stubs (as specified by PEP 561) and another six-python2-stubs.
PEP 561 requires that stus for a package *name* be installed under *name-stubs*, so puting python2-stubs is not PEP compliant. I see two solutions: 1) the stubs have if sys.version_info ... checks throughout the stubs 2) you have two packages, one for each Python version, but only install one into *name-stubs* at install time depending on the Python version they are being installed into. This seems sub-optimal as it would disallow wheels for distribution of such packages and require the stubs to be installed into the Python version you are checking. I personally think that while 1) is a bit more work, it is the best solution for version straddling code. In addition, I expect the number of packages that need to straddle 2/3 will be fewer and more far between as time goes on. Best, Ethan On Sat, Jun 27, 2020 at 10:21 AM Ivan Levkivskyi <levkivskyi@gmail.com> wrote:
On Sat, 27 Jun 2020 at 17:55, Jelle Zijlstra <jelle.zijlstra@gmail.com> wrote:
El sáb., 27 jun. 2020 a las 8:51, Ivan Levkivskyi (<levkivskyi@gmail.com>) escribió:
* What to do with packages that have different stub files for Python 2 and 3 (for example six)? Currently we just add two packages to the distribution, one six-stubs (as specified by PEP 561) and another six-python2-stubs.
Does this come up often for packages other than six? Perhaps we could just `if sys.version_info` checks.
It doesn't come up often but six itself is pretty big, so may be a pain to merge in a single package with version checks.
-- Ivan
On Sat, 27 Jun 2020 at 22:35, Ethan Smith <ethan@ethanhs.me> wrote:
Hi!
Exciting to see work on a modular typeshed happening!
* What to do with distributions that have only modules, but not packages? Since it is not easy to add a data file (like .pyi) not in a package, we use "dummy" packages named module_name-stubs with a single __init__.pyi.
PEP 561 actually says " This PEP does not support distributing typing information as part of module-only distributions. The code should be refactored into a package-based distribution and indicate that the package supports typing as described above." So your understanding is correct, stubs should be refactored into a package for module-only distributions.
* What to do with packages that have different stub files for Python 2 and 3 (for example six)? Currently we just add two packages to the distribution, one six-stubs (as specified by PEP 561) and another six-python2-stubs.
PEP 561 requires that stus for a package *name* be installed under *name-stubs*, so puting python2-stubs is not PEP compliant. I see two solutions:
1) the stubs have if sys.version_info ... checks throughout the stubs
2) you have two packages, one for each Python version, but only install one into *name-stubs* at install time depending on the Python version they are being installed into. This seems sub-optimal as it would disallow wheels for distribution of such packages and require the stubs to be installed into the Python version you are checking.
I personally think that while 1) is a bit more work, it is the best solution for version straddling code. In addition, I expect the number of packages that need to straddle 2/3 will be fewer and more far between as time goes on.
But what is the benefit of not providing a simple mechanism of distributing version-dependent stubs? All type checkers already support this because this is how typeshed is organized. As I mentioned before, six is pretty big, and I am not interested in converting it to sys.version_info based checks. -- Ivan
The benefit is that we don't (further) abuse the packaging standards to add support for a version of Python that is no longer supported. The vast majority of packages work exactly the same independent of Python version, so this is a pretty niche feature now. Also it would be a significant change to the PEP. I believe you can just recurse through each version folder and concatenate the files (with a sys.version_info check at the top) when there is a copy in both 2/ and 3/. I realize this is more painful than your proposed solution, but surely this could be automated with a couple dozen line script? Best, Ethan On Sun, Jun 28, 2020 at 7:15 AM Ivan Levkivskyi <levkivskyi@gmail.com> wrote:
On Sat, 27 Jun 2020 at 22:35, Ethan Smith <ethan@ethanhs.me> wrote:
Hi!
Exciting to see work on a modular typeshed happening!
* What to do with distributions that have only modules, but not packages? Since it is not easy to add a data file (like .pyi) not in a package, we use "dummy" packages named module_name-stubs with a single __init__.pyi.
PEP 561 actually says " This PEP does not support distributing typing information as part of module-only distributions. The code should be refactored into a package-based distribution and indicate that the package supports typing as described above." So your understanding is correct, stubs should be refactored into a package for module-only distributions.
* What to do with packages that have different stub files for Python 2 and 3 (for example six)? Currently we just add two packages to the distribution, one six-stubs (as specified by PEP 561) and another six-python2-stubs.
PEP 561 requires that stus for a package *name* be installed under *name-stubs*, so puting python2-stubs is not PEP compliant. I see two solutions:
1) the stubs have if sys.version_info ... checks throughout the stubs
2) you have two packages, one for each Python version, but only install one into *name-stubs* at install time depending on the Python version they are being installed into. This seems sub-optimal as it would disallow wheels for distribution of such packages and require the stubs to be installed into the Python version you are checking.
I personally think that while 1) is a bit more work, it is the best solution for version straddling code. In addition, I expect the number of packages that need to straddle 2/3 will be fewer and more far between as time goes on.
But what is the benefit of not providing a simple mechanism of distributing version-dependent stubs? All type checkers already support this because this is how typeshed is organized. As I mentioned before, six is pretty big, and I am not interested in converting it to sys.version_info based checks.
-- Ivan
On Sun, 28 Jun 2020 at 18:03, Ethan Smith <ethan@ethanhs.me> wrote:
The benefit is that we don't (further) abuse the packaging standards to add support for a version of Python that is no longer supported. The vast majority of packages work exactly the same independent of Python version, so this is a pretty niche feature now. Also it would be a significant change to the PEP.
Does this really has anything to do with packaging standards? This can be just a naming agreement (for packages, not distributions) between type-checking tools to use shared resources. Also most type-checking tools support Python 2 and will support it for at least few more years, because often these tools are used to facilitate Python 2 to 3 migration.
I believe you can just recurse through each version folder and concatenate the files (with a sys.version_info check at the top) when there is a copy in both 2/ and 3/. I realize this is more painful than your proposed solution, but surely this could be automated with a couple dozen line script?
There are some complications like some module stubs in six only exist in one version. Are you volunteering to do this (=merge third party stubs in typeshed)? As I said I am not going to this, since to me this looks like busywork. -- Ivan
Am 27.06.20 um 23:35 schrieb Ethan Smith:
* What to do with packages that have different stub files for Python 2 and 3 (for example six)? Currently we just add two packages to the distribution, one six-stubs (as specified by PEP 561) and another six-python2-stubs.
PEP 561 requires that stus for a package *name* be installed under *name-stubs*, so puting python2-stubs is not PEP compliant. I see two solutions:
1) the stubs have if sys.version_info ... checks throughout the stubs
2) you have two packages, one for each Python version, but only install one into *name-stubs* at install time depending on the Python version they are being installed into. This seems sub-optimal as it would disallow wheels for distribution of such packages and require the stubs to be installed into the Python version you are checking.
I personally think that while 1) is a bit more work, it is the best solution for version straddling code. In addition, I expect the number of packages that need to straddle 2/3 will be fewer and more far between as time goes on.
I don't think 1) is worth the effort. It is quite a time-intensive task to unify the existing stubs for a Python version that is not officially supported anymore and will eventually be removed from typeshed. - Sebastian
Am 27.06.20 um 17:52 schrieb Ivan Levkivskyi:
Hi!
Now that we (currently me and Jukka) are working on modular typeshed, we think there are two things underspecified in PEP 561 about stub-only packages.
Thank you for your work on that.
* What to do with distributions that have only modules, but not packages? Since it is not easy to add a data file (like .pyi) not in a package, we use "dummy" packages named module_name-stubs with a single __init__.pyi.
I would prefer if there was a way to have module-only packages. I believe that it is preferable to have the stubs follow the implementation as closely as possible. As far as I know, it doesn't make much difference for current uses of stubs, though, so this is not a big issue.
* What to do with packages that have different stub files for Python 2 and 3 (for example six)? Currently we just add two packages to the distribution, one six-stubs (as specified by PEP 561) and another six-python2-stubs.
Sounds fine to me. - Sebastian
On Mon, 29 Jun 2020 at 11:45, Sebastian Rittau <srittau@rittau.biz> wrote:
Am 27.06.20 um 17:52 schrieb Ivan Levkivskyi:
* What to do with distributions that have only modules, but not packages? Since it is not easy to add a data file (like .pyi) not in a package, we use "dummy" packages named module_name-stubs with a single __init__.pyi.
I would prefer if there was a way to have module-only packages. I believe that it is preferable to have the stubs follow the implementation as closely as possible. As far as I know, it doesn't make much difference for current uses of stubs, though, so this is not a big issue.
The module stubs will be still modules in typeshed. The transformation will be done automatically "on-the-fly" by the packaging/uploading script (will make a PR in a minute). So this is essentially only a problem for people who decide to keep their module-only distributions outside typeshed. -- Ivan
I would say that other (non built-in) stubs are also interested in this. For example, we in TypedDjango struggle quite hard to combine django@2 and django@3 stubs in a single code base. https://github.com/typeddjango/django-stubs It would really neat to be able to have something like `flow` does: https://github.com/flow-typed/flow-typed/tree/master/definitions/npm (take a look at `axios` for example). Or any other similar way. пн, 29 июн. 2020 г. в 14:09, Ivan Levkivskyi <levkivskyi@gmail.com>:
On Mon, 29 Jun 2020 at 11:45, Sebastian Rittau <srittau@rittau.biz> wrote:
Am 27.06.20 um 17:52 schrieb Ivan Levkivskyi:
* What to do with distributions that have only modules, but not packages? Since it is not easy to add a data file (like .pyi) not in a package, we use "dummy" packages named module_name-stubs with a single __init__.pyi.
I would prefer if there was a way to have module-only packages. I believe that it is preferable to have the stubs follow the implementation as closely as possible. As far as I know, it doesn't make much difference for current uses of stubs, though, so this is not a big issue.
The module stubs will be still modules in typeshed. The transformation will be done automatically "on-the-fly" by the packaging/uploading script (will make a PR in a minute).
So this is essentially only a problem for people who decide to keep their module-only distributions outside typeshed.
-- Ivan
_______________________________________________ Typing-sig mailing list -- typing-sig@python.org To unsubscribe send an email to typing-sig-leave@python.org https://mail.python.org/mailman3/lists/typing-sig.python.org/ Member address: n.a.sobolev@gmail.com
participants (5)
-
Ethan Smith -
Ivan Levkivskyi -
Jelle Zijlstra -
Sebastian Rittau -
Никита Соболев