Re: [Python-Dev] My summit notes (packaging)

At 03:06 PM 3/27/2009 -0500, Tarek Ziadé wrote:
FYI, The reason setuptools uses a different way of specifying requirements is that the PEP-proposed way could not be used without some kind of indexed repository of packages -- and PyPI did not index "provides" at the time. Also, the PEP-proposed versioning scheme was not compatible with the versioning schemes actually used in the field at the time. These conditions could be considered to have changed now, or be changeable given enough will and volunteer effort. Since setuptools was only a 1.5-person effort back in the day (i.e., me plus occasional contribs from Ian Bicking, Bob Ippolito, and Jim Fulton), and backward compatibility was a critical requirement to get adoption, using RPM-style provides/requires was not an option at that time.
- there should be an API to get metadata for a package without actually executing any of the package's installation script.
Please note that where dependencies are concerned, practical considerations require that, at least for currently-released versions of Python, a package's dependencies may well be Python-version specific. A library that targets Python 2.4 and 2.5 and uses wsgiref, sqlite, ctypes, or ElementTree, for example, may have different dependencies depending on the version it is being installed in. Of course, if future Python versions declare metadata for what they include, this could be resolved. But for existing versions it's a bit of a sticky issue. Also, it's quite likely that platform-specific dependencies may exist as well. It might be possible to accommodate these things with a sufficiently flexible format, but currently, the only way to handle them with distutils/setuptools is in the setup script.

On Mar 27, 2009, at 6:25 PM, P.J. Eby wrote:
Tarek, was there any further discussion on "Requires" vs "install_requires" or any decisions made on what to do about this? (I've got a +1 ready for including 'install_requires' in the standard package metadata and marking 'Requires' as deprecated if that was put forth :P) From what I understand of PEP 345's "Requires" and "Provides" fields is that it's not possible to use them unambigously? For example, if I'm working on a Python project and want to depend upon a project that provides a 'utils' module, there could be several packages on PyPI that declare they provide a 'utils' module. Which project would get picked then? It's also a lot more succinct to just spell out the project name in the "install_requires" field than to have to list all of the modules and packages you expect to use in the project you are depending upon. For example, this is pretty reasonable: install_requires: Django And this is a disaster: Requires: django Requires: django.conf Requires: django.contrib Requires: django.contrib.admin Requires: django.contrib.admindocs Requires: django.contrib.auth Requires: django.contrib.comments Requires: django.contrib.contenttypes Requires: django.contrib.csrf Requires: django.contrib.databrowse Requires: django.contrib.flatpages Requires: django.contrib.formtools Requires: django.contrib.gis Requires: django.contrib.humanize Requires: django.contrib.localflavor Requires: django.contrib.markup Requires: django.contrib.redirects Requires: django.contrib.sessions Requires: django.contrib.sitemaps Requires: django.contrib.sites Requires: django.contrib.syndication Requires: django.contrib.webdesign Requires: django.core Requires: django.db Requires: django.dispatch Requires: django.forms Requires: django.http Requires: django.middleware Requires: django.shortcuts Requires: django.template Requires: django.templatetags Requires: django.test Requires: django.utils Requires: django.views Not to mention that if you were upgrading to a newer version of Django you'd could no longer just say '1.0' or '1.1' in 'install_requires' but would instead have to fiddle with the Requires list to match newly introduced modules and packages. I guess one is supposed to just list "Requires: django" since that would resolve to the Django project which, and it's implied that it would then provide everything else? Or perhaps one is supposed to comb through their code and just list the particular bits of Django that they're importing and using? There are other times when a project may provide several top-level packages, and again it's a lot easier to just ask for the project name rather than list all of the top-level packages (usually this type of project is kept on an private package server where it's reasonable to occupy common top-level names such as 'utils' without worrying too much about naming conflicts). Additionally, consuming the "Requires" field is always more work, since you need to do some kind of look-up to go from module/package name to project name before you can fetch a distribution for that project.

On Fri, Mar 27, 2009 at 9:54 PM, Kevin Teague <kevin@bud.ca> wrote:
Yes that is what we were thinking of doing. (deprectating Requires and Provides and and install_requires) We are working on it. We'll try to organize our work in the wiki in the comng days, so people can participate. Regards Tarek

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Tarek Ziadé wrote:
'Provides' could actually be a potentially useful missing feature in setuptools, if it meant "virtual distutils project" rather than "specific module or package." Tres. - -- =================================================================== Tres Seaver +1 540-429-0999 tseaver@palladion.com Palladion Software "Excellence by Design" http://palladion.com -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.6 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iD8DBQFJ0MxT+gerLs4ltQ4RAgldAJ0Y0arjMeQJYK/G/6pkLqj2WV/Y/gCeL8b9 WnN4ZUWoaZUcFEysEWYsyXE= =wOf2 -----END PGP SIGNATURE-----

On Mar 27, 2009, at 9:25 PM, P.J. Eby wrote:
There's a deeper issue IMO. As Kevin pointed out, distutil's Requires data works at the module and package level, rather than at the project level. I can see some value in this, but I think working at the project level is a lot simpler and more practically useful. Jim -- Jim Fulton Zope Corporation

On Mar 27, 2009, at 6:25 PM, P.J. Eby wrote:
Tarek, was there any further discussion on "Requires" vs "install_requires" or any decisions made on what to do about this? (I've got a +1 ready for including 'install_requires' in the standard package metadata and marking 'Requires' as deprecated if that was put forth :P) From what I understand of PEP 345's "Requires" and "Provides" fields is that it's not possible to use them unambigously? For example, if I'm working on a Python project and want to depend upon a project that provides a 'utils' module, there could be several packages on PyPI that declare they provide a 'utils' module. Which project would get picked then? It's also a lot more succinct to just spell out the project name in the "install_requires" field than to have to list all of the modules and packages you expect to use in the project you are depending upon. For example, this is pretty reasonable: install_requires: Django And this is a disaster: Requires: django Requires: django.conf Requires: django.contrib Requires: django.contrib.admin Requires: django.contrib.admindocs Requires: django.contrib.auth Requires: django.contrib.comments Requires: django.contrib.contenttypes Requires: django.contrib.csrf Requires: django.contrib.databrowse Requires: django.contrib.flatpages Requires: django.contrib.formtools Requires: django.contrib.gis Requires: django.contrib.humanize Requires: django.contrib.localflavor Requires: django.contrib.markup Requires: django.contrib.redirects Requires: django.contrib.sessions Requires: django.contrib.sitemaps Requires: django.contrib.sites Requires: django.contrib.syndication Requires: django.contrib.webdesign Requires: django.core Requires: django.db Requires: django.dispatch Requires: django.forms Requires: django.http Requires: django.middleware Requires: django.shortcuts Requires: django.template Requires: django.templatetags Requires: django.test Requires: django.utils Requires: django.views Not to mention that if you were upgrading to a newer version of Django you'd could no longer just say '1.0' or '1.1' in 'install_requires' but would instead have to fiddle with the Requires list to match newly introduced modules and packages. I guess one is supposed to just list "Requires: django" since that would resolve to the Django project which, and it's implied that it would then provide everything else? Or perhaps one is supposed to comb through their code and just list the particular bits of Django that they're importing and using? There are other times when a project may provide several top-level packages, and again it's a lot easier to just ask for the project name rather than list all of the top-level packages (usually this type of project is kept on an private package server where it's reasonable to occupy common top-level names such as 'utils' without worrying too much about naming conflicts). Additionally, consuming the "Requires" field is always more work, since you need to do some kind of look-up to go from module/package name to project name before you can fetch a distribution for that project.

On Fri, Mar 27, 2009 at 9:54 PM, Kevin Teague <kevin@bud.ca> wrote:
Yes that is what we were thinking of doing. (deprectating Requires and Provides and and install_requires) We are working on it. We'll try to organize our work in the wiki in the comng days, so people can participate. Regards Tarek

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Tarek Ziadé wrote:
'Provides' could actually be a potentially useful missing feature in setuptools, if it meant "virtual distutils project" rather than "specific module or package." Tres. - -- =================================================================== Tres Seaver +1 540-429-0999 tseaver@palladion.com Palladion Software "Excellence by Design" http://palladion.com -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.6 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iD8DBQFJ0MxT+gerLs4ltQ4RAgldAJ0Y0arjMeQJYK/G/6pkLqj2WV/Y/gCeL8b9 WnN4ZUWoaZUcFEysEWYsyXE= =wOf2 -----END PGP SIGNATURE-----

On Mar 27, 2009, at 9:25 PM, P.J. Eby wrote:
There's a deeper issue IMO. As Kevin pointed out, distutil's Requires data works at the module and package level, rather than at the project level. I can see some value in this, but I think working at the project level is a lot simpler and more practically useful. Jim -- Jim Fulton Zope Corporation
participants (5)
-
Jim Fulton
-
Kevin Teague
-
P.J. Eby
-
Tarek Ziadé
-
Tres Seaver