[Python-Dev] My summit notes (packaging)

[Kevin Teague]

On Mar 27, 2009, at 6:25 PM, P.J. Eby wrote:

> At 03:06 PM 3/27/2009 -0500, Tarek Ziadé wrote:
>> They both aim at the
>>  same goal besides a few differences, and they both rely
>>  on a new metadata introduced by setuptools, wich is.
>>  "install_requires". This new metadata extends the metadata.
>>  described in PEP 314 but is slightly different from.
>>  what is descibred in the Draft PEP 345  ("Requires").
>> ..
>>  PEP 345 introduces "Requires" and "Provides" wich are
>>  are implemented in Distutils and PyP, but are not
>>  widely used. 40 out of +4000 if I remember correctly. Martin will
>>  correct me here if I am wrong.
>
> FYI, The reason setuptools uses a different way of specifying  
> requirements is that the PEP-proposed way could not be used without  
> some kind of indexed repository of packages -- and PyPI did not  
> index "provides" at the time.  Also, the PEP-proposed versioning  
> scheme was not compatible with the versioning schemes actually used  
> in the field at the time.
>
> These conditions could be considered to have changed now, or be  
> changeable given enough will and volunteer effort.  Since setuptools  
> was only a 1.5-person effort back in the day (i.e., me plus  
> occasional contribs from Ian Bicking, Bob Ippolito, and Jim Fulton),  
> and backward compatibility was a critical requirement to get  
> adoption, using RPM-style provides/requires was not an option at  
> that time.
>


Tarek, was there any further discussion on "Requires" vs  
"install_requires" or any decisions made on what to do about this?

(I've got a +1 ready for including 'install_requires' in the standard  
package metadata and marking 'Requires' as deprecated if that was put  
forth :P)

 From what I understand of PEP 345's "Requires" and "Provides" fields  
is that it's not possible to use them unambigously? For example, if  
I'm working on a Python project and want to depend upon a project that  
provides a 'utils' module, there could be several packages on PyPI  
that declare they provide a 'utils' module. Which project would get  
picked then?

It's also a lot more succinct to just spell out the project name in  
the "install_requires" field than to have to list all of the modules  
and packages you expect to use in the project you are depending upon.  
For example, this is pretty reasonable:

install_requires: Django

And this is a disaster:

Requires: django
Requires: django.conf
Requires: django.contrib
Requires: django.contrib.admin
Requires: django.contrib.admindocs
Requires: django.contrib.auth
Requires: django.contrib.comments
Requires: django.contrib.contenttypes
Requires: django.contrib.csrf
Requires: django.contrib.databrowse
Requires: django.contrib.flatpages
Requires: django.contrib.formtools
Requires: django.contrib.gis
Requires: django.contrib.humanize
Requires: django.contrib.localflavor
Requires: django.contrib.markup
Requires: django.contrib.redirects
Requires: django.contrib.sessions
Requires: django.contrib.sitemaps
Requires: django.contrib.sites
Requires: django.contrib.syndication
Requires: django.contrib.webdesign
Requires: django.core
Requires: django.db
Requires: django.dispatch
Requires: django.forms
Requires: django.http
Requires: django.middleware
Requires: django.shortcuts
Requires: django.template
Requires: django.templatetags
Requires: django.test
Requires: django.utils
Requires: django.views

Not to mention that if you were upgrading to a newer version of Django  
you'd could no longer just say '1.0' or '1.1' in 'install_requires'  
but would instead have to fiddle with the Requires list to match newly  
introduced modules and packages. I guess one is supposed to just list  
"Requires: django" since that would resolve to the Django project  
which, and it's implied that it would then provide everything else? Or  
perhaps one is supposed to comb through their code and just list the  
particular bits of Django that they're importing and using? There are  
other times when a project may provide several top-level packages, and  
again it's a lot easier to just ask for the project name rather than  
list all of the top-level packages (usually this type of project is  
kept on an private package server where it's reasonable to occupy  
common top-level names such as 'utils' without worrying too much about  
naming conflicts). Additionally, consuming the "Requires" field is  
always more work, since you need to do some kind of look-up to go from  
module/package name to project name before you can fetch a  
distribution for that project.