[Distutils] [Python-Dev] PEP 365 (Adding the pkg_resources module)

[Phillip J. Eby]

At 03:43 PM 3/18/2008 -0500, Guido van Rossum wrote:
>Only very few people would care about writing a setup
>script that works with this bootstrap module; basically only package
>manager implementers.

That's true today, sure, but as soon as it is widely available, 
others are sure to want to use it too.  I just want a bright-line 
distinction between what is and isn't bootstrappable, rather than a 
murky region of "maybe, if you're not doing anything too complicated".


>There seems to be a misunderstanding about what I am proposing we do
>instead. The boostrap installer should only be powerful enough to
>allow it to be used to install a real package manager like setuptools.

Which is why PEP 365 proposed only downloading an archive to a cache 
directory, and optionally running something from it.  It explicitly 
disavows "installation" of anything, since the downloaded archive 
wouldn't have been added to sys.path except for the duration of the 
bootstrap process, and no scripts were to be installed.  (Indeed, 
apart from the methods it would have used to locate the archive on 
PyPI, and to determine what to run from inside it, there was nothing 
particularly egg-specific about the proposed bootstrapping process.)

So, to fully egg-neutralize the bootstrapping approach, we need only 
know how to locate an appropriate archive, and how to determine what 
to run from it.

For the latter, we could use the already-in-2.6 convention of running 
__main__ from a zipfile or directory.  (Too bad distutils source 
distributions have an extra directory name embedded in them, so one 
can't just execute them directly.  Otherwise, we could've just let 
people drop in a __main__.py next to setup.py.  OTOH, maybe it would 
be enough to use setuptools' algorithm for finding setup.py to locate 
__main__.py, and I'm fairly sure *that* can be briefly expressed in the PEP.)

The other open question is a naming convention and version detection, 
so that the bootstrap tool can identify which of the files listed on 
PyPI is suitable for its use.  (Both with regard to the version 
selection, and file type.)  However, if PyPI were to grow support for 
designating the appropriate files and/or versions in some other way, 
we wouldn't need a naming convention as such.

Without one or the other, the bootstrap tool would have to grow a 
version parsing scheme of some type, and play guessing games with 
file extensions.  (Which is one reason I limited PEP 365's scope to 
downloading eggs actually *uploaded* to PyPI, rather than arbitrary 
packages *linked* from PyPI.)

So, if I had to propose something right now, I would be inclined to propose:

* using setuptools' version parsing semantics for interpretation of 
alpha/beta/dev/etc. releases

* having a bdist_bootstrap format that's essentially a bdist_dumb 
.zip file with the internal path prefixes stripped off, making it an 
importable .zip with a different file extension.  (Or maybe just 
.pyboot.zip?)  The filename convention would use setuptools' 
canonicalization and escaping of names and version numbers, to allow 
unambiguous machine parsing of the filename.  A __main__ module would 
have to be present for the archive to be run, as opposed to just 
being downloaded to a temporary directory.

* calling the bootstrap module 'bootstrap', as in 'python -m 
bootstrap projectname optionalversion'.  The module would expose an 
API to allow it to be used programmatically as well as the command 
line, so that bootstrapped packages can use the bootstrap process to 
locate dependencies if they so desire.  (Today's package management 
tools, at least, are all based on setuptools, so if it's not present 
they'll need to download that before beginning their own 
bootstrapping process.)

Apart from keeping the PEP self-contained and short, is there 
anything in this that you think you would object to?  (You may 
reserve the right, of course, to later not like something in the 
details of setuptools' version/filename rules, after I've put them 
into the PEP, or really, anything else.  I'm just asking if there's 
anything that's obviously offensive at this point, before I spend 
time on a new PEP.)