[Distutils] [Python-Dev] distlib updated with resources API

[Daniel Holth]

On Thu, Oct 4, 2012 at 8:55 AM, Paul Moore <p.f.moore at gmail.com> wrote:
> On 28 September 2012 18:05, Nick Coghlan <ncoghlan at gmail.com> wrote:
>> On Fri, Sep 28, 2012 at 10:07 AM, Daniel Holth <dholth at gmail.com> wrote:
>>> Are we trying to kill setuptools? I'm not entirely sure, but we should
>>> stop trying to do that. The migration should take essentially forever
>>> as soon as it makes sense for each pypi publisher.
>>
>> I'd certainly like to kill easy_install, and see any popular elements
>> of setuptools metadata become officially defined *independently* of
>> any given implementation.

> 3. A standard layout for installed files. This gets harder, because OS
> conventions come into play. But basically, the sysconfig locations are
> the way to encapsulate this. Oh, and kill the egg format :-)
> (Seriously, does the egg format offer any benefit other than the old
> multiversion support that no-one uses any more? If not, it probably
> should be allowed to die off).
>
> 4. A standard binary install format (wheel! :-))

I envision a system that uses a PEP
site-packages-directory-with-one-distribution as the interface between
build and install.

The build system always installs into an empty directory with a
predictable layout:

module.py
package/__init__.py
distname-1.0.data/scripts/a-script.py
distname-1.0.dist-info/METADATA

The build system does not have to worry about annoying details like
counting the number of packages that use a particular __init__.py or
rename-installed-configuration-file-to-.orig only if it was modified
from the previous install. You don't have to give the build system
elevated privileges because it does not install.

Once the build system is done, the installer knows how to copy the
dist to its destination paths (potentially running as root, or on
another machine that does not have a compiler) instead of necessarily
putting it all under a single path. The installer worries about not
overwriting config files and so forth.

When the interface between build and install is simple "build a
standard thing" -> "install a standard thing" it's not a problem to
implement features like "index the metadata with sqlite" that you
could never put in all the build systems. The other huge advantage to
all the setuptools and distutils haters out there is that you no
longer need to have the build system installed in the target
environment, so you can use setuptools to build without having to
install it to run.


The egg format is technically equivalent to a site packages directory
with only one thing in it. The only thing we change is to rename
EGG-INFO to distname-1.0.dist-info and rename the searchable name
PKG-INFO to the un-googleable name METADATA, and put more of the
metadata into one file METADATA. Nothing is lost, so people who need
to use lots of site-packages directories can still do so. Maybe those
people will name them distname-1.0.dist/ or like gem put them in a
folder where every subfolder is a dist. I don't see a reason to
violently kill eggs as long as it is very easy to avoid using them.


Instead of a RESOURCES file with all the files in a dist listed
line-by-line (slow) the installer should be able to write the list of
installation prefixes to a file distribution/_install_locations.py
(the name is configurable, and it is only written on request). This
idea comes from Bento.

purelib = "../relative/path/to-purelib"
platlib = "../relative/path/to-platlib"


IIRC distutils in the standard library only knows how to generate
PKG-INFO at the root of an sdist and it never does anything with the
metadata. It would not be a big deal to copy that into a .dist-info
directory. I would support a subset of the pysetup setup.cfg
[metadata] for setup- and install- requirements instead of adding them
to setup.py.