[Distutils] PEP 376, the INSTALLER file, and system packages

[Donald Stufft]

> On Jan 22, 2016, at 2:09 PM, Robert Collins <robertc at robertcollins.net> wrote:
> 
> I think requiring a force flag to replace files installed by another
> packaging tool is an unbreakme option as things stand, and so I'm very
> concerned about the resulting user experience.

I don't think the current behavior makes *any* sense and represents nothing but
a footgun where you end up with two systems fighting over what the content of a
particular file is.

Currently there are two sort of scenarios where something like this comes up:

Debuntu
-------

Current behavior is we'll uninstall the file sitting at
/usr/lib/pythonX.Y/dist-packages/* and then we'll go ahead and install our own
files into /usr/local/lib/pythonX.Y/dist-packages/*. Then, the next time that
package gets an update the files are put back into place by dpkg into
/usr/lib/pythonX.Y/dist-packages/* and we're finally at a consistent state
where both package managers know the state of their respective systems.

In the "you need a flag" future world, we would just skip uninstalling the
files from /usr/lib/pythonX.Y/dist-packages/* and just install our own files
into /usr/local/lib/pythonX.Y/dist-packages. Thus we never touch anything that
isn't owned by us and Debian's internal systems stay consistent. If someone did
only ``pip uninstall requests`` and that was located in the OS files, then we
would refuse to uninstall it, directing them to the platform tooling instead.


Everyone Else
-------------

The /usr/lib and /usr/local/lib split is Debian specific (though at some point
I want to bring it to upstream Python) so on other systems you have a single
location, typically in /usr/lib/.../site-packages where both the OS and pip
will be trying to modify files. Right now this is pure chaos, if
python-requests is installed by the system, pip will come along and uninstall
it then upgrade requests, then the next time an update is released, the system
tooling will then come along and write all over the files that pip dropped
into place, possibly corrupting things depending on how their patching works.
Even for just ``pip uninstall foo``, it's really only uninstall until the
next time that package gets updated.

In the "you need a flag future", if python-requests is installed by the OS and
it's in the same location that *we* want to write a file to, then we'll error
out and require a --force flag. If it isn't already installed and nothing is
in our way, then we'll go ahead and install to that location. This isn't as
nice as the Debuntu world, but nothing will be without the patch that Debian
has.


Essentially though, ``pip install -U anything-installed-by-the-os`` is at best
a no-op that deletes files for no good reason (Debuntu) or it's a field full of
footguns where you're going to have two independent systems shitting all over
each others files trying to do what they think the user wants.

-----------------
Donald Stufft
PGP: 0x6E3CBCE93372DCFA // 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 842 bytes
Desc: Message signed with OpenPGP using GPGMail
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20160122/f000044e/attachment.sig>