[Distutils] PEP 376, the INSTALLER file, and system packages

[Robert Collins]

On 23 January 2016 at 07:10, Donald Stufft <donald at stufft.io> wrote:
> PEP 376 added a file to the .dist-info directory called "INSTALLER" which was
> supposed to be:
>
>     This option is the name of the tool used to invoke the installation.
>
> However, nothing has really ever implemented it and it's gone largely ignored
> until just recently pip 8.0 started writing the INSTALLER file into the
> metadata directories with a value of "pip".
>
> I'd like to propose adding a special cased value to add to the installer file
> that will tell projects like pip that this particular installed thing is being
> managed by someone else, and we should keep our hands off of it. According to
> PEP 376 the supported values for this file are r"[a-z0-9_-.]", however I think
> since nobody has ever implemented it, we could expand that so that it so you
> can also have a special value, of "dpkg (system)" or maybe that's not worth it
> and we could just have "system" as a special value.
>
> The benefit of doing this, is that with a special value in that file that says
> "this file belongs to the OS", then pip could start looking for that file and
> require a --force flag before it modifies any files belonging to that project.
> Then distributors like Debian, Fedora, etc could simply write out the INSTALLER
> file with the correct value, and pip would start to respect their files by
> default.
>
> Thoughts? Does this sound reasonable to everyone? Do we think it needs a new
> PEP or can we just amend PEP 376 to add this extra bit of information?

I think asking other distros to export the installing information to
us is a good thing.

I think requiring a force flag to replace files installed by another
packaging tool is an unbreakme option as things stand, and so I'm very
concerned about the resulting user experience.

If someone runs 'sudo pip install -U pip' they are already signalling
precisely what they want, and as a user I very much doubt that:

"Refusing to uninstall pip because it was not installed by pip"

will make even 5% of them pause longer than required to look up the
--force-alternative-installer flag (or whatever it is) and pass it in.

The only reason we're touching these other installer installed files
is because there is no location on the Python and exec paths where we
can install the new pip without overwriting the one dpkg (or conda or
whatever) installed. If there was such a place we could just write
(and upgrade things) there and not have to deal with shadowing, or
ever uninstalling dpkg etc installed files. In the absence of such a
place, removing the other-install-system's installed files is the
right thing to do when a user asks us to, without needing an
additional option.

We could add an option folk could set to turn *on* such a safety belt
- but I don't know who it would /really/ be helping. (I'm fairly sure
I know the places folk think it would help, I just don't agree that it
would :)).

Having a interface to the system package manager as has been
previously mooted - and I'm going to get back to that - might help a
little, but uninstalls are quite different to installs. Uninstalls get
blocked by things like 'app-foo requires requests', and I would be
super suprised (and delighted!) if we were able to override that when
upgrading requests via pip :)

-Rob


-- 
Robert Collins <rbtcollins at hpe.com>
Distinguished Technologist
HP Converged Cloud