[Distutils] PEP 376, the INSTALLER file, and system packages

[Donald Stufft]

> On Jan 22, 2016, at 3:38 PM, Paul Moore <p.f.moore at gmail.com> wrote:
> 
> On 22 January 2016 at 18:46, Nathaniel Smith <njs at pobox.com> wrote:
>>> I'd like to propose adding a special cased value to add to the installer
>>> file
>>> that will tell projects like pip that this particular installed thing is
>>> being
>>> managed by someone else, and we should keep our hands off of it. According
>>> to
>>> PEP 376 the supported values for this file are r"[a-z0-9_-.]", however I
>>> think
>>> since nobody has ever implemented it, we could expand that so that it so
>>> you
>>> can also have a special value, of "dpkg (system)" or maybe that's not
>>> worth it
>>> and we could just have "system" as a special value.
>> 
>> I think we want more than just "system", because the same user could have
>> some packages managed by dpkg and some by conda, both of which have their
>> own dependency resolution mechanisms that are outside pip's and could get
>> broken if pip removes stuff willy-nilly. And when pip errors out, you want
>> to be able to explain to the user "this package is managed by conda, and
>> using pip on it may break your conda setup..." versus "this package is
>> managed by Debian, and using pip on it may break your Debian setup...".
> 
> Well, I would expect conda to be specifying "conda" in the INSTALLER
> file. That's what the file is intended for, surely? To record what
> tool installed the package?
> 
> Equally, there seems to me to be no reason dpkg couldn't use "dpkg",
> and yum use "yum", etc. I just see Donald's suggestion of using
> "system" as a simple flag saying "the OS" in a generic way for
> distributions that don't (for some reason) *want* to name the specific
> installer in the INSTALLER file. Well, that and the fact that they
> could have done that for years now, so being able to say "just put
> "system" in there and be done with it please!" is a nice easy message
> to give to distributions.
> 
> Honestly, this seems to me to be a bit of a non-discussion.
> 
> My recollection of PEP 376 and the discussions around INSTALLER are
> that it was simply a place where install tools could say "hey, it was
> me that did this". It never got used largely because nobody seemed to
> care about having that information. Now pip is starting to care, but
> as far as I can see there are only 3 cases pip should care about:
> 
> 1. The file says "pip". Do just as we do now, we control that package.
> 2. The file says something *other* than pip. We don't own the files,
> take appropriate action. Warn, require an extra flag, whatever - we
> can use what's in the file to say "files owned by FOO" but other than
> that there's no reason we should behave differently depending on what
> specific non-pip value we see.
> 3. There's no INSTALLER file. This is the awkward one, as we can't
> (yet) say that this means the file isn't owned by pip. In a few years
> maybe, but for now we have to assume it could be either of the above
> cases.
> 
> So the message should be "if you want pip to be careful with your
> files, put something (probably your installer name, but in practice
> anything other than "pip") in INSTALLER".
> 
> Longer term, other tools might taking the same approach as pip, at
> which point not using the same name as other tools becomes useful - at
> that point, a generic term like "system" could be a bad choice...
> Paul

Hmm, in my head the three cases were more like:

1) The installed project is managed by a Python level tool which uses the Python metadata as it’s database. Thus if you’re this kind of tool too, then you can muck around here because things will be fine.
2) The installed project is managed by something that isn’t a Python level tool, and it has it’s own database and the .(egg|dist)-info files are not understood by it.
3) We don’t know what is managing the project.

The first would be things like, pip, easy_install, and distil. These can all easily interopt with each other because they’re all going to lay down .egg-info or .dist-info files and read those same files. The second is things like Conda, dpkg, yum, etc which are going to treat .egg-info or .dist-info files as just another opaque file that is included in it’s package that it has to lay down.

-----------------
Donald Stufft
PGP: 0x6E3CBCE93372DCFA // 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 842 bytes
Desc: Message signed with OpenPGP using GPGMail
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20160122/21a47a54/attachment.sig>