[Distutils] PEP 438, pip and --allow-external (was: "pip: cdecimal an externally hosted file and may be unreliable" from python-dev)

[Donald Stufft]

On May 11, 2014, at 6:04 PM, Nick Coghlan <ncoghlan at gmail.com> wrote:

> 
> On 11 May 2014 23:18, "Donald Stufft" <donald at stufft.io> wrote:
> >
> > You’re worried that this change is a (or will at least be perceived as such) FU to Stefan and MAL, I’m worried that not fixing this is a FU to *everyone else*.
> 
> Keep in mind that I am *agreeing* that "allow external" at the package level needs to be the "just make it work" option, and hence should provide the current "allow unverifiable" behaviour.
> 
> The only point of contention is what to do with the current "allow external" behaviour:
> 
> 1. Delete it entirely
> 2. Rename it
> 3. Only have it available as a global flag
> 
> The relevant paragraph of PEP 438 that we're considering deciding is wrong is this one from the phase 2 description:
> 
> """Installers should provide a blanket option to allow installing any verifiable external link. Non-verifiable external links should only be installed if the user-provided option specifies exactly which external domains can be used or for which specific package names external links can be used."""
> 
> So, re-reading that, my preference is for option 3: keeping the global allow-all-external flag, but renaming it as something like "allow-all-verifiable-external". It's only dropping that flag entirely or making it mean "allow all unverifiable" that would mean moving away from the previously agreed approach in the PEP.
> 
> There's no requirement in the PEP for a per-package flag to accept verifiable downloads, so making allow-external mean the same thing as allow-unverifiable isn't a problem from that perspective. The PEP also doesn't mandate particular option names.
> 
> Cheers,
> Nick.
> 
> P.S. I wrote most of the above before catching up on the PR comments, including Paul's one about taking the PEP as authoritative. Indeed, I do, and I don't think writing a replacement just to delete one not-especially-useful option is a good use of time and energy :)

The problem is, if you’re reading the documentation it looks like
--allow-all-verifiable-external is the option you want. People fundamentally
don’t grasp the difference between safe and unsafe external hosting. I've tried
to explain it over and over and over and the most common reaction is confusion.
You read the documentation and you’re told “We don’t allow externally hosted
things by default, you can use —allow-external <package> to allow it for a
particular package which will include unsafely hosted ones, or you can use
--allow-all-verifiable-external to allow all safely hosted ones.

Is there any person who is confronted with an option to allow it unsafely for
a single package, or safely for all packages is going to pick the unsafely for
a single one? Even though this is the one that they almost always want? I can't
imagine that anyone would. So it's going to require documentation to say "even
though you think you want this other option you most likely don't".

It's replacing one footgun with another different footgun.

-----------------
Donald Stufft
PGP: 0x6E3CBCE93372DCFA // 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20140511/0e941476/attachment-0001.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 801 bytes
Desc: Message signed with OpenPGP using GPGMail
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20140511/0e941476/attachment-0001.sig>