[Distutils] PEP 438, pip and --allow-external (was: "pip: cdecimal an externally hosted file and may be unreliable" from python-dev)

[Nick Coghlan]

On 12 May 2014 08:12, "Donald Stufft" <donald at stufft.io> wrote:
> On May 11, 2014, at 6:04 PM, Nick Coghlan <ncoghlan at gmail.com> wrote:
>> So, re-reading that, my preference is for option 3: keeping the global
allow-all-external flag, but renaming it as something like
"allow-all-verifiable-external". It's only dropping that flag entirely or
making it mean "allow all unverifiable" that would mean moving away from
the previously agreed approach in the PEP.
>>
>> There's no requirement in the PEP for a per-package flag to accept
verifiable downloads, so making allow-external mean the same thing as
allow-unverifiable isn't a problem from that perspective. The PEP also
doesn't mandate particular option names.
>
>
> The problem is, if you’re reading the documentation it looks like
> --allow-all-verifiable-external is the option you want. People
fundamentally
> don’t grasp the difference between safe and unsafe external hosting. I've
tried
> to explain it over and over and over and the most common reaction is
confusion.
> You read the documentation and you’re told “We don’t allow externally
hosted
> things by default, you can use —allow-external <package> to allow it for a
> particular package which will include unsafely hosted ones, or you can use
> --allow-all-verifiable-external to allow all safely hosted ones.

I agree we'd never recommend the global flag, as it probably won't work.
This is really about managing the removal process in a way that makes it
clear that we're acting as responsible stewards in the spirit of PEP 438,
rather than being cavalier in removing options.

In that vein, deprecation of the global flag would be more appropriate than
removal (I'm assuming you don't want to change it to globally opt in to
unverifiable external downloads).

Specifically, what I would recommend is:

1. Remove any references to the global flag from other documentation and
error messages - always recommend the use of per-package flags to enable
external downloads.
2. Deprecate the global flag in 1.6
3. Remove it in 1.7 unless there is a marked increase in the proportion of
verifiable vs unverifiable external hosting on PyPI.

Adding a more explicit alias in step 2 would also be reasonable (since the
meaning of the base "allow external" will be changing). Either way, the
docs for the global option should acquire a caveat:

"Note: The vast majority of links from PyPI to externally hosted packages
are missing the hash information needed for verification, so this option
won't actually allow downloading of most externally hosted packages - to
handle such cases, use "allow-external" with the specific package name"

Cheers,
Nick.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20140512/f822ee91/attachment.html>