PEP 438, pip and --allow-external (was: "pip: cdecimal an externally hosted file and may be unreliable" from python-dev)
So there's an ongoing debate over pip's behaviour around disallowing external hosting by default (see thread "pip: cdecimal an externally hosted file and may be unreliable" over on python-dev for the latest round). It appears that the reason for disallowing external hosting (as opposed to unverifiable downloads) is purely about reliability - we can't be sure that an external host provides the same level of uptime as PyPI[1]. Given that, it seems to me that the situation is, for an externally hosted package foo: `pip install foo` - fails immediately, 100% of the time `pip install --allow-external foo foo` - works in all but a few cases where foo's host is down[2] I cannot understand how guaranteed failure is ever better than "occasional but rare" failure. For situations where it is critical to minimise the risk of an external host outage causing a deployment to fail, the only answer is to not use foo, or to host foo on your own private index. In both cases, all you need is to know that foo is externally hosted to do that - you certainly don't need pip to fail. As a concrete proposal: 1. Remove --allow-external/--allow-all-external and make it the default behaviour. 2. Add a new command to pip, maybe something like `pip check-external` which checks a set of reuirements, and reports the requirements that are externally hosted and which hosts they rely on. That gives users who need 100% reliability the information they need to implement the appropriate solution. Without causing pain for users who don't. Note that the above is based on the fact[3] that *there are no security risks to --allow-external*. I am not arguing for a reduction in security, or a change to any sort of threat model. Comments? Paul [1] Donald explicitly stated that this is the case in the earlier thread (https://mail.python.org/pipermail/python-dev/2014-May/134454.html). I think Nick confirmed this (although I don't have a reference to hand). If it's not true then we need to be a lot clearer and more explicit about *why* ignoring external hosting by default is needed, because it seems nobody knows :-( [2] BTW, the syntax of `--allow-external` is hideous, but that's a side issue I don't want to debate right now. [3] See the first note.
participants (16)
-
Chris Jerdonek
-
Daniel Holth
-
David Arnold
-
Donald Stufft
-
Eric V. Smith
-
Ian Cordasco
-
Justin Cappos
-
M.-A. Lemburg
-
Mark Young
-
Nick Coghlan
-
Noah Kantrowitz
-
Paul Moore
-
Stefan Krah
-
Trishank Karthik Kuppusamy
-
Vinay Sajip
-
Vladimir Diaz