[Distutils] Need for respect (was: PEP 438, pip and --allow-external)

[Stefan Krah]

Paul Moore <p.f.moore at gmail.com> wrote:
> > "Installers should provide a blanket option to allow installing any verifiable
> >  external link."
> >
> > Perhaps something like --allow-verifiable-external would do?  I would not be
> > unhappy if link-spidering were to be removed, I find it reasonable to provide
> > the explicit link.
> 
> I believe that option has been there for a while as
> --allow-[all]-external. Again, naming and discoverability may be an
> issue, but the functionality is available.

Yes, but I understood that the latest proposals in this thread wanted
to get rid of even this functionality. Did I get that wrong?

With "link-spidering" I mean "everything that pip does when no file is
uploaded and no explicit download link is given".


The term may be incorrect, but I hope that way it's clear.


> One issue *may* be that it's not clear to everyone what "verifiable"
> involves. In another thread I'm trying to clarify with MAL over his
> understanding of how the egenix packages are linked. There is a chain
> of links with hashes, but the PEP doesn't allow for a chain to be
> "verifiable", just a direct link to a downloadable file. Is that what
> you mean by link-spidering? If so, then as it stands link-spidering is
> allowed, but will never be verifiable. I could easily imagine some
> package maintainers feeling that characterising a 2-link chain as "not
> verifiable" is inaccurate and/or scaremongering. Suggestions for
> better terminology would be useful here. (Note that direct links vs
> link chains is an important distinction for pip, as it has performance
> implications as well as security ones - link spidering was a huge
> performance issue at one point, and a lot of the non-controversial
> benefits in PEP 438 are in terms of performance. It would be a shame
> to lose those.)

I think MAL has recently said that he'd use explicit links if allowing
verifiable external links per default is considered (though I may be
twisting his words a little in this summary).

External and verifiable packages have the same security as uploaded files
(though I would like to use sha256 instead of md5 the URL).


What is the use case for the opt-in?  Unless there are many alternatives
for a package, a user is hardly going to reject a package on the grounds
that the combination of launchpad.net + python.org has a cumulative
uptime of 99.99% instead of 99.999%.


So, assuming for a moment that the explicit links are present, it would be
reasonable to have the download work by default and have pip emit a neutral
remark (not even a warning):

"remark: egenix-mx-base is a secure external package."


FreeBSD ports have been using the download-from-many-but-verify strategy
for a long time.  I don't see why users should find this surprising.



Stefan Krah