[Distutils] PEP470 installation security problems

[Donald Stufft]

> On Oct 8, 2014, at 6:06 AM, holger krekel <holger at merlinux.eu> wrote:
> 
> On Wed, Oct 08, 2014 at 05:44 -0400, Donald Stufft wrote:
>> 
>> I think raising the issue is FUDish because it has nothing to do with using
>> multi repository support for things that are registered on PyPI. 
> 
> Well, the PEP has two central paragraphs motivating multi-index operations:
> 
>    The two common installer tools, pip and easy_install/setuptools, both
>    support the concept of additional locations to search for files to
>    satisify the installation requirements and have done so for many years.
>    This means that there is no need to "phase" in a new flag or concept and
>    the solution to installing a project from a repository other than PyPI
>    will function regardless of how old (within reason) the end user's
>    installer is. Not only has this concept existed in the Python tooling
>    for some time, but it is a concept that exists across languages and even
>    extending to the OS level with OS package tools almost universally using
>    multiple repository support making it extremely likely that someone is
>    already familar with the concept.
> 
>    Additionally, the multiple repository approach is a concept that is
>    useful outside of the narrow scope of allowing projects which wish
>    to be included on the index portion of PyPI but do not wish to
>    utilize the repository portion of PyPI. This includes places where a
>    company may wish to host a repository that contains their internal
>    packages or where a project may wish to have multiple "channels" of
>    releases, such as alpha, beta, release candidate, and final release.
> 
> and then it concretely suggests "--extra-index-url" and gives an example.
> It does not say that this is only good if you are using private projects
> that have a presence on PyPI.  It rather suggests multi-index is the thing 
> to go for today, generally, does it not?
> 
> Given that PyPI is a wiki and Linux Distros are a curated index, i
> insist it's dangerous to recommend to mix multiple indexes with pip if
> you don't know quite exactly what you are doing.  Do you really disagree
> on this?

It is not dangerous to mix multiple indexes in the case that PEP 470 is
specifying, which is when you want to have files for a project listed on the
PyPI index hosted on a different repository. The use of --extra-index-url in
PEP 470 is to show how someone would add one of the extra repositories for a
project that is indexed on PyPI, which is again roughly as safe as installing
from PyPI at all.

If you use the multiple repository support to install things which are not
claimed on PyPI and you do not disable the PyPI index, then yes that is
dangerous. It also has nothing to do with whether it's safe for someone to
add an additional repository that points to the repository that PIL is located
at.

I've also never suggested to anyone that their company should rely on PyPI
and instead I point them towards either making their own repository with
Apache/Index/Twisted Web or using devpi. My goal is to make PyPI as safe as
possible for people who don't do that, but there are limits to what is possible.

---
Donald Stufft
PGP: 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA