[Distutils] PEP470 installation security problems

[Donald Stufft]

> On Oct 8, 2014, at 8:55 AM, M.-A. Lemburg <mal at egenix.com> wrote:
> 
> On 08.10.2014 14:30, Nick Coghlan wrote:
>> On 8 October 2014 22:22, Donald Stufft <donald at stufft.io> wrote:
>>> 
>>>> On Oct 8, 2014, at 8:17 AM, holger krekel <holger at merlinux.eu> wrote:
>>>> 
>>>> Also, i am worried on principle grounds if pip maintainers are putting
>>>> themselves outside PEP reach, yet pip is distributed along with Python.
>>> 
>>> We’re not “putting ourselves outside of PEP reach”. We are an external
>>> project and we are not bound by the PEP process. Devpi, py.test, Django,
>>> requests, etc are also not bound by the PEP process.
>> 
>> Note also that even for CPython itself, it is *up to us as core
>> developers* to decide when something needs to be escalated through the
>> PEP process. The vast majority of CPython changes are handled directly
>> through the issue tracker, and there's still the occasional change
>> that doesn't even make it that far (e.g. if we notice a problem while
>> working on something else, we have the option of just committing the
>> fix directly).
>> 
>> PEPs are primarily for changes which have broad ecosystem implications
>> where the additional overhead is justified. We don't write PEPs for
>> every change to the CPython command line interface (e.g. there's no
>> PEP for isolated mode), and the same kind of assessment of external
>> impact applies to pip and the PyPA in general when decided whether a
>> change can be handled within the scope of an individual project, or if
>> it needs to be escalated for broader discussion.
> 
> I don't follow Donald's reasoning and I'm not sure I understand
> whether your comments are meant as clarification of pip being
> subject to the PEP process or support for Donald's reasoning :-)
> 
> Changes to pip and PyPI *do* have a global effect on the Python
> ecosystem and thus need to be covered by the PEP process.
> 
> If pip decides to go with a strategy that ignores this, I think we
> have a problem. The core developers put trust into pip when allowing
> it to (effectively) get distributed with Python and making it the
> default Python packaging manager. Please use that trust with the
> appropriate care and respect.

I don’t think we’ve *ever* not used that trust with care and respect and
we’ve been trusted by the Python community for far longer than PEP 453
has existed. We attempt to follow PEPs where we can and where they make
good sense. Nobody on the pip team is saying we’re going to flat out
ignore PEPs or whatever.

We (or at least I am) are saying that dictating UX via PEP process has
been shown to us *not* to work and that we are not obligated to implement
or listen to a PEP. This was explicitly spelled out in PEP 453 that we
remain an external project even with the fact we’re now bundled with
Python. This does not mean we won’t generally try to use the PEP process
where our changes have cross cutting concerns between different projects
but it does mean that we implement or follow PEPs at our discretion. This
isn’t up for debate, it was an explicit inclusion in PEP 453 and if there
was a problem with pip maintaining it’s own project the time to bring that
up was a year ago. 

---
Donald Stufft
PGP: 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA