[Catalog-sig] V3 PEP-draft for transitioning to pypi-hosting of release files

Daniel Holth dholth at gmail.com
Wed Mar 13 20:40:08 CET 2013

On Wed, Mar 13, 2013 at 3:33 PM, M.-A. Lemburg <mal at egenix.com> wrote:
> On 13.03.2013 20:08, Donald Stufft wrote:
>> On Mar 13, 2013, at 2:57 PM, "M.-A. Lemburg" <mal at egenix.com> wrote:
>>> On 13.03.2013 12:21, holger krekel wrote:
>>>> [V3 proposal]
>>> I must say, don't like this change in motivation compared
>>> to V1 and V2.
>>> The original of the discussion was to make PyPI more secure
>>> and the installation process faster and more reliable
>>> by moving away from crawling arbitrary external web pages.
>>> Both can be had by:
>>> * limiting the crawling to package author defined specific
>>>  URLs, with added hashes to make sure that the URLs and
>>>  their target content is not modified (this is the securing
>>>  external downloads part - see here for an example:
>>>  https://pypi.python.org/pypi/egenix-pyopenssl/,
>>>  and
>>> * adding a way for the package authors to say "PyPI, please go
>>>  ahead and cache/copy my distributions files" (this is the
>>>  increase download reliability part - can be had by doing
>>>  opt-in CDN caching/proxying of external links via PyPI)
>>> Now, with V3 of the proposal, you are moving towards a system
>>> that basically says "do it this way, or stay out of our eco
>>> system", which, in my book, is not what the Python eco system
>>> is all about.
>> I don't see how? The -with-externals index will still contain all the existing links, and indeed PJ Elby has already stated that setuptools will move to support this index by default but with proper warnings to people so they know they are installing a package off site.
>> This allows existing tools to be moved to a secure by default position. Allows future tools to choose if they want to enable the existing behavior through use of -with-externals (hopefully with a warning or opt-in sort of thing like laid out by PJE, but it's certainly not required). And even allows users of existing tools to opt into the old behavior via the -i option.
>> Maybe i'm missing it but in what way does this force authors to "do it this way or stay out of our eco system" since all the same options are available as there are today?
> The proposal marks all external links as evil, and instead of
> making external links more secure, the user is left with the option
> to either not enable external links at all, or to let the
> "devil" in :-)
> That's not nice. It's also security theater.
> The real problem is unreviewed code getting executed by users,
> or worse, automated build systems. Yet, we let users believe
> that everything is secured on PyPI.
> Taking an extreme position, it would probably be better just
> leave everything as it is and instead educate users about the
> risk they are taking with a "pip install AngryBirds", signed
> with keys issued by the PSF on the official PyPI server,
> delivered straight to your drive via the latest in crypto
> technology, only to wipe your notebook...
> But then, I don't like extreme positions, so would rather
> like to incrementally improve the situation both from the
> server and the client side, both addressing user and author
> concerns, and keeping the Python eco system a friendly place
> to be.
>>> Your V2 was much more inviting in this respect.

Perhaps it would be better to decide whether it is "reliability
theater" and concentrate on consistency rather than whether the code
actually does what you want. It is nice to have a system that at least
prevents targeted third party bad-package attacks.

More information about the Catalog-SIG mailing list