[Distutils] Surviving a Compromise of PyPI - PEP 458 and 480

Mon Dec 22 19:15:36 CET 2014

On Mon, Dec 22, 2014 at 11:30 AM, Nick Coghlan <ncoghlan at gmail.com> wrote:

> On 23 December 2014 at 01:46, Vladimir Diaz <vladimir.v.diaz at gmail.com>
> wrote:
>
>> ::ping::
>>
>> Has anyone in the community gotten a chance to review PEP 458 and/or PEP
>> 480?  Community feedback would be appreciated.
>>
>
> Sorry about that Vladimir - I think the main PyPA devs have been focused
> on getting PEP 440 implemented, and the associated setuptools, pip and
> virtualenv updates out the door, and now we're into the end of year holiday
> period for many of us.
>

Considering that the previous drafts of PEP 458 generated quite a bit of
feedback and questions, I was surprised by the lack of responses this time
around (almost relieved :).  Receiving feedback from the main PyP
developers after the holiday period is definitely not a problem.

> From my perspective, the split into two PEPs meant most of the areas I
> have doubts about have been moved to the end-to-end security model in PEP
> 480, leaving PEP 458 to cover the simpler task of securing the link from
> PyPI to the end user in such a way that public mirrors of packages can be
> trusted to accurately reflect the content published by PyPI.
>

I think splitting the proposal into two PEPs was the right decision.  We
hope working with Donald on the end-to-end security model (PEP 480), and
feedback from the community will help to address any remaining questions.
Excluding the end-to-end option from the revised version of PEP 458 also
made room for an overview of the metadata and framework, which was
requested by multiple members of the community.

> The new appendix C raises some good questions regarding how we would like
> to deal with externally hosted packages. I personally agree with the PEP's
> recommendation to require TUF metadata generated with a developer key in
> that case, with a slight preference for publishing that metadata (including
> the corresponding security delegation) through PyPI. However, I'm open to
> practicality arguments suggesting that one of the other options may be more
> feasible for maintainers of externally hosted repositories.
>
> The root key management question is the other one that will be
> interesting, given the distributed nature of both PSF Infrastructure
> maintenance and pip/PyPI development. A partial root key compromise would
> effectively become a CVE against pip and CPython (and hence flowing on to
> Linux distributions and potentially other redistributors), with the
> severity depending on whether or not the signing threshold has been reached.
>
> I suspect before we sign off on that last part, we're going to need to get
> quite specific with exactly *who* will hold signing authority over those
> root keys (just as the CPython release PEPs specifically name the release
> managers for the source tarballs and the platform specific binaries, who
> then have signing authority for their respective artifacts)
>

We can decide in the coming weeks who should be explicitly named, and the
number of individuals that should hold signing authority over the root
keys.  If reaching a reasonable threshold of root keys remains a problem,
we can also assist with meeting this threshold.  I will discuss this matter
with the rest of the authors to make sure we are all on the same page wrt
root key management.

Regards,
Nick.

> Thanks,
> Vlad
>
> On Wed, Dec 10, 2014 at 10:16 PM, Vladimir Diaz <vladimir.v.diaz at gmail.com
> > wrote:
>
>> Hello everyone,
>>
>> I am a research programmer at the NYU School of Engineering.  My
>> colleagues (Trishank Kuppusamy and Justin Cappos) and I are requesting
>> community feedback on our proposal, "Surviving a Compromise of PyPI."  The
>> two-stage proposal can be reviewed online at:
>>
>> PEP 458
>> http://legacy.python.org/dev/peps/pep-0458/
>>
>> PEP 480
>> http://legacy.python.org/dev/peps/pep-0480/
>>
>>
>> Summary of the Proposal:
>>
>> "Surviving a Compromise of PyPI" proposes how the Python Package Index
>> (PyPI) can be amended to better protect end users from altered or malicious
>> packages, and to minimize the extent of PyPI compromises against affected
>> users.  The proposed integration allows package managers such as pip to be
>> more secure against various types of security attacks on PyPI and defend
>> end users from attackers responding to package requests. Specifically,
>> these PEPs describe how PyPI processes should be adapted to generate and
>> incorporate repository metadata, which are signed text files that describe
>> the packages and metadata available on PyPI.  Package managers request
>> (along with the packages) the metadata on PyPI to verify the authenticity
>> of packages before they are installed.  The changes to PyPI and tools will
>> be minimal by leveraging a library, The Update Framework
>> <https://github.com/theupdateframework/tuf>, that generates and
>> transparently validates the relevant metadata.
>>
>> The first stage of the proposal (PEP 458
>> <http://legacy.python.org/dev/peps/pep-0458/>) uses a basic security
>> model that supports verification of PyPI packages signed with cryptographic
>> keys stored on PyPI, requires no action from developers and end users, and
>> protects against malicious CDNs and public mirrors. To support continuous
>> delivery of uploaded packages, PyPI administrators sign for uploaded
>> packages with an online key stored on PyPI infrastructure. This level of
>> security prevents packages from being accidentally or deliberately tampered
>> with by a mirror or a CDN because the mirror or CDN will not have any of
>> the keys required to sign for projects.
>>
>> The second stage of the proposal (PEP 480
>> <http://legacy.python.org/dev/peps/pep-0480/>) is an extension to the
>> basic security model (discussed in PEP 458) that supports end-to-end
>> verification of signed packages. End-to-end signing allows both PyPI and
>> developers to sign for the packages that are downloaded by end users.  If
>> the PyPI infrastructure were to be compromised, attackers would be unable
>> to serve malicious versions of these packages without access to the
>> project's developer key.  As in PEP 458, no additional action is required
>> by end users.  However, PyPI administrators will need to periodically
>> (perhaps every few months) sign metadata with an offline key.  PEP 480 also
>> proposes an easy-to-use key management solution for developers, how to
>> interface with a potential build farm on PyPI infrastructure, and discusses
>> the security benefits of end-to-end signing.  The second stage of the
>> proposal simultaneously supports real-time project registration and
>> developer signatures, and when configured to maximize security on PyPI,
>> less than 1% of end users will be at risk even if an attacker controls PyPI
>> and goes undetected for a month.
>>
>> We thank Nick Coghlan and Donald Stufft for their valuable contributions,
>> and Giovanni Bajo and Anatoly Techtonik for their feedback.
>>
>> Thanks,
>> PEP 458 & 480 authors.
>>
>> _______________________________________________
>> Distutils-SIG maillist  -  Distutils-SIG at python.org
>> https://mail.python.org/mailman/listinfo/distutils-sig
>>
>>
>
> _______________________________________________
> Distutils-SIG maillist  -  Distutils-SIG at python.org
> https://mail.python.org/mailman/listinfo/distutils-sig
>
>
>
>
> --
> Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20141222/b7d348a4/attachment.html>