[Distutils] Surviving a Compromise of PyPI - PEP 458 and 480

Nick Coghlan ncoghlan at gmail.com
Mon Dec 22 17:30:24 CET 2014

On 23 December 2014 at 01:46, Vladimir Diaz <vladimir.v.diaz at gmail.com>

> ::ping::
> Has anyone in the community gotten a chance to review PEP 458 and/or PEP
> 480?  Community feedback would be appreciated.

Sorry about that Vladimir - I think the main PyPA devs have been focused on
getting PEP 440 implemented, and the associated setuptools, pip and
virtualenv updates out the door, and now we're into the end of year holiday
period for many of us.

>From my perspective, the split into two PEPs meant most of the areas I have
doubts about have been moved to the end-to-end security model in PEP 480,
leaving PEP 458 to cover the simpler task of securing the link from PyPI to
the end user in such a way that public mirrors of packages can be trusted
to accurately reflect the content published by PyPI.

The new appendix C raises some good questions regarding how we would like
to deal with externally hosted packages. I personally agree with the PEP's
recommendation to require TUF metadata generated with a developer key in
that case, with a slight preference for publishing that metadata (including
the corresponding security delegation) through PyPI. However, I'm open to
practicality arguments suggesting that one of the other options may be more
feasible for maintainers of externally hosted repositories.

The root key management question is the other one that will be interesting,
given the distributed nature of both PSF Infrastructure maintenance and
pip/PyPI development. A partial root key compromise would effectively
become a CVE against pip and CPython (and hence flowing on to Linux
distributions and potentially other redistributors), with the severity
depending on whether or not the signing threshold has been reached.

I suspect before we sign off on that last part, we're going to need to get
quite specific with exactly *who* will hold signing authority over those
root keys (just as the CPython release PEPs specifically name the release
managers for the source tarballs and the platform specific binaries, who
then have signing authority for their respective artifacts)


> Thanks,
> Vlad
> On Wed, Dec 10, 2014 at 10:16 PM, Vladimir Diaz <vladimir.v.diaz at gmail.com
> > wrote:
>> Hello everyone,
>> I am a research programmer at the NYU School of Engineering.  My
>> colleagues (Trishank Kuppusamy and Justin Cappos) and I are requesting
>> community feedback on our proposal, "Surviving a Compromise of PyPI."  The
>> two-stage proposal can be reviewed online at:
>> PEP 458
>> http://legacy.python.org/dev/peps/pep-0458/
>> PEP 480
>> http://legacy.python.org/dev/peps/pep-0480/
>> Summary of the Proposal:
>> "Surviving a Compromise of PyPI" proposes how the Python Package Index
>> (PyPI) can be amended to better protect end users from altered or malicious
>> packages, and to minimize the extent of PyPI compromises against affected
>> users.  The proposed integration allows package managers such as pip to be
>> more secure against various types of security attacks on PyPI and defend
>> end users from attackers responding to package requests. Specifically,
>> these PEPs describe how PyPI processes should be adapted to generate and
>> incorporate repository metadata, which are signed text files that describe
>> the packages and metadata available on PyPI.  Package managers request
>> (along with the packages) the metadata on PyPI to verify the authenticity
>> of packages before they are installed.  The changes to PyPI and tools will
>> be minimal by leveraging a library, The Update Framework
>> <https://github.com/theupdateframework/tuf>, that generates and
>> transparently validates the relevant metadata.
>> The first stage of the proposal (PEP 458
>> <http://legacy.python.org/dev/peps/pep-0458/>) uses a basic security
>> model that supports verification of PyPI packages signed with cryptographic
>> keys stored on PyPI, requires no action from developers and end users, and
>> protects against malicious CDNs and public mirrors. To support continuous
>> delivery of uploaded packages, PyPI administrators sign for uploaded
>> packages with an online key stored on PyPI infrastructure. This level of
>> security prevents packages from being accidentally or deliberately tampered
>> with by a mirror or a CDN because the mirror or CDN will not have any of
>> the keys required to sign for projects.
>> The second stage of the proposal (PEP 480
>> <http://legacy.python.org/dev/peps/pep-0480/>) is an extension to the
>> basic security model (discussed in PEP 458) that supports end-to-end
>> verification of signed packages. End-to-end signing allows both PyPI and
>> developers to sign for the packages that are downloaded by end users.  If
>> the PyPI infrastructure were to be compromised, attackers would be unable
>> to serve malicious versions of these packages without access to the
>> project's developer key.  As in PEP 458, no additional action is required
>> by end users.  However, PyPI administrators will need to periodically
>> (perhaps every few months) sign metadata with an offline key.  PEP 480 also
>> proposes an easy-to-use key management solution for developers, how to
>> interface with a potential build farm on PyPI infrastructure, and discusses
>> the security benefits of end-to-end signing.  The second stage of the
>> proposal simultaneously supports real-time project registration and
>> developer signatures, and when configured to maximize security on PyPI,
>> less than 1% of end users will be at risk even if an attacker controls PyPI
>> and goes undetected for a month.
>> We thank Nick Coghlan and Donald Stufft for their valuable contributions,
>> and Giovanni Bajo and Anatoly Techtonik for their feedback.
>> Thanks,
>> PEP 458 & 480 authors.
>> _______________________________________________
>> Distutils-SIG maillist  -  Distutils-SIG at python.org
>> https://mail.python.org/mailman/listinfo/distutils-sig
> _______________________________________________
> Distutils-SIG maillist  -  Distutils-SIG at python.org
> https://mail.python.org/mailman/listinfo/distutils-sig

Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20141223/96b47df7/attachment-0001.html>

More information about the Distutils-SIG mailing list