[Distutils] Surviving a Compromise of PyPI - PEP 458 and 480

[Vladimir Diaz]

Paul,

I understand your point, and made an attempt at what you're suggesting with
my initial post to this mailing list (addressing a different target
audience):

"Surviving a Compromise of PyPI" proposes how the Python Package Index
(PyPI) can be amended to better protect end users from altered or malicious
packages, and to minimize the extent of PyPI compromises against affected
users.  The proposed integration allows package managers such as pip to be
more secure against various types of security attacks on PyPI and defend
end users from attackers responding to package requests. Specifically,
these PEPs describe how PyPI processes should be adapted to generate and
incorporate repository metadata, which are signed text files that describe
the packages and metadata available on PyPI.  Package managers request
(along with the packages) the metadata on PyPI to verify the authenticity
of packages before they are installed.  The changes to PyPI and tools will
be minimal by leveraging a library, The Update Framework
<https://github.com/theupdateframework/tuf>, that generates and
transparently validates the relevant metadata.

The first stage of the proposal (PEP 458
<http://legacy.python.org/dev/peps/pep-0458/>) uses a basic security model
that supports verification of PyPI packages signed with cryptographic keys
stored on PyPI, requires no action from developers and end users, and
protects against malicious CDNs and public mirrors. To support continuous
delivery of uploaded packages, PyPI administrators sign for uploaded
packages with an online key stored on PyPI infrastructure. This level of
security prevents packages from being accidentally or deliberately tampered
with by a mirror or a CDN because the mirror or CDN will not have any of
the keys required to sign for projects."

Perhaps parts of this introduction (and your suggestion) will work better
as an abstract, instead of assuming that the PyPI admins will be the only
ones reading PEP 458.

Thanks again for all the feedback.

On Fri, Jan 2, 2015 at 12:23 PM, Paul Moore <p.f.moore at gmail.com> wrote:

> On 2 January 2015 at 16:14, Vladimir Diaz <vladimir.v.diaz at gmail.com>
> wrote:
> > There's a lot of background documentation and technical details excluded
> > from the PEPs (to avoid turning the PEP into a 15+ page behemoth), but I
> do
> > agree that we should explicitly cover some of these implementation
> details
> > in PEP 458.  Subsections on the exact format of metadata, explanation on
> how
> > metadata is signed, and how the roles are "delegated" with the library,
> > still remain.  As Paul as indicated, terminology can also be improved so
> as
> > to be more readable for "non-experts."
>
> There's a tension here, in that while the detail is useful, the
> non-expert content needs to be *less* words, not more - the problem is
> one of an intimidating wall of text, much more than that the concepts
> are difficult. Adding more content, however necessary it is to the
> implementation side, probably detracts here.
>
> Just as an example, take the first paragraph of the abstract:
>
> """
> This PEP proposes how the Python Package Index (PyPI [1] ) should be
> integrated with The Update Framework [2] (TUF). TUF was designed to be
> a flexible security add-on to a software updater or package manager.
> The framework integrates best security practices such as separating
> role responsibilities, adopting the many-man rule for signing
> packages, keeping signing keys offline, and revocation of expired or
> compromised signing keys. For example, attackers would have to steal
> multiple signing keys stored independently to compromise a role
> responsible for specifying a repository's available files. Another
> role responsible for indicating the latest snapshot of the repository
> may have to be similarly compromised, and independent of the first
> compromised role.
> """
>
> There's no way I, even with the benefit of the discussions both on the
> list here and off-list, can skim that sentence and get anything more
> meaningful than "blah, blah, blah" from it. Taking a moment to think
> about it is fine, but at that point you've lost me (in terms of
> encouraging me to read the document, rather than just debate based on
> what I think it's going to say).
>
> None of the "best security practices" mentioned in sentence 3 mean
> anything to me - they all basically look like "do more security stuff"
> to me, and "doing security stuff" unfortunately, from my experience,
> translates to "jump through annoying hoops that stop me getting my job
> done"... Similarly, the last 2 sentences (the example) read as saying
> "attackers have to work harder". Well good, I'd sort of hope the PEP
> provided a benefit of some sort :-)
>
> The following (somewhat off the cuff) may work better:
>
> """
> This PEP proposes some changes to the Python Package Index (PyPI)
> infrastructure to enhance its resistance to attack without requiring
> any changes in workflow from project authors. It is similar to recent
> changes to use TLS for all PyPI traffic, but offers additional
> security over TLS. Future PEPs may look at further security
> enhancements that could require additional action from project
> authors, but this PEP is restricted to the PyPI infrastructure and
> mirroring software.
> """
>
> Sorry for what is pretty much a hatchet job on a single paragraph take
> out of context from the PEP, but hopefully you can see what I'm trying
> to say - skip details of security concepts, attack vectors, etc, and
> concentrate on "we're making things better - it's as low impact as the
> change to TLS - project authors won't need to do anything (at this
> stage)". Those are the key messages to get across straight away. A few
> more paragraphs in the abstract explaining in non-technical terms
> what's being done and why, and you're done. You can say "the rest of
> the document is the technical details for people with an understanding
> of (or interest in) the security concepts" and get on with the
> specifics.
>
> If you can get the non-specialists to read just the abstract section,
> and go away knowing they are happy to let the expects get on with the
> process, you've won. Put the details and the scary terms in the "for
> experts and interested parties only" main body.
>
> Paul.
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20150102/cbdc4d27/attachment.html>