[Distutils] PEP 458: Surviving a Compromise of PyPI: Round 1

[Justin Cappos]

>
> These are not design - these are implementation details. What's the idea
> about that metadata? I don't get it. I already spent 15 minutes reading
> here and there and still can't see any short concept description. Only
> vague "end-to-end" "security best practices" buzzwords.
>
>
I'm confused by your comments.   Is the document too low level
(implementation details) or high level (vague)?

While the PEP is self-contained, if you want to look at supplemental
materials, the TUF spec has more low level information.   The Ruby folks
have reimplemented TUF based upon it so it seems pretty clear to them.
Conversely, if you want more high level details, the TUF paper is a good
source.   It was published at a top security conference so must have been
readable to (at least) the reviewers.

 > * Developers do not have to opt-in to secure their projects with their
> > own TUF metadata. In that case, PyPI will sign these "unclaimed"
> > projects on their behalf. However, unclaimed projects will not be secure
> > against a PyPI compromise.
>
> So PyPI will sign malicious packages from compromised developer
> account who still uses Python 2 for his enterprise application and was not
> careful enough to pay attention to http://bugs.python.org/issue12226
> and uploaded his package through unsecure PyCon WiFi network.
>

We are trying to protect all users in the case that PyPI or its
infrastructure is compromised.   Ultimately with any software distribution
scheme I am aware of, you trust the author of your software not to be
malicious (or claim the user needs to do something impractical like read
all the code before installing).


> > * To protect against a PyPI compromise, developers may choose to
> > register their public keys with Warehouse and upload their own signed
> > TUF metadata about their projects.
>
> If developer account is hacked, there is no protection. The real protection
> is only known hashsize of the known package of specific version.
> Protection against change of package version contents, not against new
> malicious release / upload. If PyPI is a central repository, and hashsizes
> are not distributed, it is easy to hack PyPI, alter hashsizes, serve
> malicious package to few target recipients and return everything back. If
> hashsizes are distributed, you can't go unnoticed.
>

Look at the document for details, but it isn't possible to do as you say
with the changes detailed in the PEP.   The metadata is laid out so that
even if you compromise every key on the repository, you cannot make
metadata that will have a client trust a project like Django, which manages
its own keys.   This is because the root, targets, claimed and Django keys
are all offline.

> * Therefore, developers do not have to concern themselves with key
> > management in case they leave their projects as "unclaimed". When they
> > do claim their projects, they simply have to register their keys once
> > with Warehouse. After that, they may delegate signing for distributions
> > as they wish without depending on Warehouse.
>
> Use case explanation needed. I'd split in two:
> 1. developers who don't care don't need to care
> 2. developers who care need to "claim project" by registering keys, and
>     then delegate signing without depending on WH to who?
>
> "unclaimed" project. What's this? What is the process of "claiming a
> project"? Is there a better terminology? This reads like picking abandoned
> project or project without authorship.
>

Yes, it is essentially a project where the owner hasn't uploaded a public
key to signal they will manage their own project.   So it seems like you
got the gist of this from the name.

> * Clients will be instructed to first search for a project in the more
> > secure claimed metadata (protected by offline keys) before looking for
> > it in the less secure unclaimed metadata (protected by online keys).
>
> What are the main points that makes offline signatures more secure
> once more?
>

It is not stored on the main server.   The strength of offline / air gapped
keys is well known and used widely in practice.   See the TUF CCS paper for
more details / references.


> > The official PEP is here:
> >
> > http://www.python.org/dev/peps/pep-0458/
>
> Too wordy. Complicated. Picture is good, but didn't help. The document
> should be readable to ordinary corporate Joe, who knows that he got PGP,
> but doesn't know what root keys are signed for. And it should be concise
> enough for people with knowledge. I don't know if it is possible to do
> both.
>

As Nick pointed out, this is meant for PyPI devels, not ordinary Joe.
Ordinary Joe is not going to notice TUF at all unless an attack is underway.

Details for how ordinary Joe publishes a package will be forthcoming, but
will require a very minimal amount of additional effort.

Justin
P.S.   I feel there is quite a bit of confusion.   Feel free to respond to
me directly if you'd prefer to discuss this more without spamming DistUtils
or theupdateframework.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20131122/b9be6a87/attachment.html>