[Catalog-sig] A modest proposal for securing PyPI with TUF

[Daniel Holth]

On Wed, Mar 13, 2013 at 5:13 AM, Trishank Karthik Kuppusamy
<tk47 at students.poly.edu> wrote:
> Hello Nick,
>
>
> On 3/13/13 4:09 AM, Nick Coghlan wrote:
>>
>>
>> - the PSF board generally stays out of the technical details of
>> running the python.org infrastructure, so it's likely that any root
>> keys would be handled by the PSF infrastructure committee. A (2, 4) or
>> (3, 5) trust configuration would likely be manageable at this level.
>
>
> Understood. We think a higher (t, n) [where t out of n signatures are needed
> to trust the metadata for a role] is better for the root role simply because
> its crucial metadata (the authorized keys for top-level roles) should change
> very rarely.
>
>
>> - at the target delegation level, PyPI supports the registration of
>> new projects through the web service (see
>> http://docs.python.org/2/distutils/packageindex.html). If my
>> understanding of target delegation is correct, this means the "simple"
>> and "packages/source/<letter>" delegations will need to be (1, 1) and
>> online.
>> - higher levels of the target delegation hierarchy could conceivably
>> be kept offline, but there seems little value in doing so if they're
>> trusting on online (1, 1) key
>
>
> Fortunately, the "targets/simple" and "targets/packages/(version)/(letter)/"
> roles should not require (1, 1) online keys, as their metadata (simply
> target delegations and no actual target files) should also fluctuate fairly
> rarely. I should make this clearer in our design document.
>
>
>> - many PyPI packages are maintained by single developers, so (1, 1) or
>> (1, n) is likely to be the only generally feasible level of signing at
>> the project level.
>
>
> Yes, the package developers themselves could choose any (t, n) they like. In
> our design, we propose that PyPI could eventually delegate to "stable"
> packages which need little change (and use more security with more offline
> keys) and to "unstable" packages which need frequent change (and use less
> security with more online keys).
>
>
>> With the current focus being on getting an improvement from the status
>> quo that we can successfully deploy in a reasonable period of time,
>> the target delegation side of things probably needs to be
>> substantially simpler in the initial iteration. Yes, it leaves us open
>> to certain vulnerabilities we would like to remove in the long run,
>> but we need to be very cautious in the additional demands we place on
>> the users uploading to PyPI. It may even mean the initial iteration
>> allows projects to rely on a PyPI provided signing key for their TUF
>> metadata, using the existing upload mechanisms to add the files to
>> PyPI.
>
>
> I agree that there is a delicate problem of balancing security with
> usability here, especially in the beginning.
>
> You raised a very good issue there: on first migration, how would PyPI
> accommodate packages which have not had their target files delegated to
> their developers? We imagine that in this case, PyPI could assume initial
> responsibility for these packages, and later PyPI would delegate those
> packages to their respective developers.
>
> Thanks for your input,
> Trishank

With all the different kinds of metadata, It's interesting to note
that currently TUF seems to only be concerned with the available file
names and their integrity. (Some of us will think of PEP 426
"PKG-INFO" first when we hear the word metadata.)

It looks like the D metadata lists all the filenames for Django, and
then Django lists them again with hashes and signatures. Why all the
lists? Does every Django release re-assert all the versions of Django
that are available on the index?

How might I deal with producing the official source distribution
myself and having a friend produce the official Windows build of a
package?

As an aside PyPI has been doubling in size every 1.5 - 2 years.

Thanks

Daniel Holth