[Distutils] Status report on PyPI+pip+TUF

[Trishank Karthik Kuppusamy]

Hello Nick and the PyPI community,

This is a brief status report on the integration of PyPI and pip with TUF.

(A quick reminder: TUF is a general "plug-n-play" update framework 
designed to introduce usable security to community software repositories 
such as PyPI. If you think of PyPI as HTTP, then TUF is like adding SSL, 
and more, to HTTP. More information may be found at 
[https://www.updateframework.com/].)

Firstly, thanks to the generous funding of the National Science 
Foundation, we are pleased to introduce the addition of a full-time 
developer, Vladimir Diaz, to our team. Vladimir has been instrumental to 
the development of TUF, and we are excited to have him join us 
full-time. (Now we do not just have one PhD student who works on TUF 
when he is not busy working on other projects!) We are also happy to 
have a few interns --- Zane Fisher, Tian Tian, John Ward, and Yuyu Zheng 
--- on board for the summer.

Since the security attacks on the Python wiki infrastructure earlier 
this year, we have been closely following Distutils-SIG to see what we 
could do to help secure PyPI. We use Python heavily in all of our 
projects, and would love to help in any way we can.

Here is what we have done:
==========================

1. At PyCon 2013, we showed that pip needs very little modification to 
work with a TUF-enabled PyPI mirror.

2. Soon after (during the spring break), we wrote automation to build a 
TUF-secured PyPI mirror (which is indistinguishable from any other PyPI 
mirror except that it has signed metadata about all of the files on PyPI).

3. At the same time, thanks to efforts of Konstantin Andrianov, we also 
wrote a lot of unit and integration tests to show the attacks that are 
possible without TUF and impossible with TUF.

4. After that, we started investigating the most efficient way to build 
TUF metadata for PyPI. We found that requiring a separate key for every 
package on PyPI may sound like a good idea, but besides generating too 
much metadata, this scheme also makes key management difficult.

Here is what we are doing now:
==============================

We are designing a usable key management scheme, coupled with efficient 
generation and download of metadata, which we think should make for a 
smooth integration of PyPI with TUF. We are actively working on this and 
think that we are almost there. As a conservative estimate, we do not 
believe that this should take longer than two weeks.

Here is what we are going to do next:
=====================================

In about a month, we will present to you a demonstration of a PyPI 
mirror and a pip client which are robust against entire classes of 
security attacks. We welcome you then to try our demo, be really 
critical of it and tell us what you think about what we could do better. 
Our goal with TUF is to provide a framework that works with as many 
software community repositories as possible and that secures as many 
users as possible.

More details on our development are available at our mailing list: 
https://groups.google.com/forum/#!forum/theupdateframework

We hope this gives you a good idea of the current status of integrating 
TUF with PyPI and pip. Let us know if you have questions.

Thanks,
The TUF team