[Distutils] Status report on PyPI+pip+TUF
Trishank Karthik Kuppusamy
tk47 at students.poly.edu
Wed Jul 31 13:27:20 CEST 2013
Hello Nick and the PyPI community,
This is a brief status report on the integration of PyPI and pip with TUF.
(A quick reminder: TUF is a general "plug-n-play" update framework
designed to introduce usable security to community software repositories
such as PyPI. If you think of PyPI as HTTP, then TUF is like adding SSL,
and more, to HTTP. More information may be found at
Firstly, thanks to the generous funding of the National Science
Foundation, we are pleased to introduce the addition of a full-time
developer, Vladimir Diaz, to our team. Vladimir has been instrumental to
the development of TUF, and we are excited to have him join us
full-time. (Now we do not just have one PhD student who works on TUF
when he is not busy working on other projects!) We are also happy to
have a few interns --- Zane Fisher, Tian Tian, John Ward, and Yuyu Zheng
--- on board for the summer.
Since the security attacks on the Python wiki infrastructure earlier
this year, we have been closely following Distutils-SIG to see what we
could do to help secure PyPI. We use Python heavily in all of our
projects, and would love to help in any way we can.
Here is what we have done:
1. At PyCon 2013, we showed that pip needs very little modification to
work with a TUF-enabled PyPI mirror.
2. Soon after (during the spring break), we wrote automation to build a
TUF-secured PyPI mirror (which is indistinguishable from any other PyPI
mirror except that it has signed metadata about all of the files on PyPI).
3. At the same time, thanks to efforts of Konstantin Andrianov, we also
wrote a lot of unit and integration tests to show the attacks that are
possible without TUF and impossible with TUF.
4. After that, we started investigating the most efficient way to build
TUF metadata for PyPI. We found that requiring a separate key for every
package on PyPI may sound like a good idea, but besides generating too
much metadata, this scheme also makes key management difficult.
Here is what we are doing now:
We are designing a usable key management scheme, coupled with efficient
generation and download of metadata, which we think should make for a
smooth integration of PyPI with TUF. We are actively working on this and
think that we are almost there. As a conservative estimate, we do not
believe that this should take longer than two weeks.
Here is what we are going to do next:
In about a month, we will present to you a demonstration of a PyPI
mirror and a pip client which are robust against entire classes of
security attacks. We welcome you then to try our demo, be really
critical of it and tell us what you think about what we could do better.
Our goal with TUF is to provide a framework that works with as many
software community repositories as possible and that secures as many
users as possible.
More details on our development are available at our mailing list:
We hope this gives you a good idea of the current status of integrating
TUF with PyPI and pip. Let us know if you have questions.
The TUF team
More information about the Distutils-SIG