[Distutils] [tuf] Re: vetting, signing, verification of release files

[Justin Cappos]

If there is not a compromise of PyPI, then all updates happen essentially
instantly.

Developers that do not sign packages and so PyPI signs them, may have their
newest packages remain unavailable for a period of up to 3 months *if there
is a compromise of PyPI*.

Thanks,
Justin



On Wed, Jul 17, 2013 at 9:46 PM, Donald Stufft <donald at stufft.io> wrote:

>
> On Jul 17, 2013, at 9:29 PM, Trishank Karthik Kuppusamy <
> tk47 at students.poly.edu> wrote:
>
> > On 07/18/2013 03:24 AM, Ronald Oussoren wrote:
> >> I'm trying to understand what this means for package maintainers. If I
> understand you correctly maintainers would upload packages just like they
> do now, and packages are then automaticly signed by the "unstable" role.
>  Then some manual process by the PyPI maintainers can sign a package with a
> stable row. Is that correct? If it is, how is this supposed to scale? The
> contents of PyPI is currently not vetted at all, and it seems to me that
> manually vetting uploads for even the most popular packages would be a
> significant amount of work that would have to be done by what's likely a
> small set of volunteers.
> >
> > I think Daniel put it best when he said that we have been focusing too
> much on deciding whether or not a package is malicious. As he said, it is
> important that any security proposal must limit what targeted attacks on
> the PyPI infrastructure can do.
>
> As I've mentioned before an online key (as is required by PyPI) means that
> if someone compromises PyPI they compromise the key. It seems to me that
> TUF is really designed to handle the case of the Linux distribution (or
> similar) where you have vetted maintainers who are given a subsection of
> the total releases. However PyPI does not have vetted authors nor the man
> power to sign authors keys offline.
>
> PyPI and a Linux Distro repo solve problems that appear similar but are
> actually quite different under the surface.
>
> I do agree however that PyPI should not attempt to discern what is
> malicious or not.
>
> >
> > You are right that asking people to vet through packages for inclusion
> into the stable role would be generally unscalable. I think the best way to
> think about it is that we can mostly decide a "stable" set of packages with
> a simple rule, and then *choose* to interfere (if necessary) with decisions
> on which packages go in or out of the stable role. The stable role simply
> has to sign this automatically computed set of "stable" packages every now
> and then, so that the impacts of attacks on the PyPI infrastructure are
> limited. Users who install the same set of stable packages will see the
> installation of the same set of intended packages.
> >
> > Presently, I use a simple heuristic to compute a nominal set of stable
> packages: all files older than 3 months are considered to be "stable".
> There is no consideration of whether a package is malicious here; just that
> it has not changed long enough to be considered mature.
> >
> >> Also, what are you supposed to do when FooBar 2.0 is signed by the
> stable role and FooBar 2.0.1 is only signed by the unstable role, and you
> try to fetch FooBar 2.0.* (that is, 2.0 or any 2.0.x point release)?
> >>
> >
> > In this case, I expect that since we have asked pip to install FooBar
> 2.0.*, it will first fetch the /simple/FooBar/ PyPI metadata (distinct from
> TUF metadata) to see what versions of the FooBar package are available. If
> FooBar 2.0.1 was recently added, then the latest version of the
> /simple/FooBar/ metadata would have been signed for the unstable role.
> There are two cases for the stable role:
> >
> > 1. The stable role has also signed for the FooBar 2.0.1 package. In this
> case, pip would find FooBar 2.0.1 and install it.
> > 2. The stable role has not yet signed for the FooBar 2.0.1 package. In
> this case, pip would find FooBar 2.0 and install it.
>
> And things are stable after 3 months? This sounds completely insane. So if
> a package releases a security update it'll be 3 months until people get
> that fix by default?
>
> >
> > Why would this happen? In this case, we have specified in the TUF
> metadata that if the same file (in this case, the /simple/FooBar/ HTML
> file) has been signed for by both the stable and unstable roles, then the
> client must prefer the version from the stable role.
> >
> > Of course, there are questions about timeliness. Sometimes users want
> the latest packages, or the developers of the packages themselves may want
> this to be the case. For the purposes of bootstrapping PyPI with TUF, we
> have presently decided to simplify key management and allow for the
> protection of some valuable packages on PyPI (with limited timeliness
> trade-off) while allowing for the majority of the packages to be
> continuously released.
> >
> > There are a few ways to ensure that the latest intended versions of the
> FooBar package will be installed:
> > 1. Do not nominate FooBar into the "stable" set of packages, which
> should ideally be reserved --- for initial bootstrapping purposes at least
> --- for perhaps what the community thinks are the "canonical" packages that
> must initially be protected from attacks.
> > 2. The stable role may delegate its responsibility about information on
> the FooBar package to the FooBar package developers themselves.
> > 3. Explore different rules (other than just ordering roles by trust) to
> balance key management, timeliness and other issues without significantly
> sacrificing security.
> >
> > We welcome your thoughts here. For the moment, we are planning to wrap
> up as soon as possible our experiments on how PyPI+pip perform with and
> without TUF with this particular scheme of stable and unstable roles.
> >
>
>
> -----------------
> Donald Stufft
> PGP: 0x6E3CBCE93372DCFA // 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372
> DCFA
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20130717/48435bb4/attachment-0001.html>