Summary of PyPI overhaul in new LWN article

Today, LWN published my new article "A new package index for Python". https://lwn.net/Articles/751458/ In it, I discuss security, policy, UX and developer experience changes in the 15+ years since PyPI's founding, new features (and deprecated old features) in Warehouse, and future plans. Plus: screenshots! If you aren't already an LWN subscriber, you can use this subscriber link for the next week to read the article despite the LWN paywall. https://lwn.net/SubscriberLink/751458/81b2759e7025d6b9/ This summary should help occasional Python programmers -- and frequent Pythonists who don't follow packaging/distro discussions closely -- understand why a new application is necessary, what's new, what features are going away, and what to expect in the near future. I also hope it catches the attention of downstreams that ought to migrate. -- Sumana Harihareswara Warehouse project manager Changeset Consulting https://changeset.nyc

On Wed, Apr 11, 2018 at 10:30 PM, Sumana Harihareswara <sh@changeset.nyc> wrote:
Thanks for the summary, and all your hard work, Sumana :) Happy to see this bit about TUF in future horizons: Warehouse's signature handling demonstrates a shift in Python's thinking
We would love to help with this efforts any way we can. -- curl https://keybase.io/trishankdatadog/pgp_keys.asc | gpg --import

From "TUF, Warehouse, Pip, PyPA, ld-signatures, ed25519"
https://mail.python.org/pipermail/distutils-sig/2018-March/032081.html :
Read and review these PEPs: "PEP 458 -- Surviving a Compromise of PyPI" https://www.python.org/dev/peps/pep-0458/" "PEP 480 -- Surviving a Compromise of PyPI: The Maximum Security Model" https://www.python.org/dev/peps/pep-0480/ On Thursday, April 12, 2018, Trishank Kuppusamy < trishank.kuppusamy@datadoghq.com> wrote:

Hi Wes, On Thu, Apr 12, 2018 at 1:22 PM, Wes Turner <wes.turner@gmail.com> wrote:
For client support in pip, we are discussing this patch with Donald Stufft: https://github.com/pypa/pip/compare/release/9.0.3...trishankatdatadog:trisha... Happy to discuss more with anyone interested! -- curl https://keybase.io/trishankdatadog/pgp_keys.asc | gpg --import

On Thu, Apr 12, 2018 at 5:29 PM, Trishank Kuppusamy < trishank.kuppusamy@datadoghq.com> wrote:
Sorry for the noise, but our repo has now moved here: https://github.com/pypa/pip/compare/9.0.3...DataDog: trishankatdatadog/9.0.3.tuf-in-toto -- https://keybase.io/trishankdatadog

Hey, what's the latest on this? - Python PEP458: - https://www.pypa.io/en/latest/roadmap/#pypi-integrate-tuf On Wed, Apr 18, 2018 at 5:43 PM Trishank Kuppusamy < trishank.kuppusamy@datadoghq.com> wrote:

On Thu, Jul 26, 2018 at 1:37 PM Wes Turner <wes.turner@gmail.com> wrote:
I'm not aware of what is happening on the Warehouse end, but I'm happy to report we have a version of pip + TUF Datadog is using in production: https://github.com/DataDog/pip/tree/trishankatdatadog/10.0.1.tuf I'm fairly confident we can use this same version on Warehouse + TUF :) -- https://keybase.io/trishankdatadog

On Thu, Jul 26, 2018 at 1:58 PM Trishank Kuppusamy < trishank.kuppusamy@datadoghq.com> wrote:
When I get some downtime, I'll start writing some code for Warehouse + TUF... -- https://keybase.io/trishankdatadog

Hi Wes, Many members of the PyPA have been trying to obtain funding to try and work on this and other security features I believe. I don't believe, but could be wrong, anything has occurred in implementation here. But Ernest, Donald and Dustin would know best. Are you interested in helping? Cooper

On Wed, Apr 11, 2018 at 10:30 PM, Sumana Harihareswara <sh@changeset.nyc> wrote:
Thanks for the summary, and all your hard work, Sumana :) Happy to see this bit about TUF in future horizons: Warehouse's signature handling demonstrates a shift in Python's thinking
We would love to help with this efforts any way we can. -- curl https://keybase.io/trishankdatadog/pgp_keys.asc | gpg --import

From "TUF, Warehouse, Pip, PyPA, ld-signatures, ed25519"
https://mail.python.org/pipermail/distutils-sig/2018-March/032081.html :
Read and review these PEPs: "PEP 458 -- Surviving a Compromise of PyPI" https://www.python.org/dev/peps/pep-0458/" "PEP 480 -- Surviving a Compromise of PyPI: The Maximum Security Model" https://www.python.org/dev/peps/pep-0480/ On Thursday, April 12, 2018, Trishank Kuppusamy < trishank.kuppusamy@datadoghq.com> wrote:

Hi Wes, On Thu, Apr 12, 2018 at 1:22 PM, Wes Turner <wes.turner@gmail.com> wrote:
For client support in pip, we are discussing this patch with Donald Stufft: https://github.com/pypa/pip/compare/release/9.0.3...trishankatdatadog:trisha... Happy to discuss more with anyone interested! -- curl https://keybase.io/trishankdatadog/pgp_keys.asc | gpg --import

On Thu, Apr 12, 2018 at 5:29 PM, Trishank Kuppusamy < trishank.kuppusamy@datadoghq.com> wrote:
Sorry for the noise, but our repo has now moved here: https://github.com/pypa/pip/compare/9.0.3...DataDog: trishankatdatadog/9.0.3.tuf-in-toto -- https://keybase.io/trishankdatadog

Hey, what's the latest on this? - Python PEP458: - https://www.pypa.io/en/latest/roadmap/#pypi-integrate-tuf On Wed, Apr 18, 2018 at 5:43 PM Trishank Kuppusamy < trishank.kuppusamy@datadoghq.com> wrote:

On Thu, Jul 26, 2018 at 1:37 PM Wes Turner <wes.turner@gmail.com> wrote:
I'm not aware of what is happening on the Warehouse end, but I'm happy to report we have a version of pip + TUF Datadog is using in production: https://github.com/DataDog/pip/tree/trishankatdatadog/10.0.1.tuf I'm fairly confident we can use this same version on Warehouse + TUF :) -- https://keybase.io/trishankdatadog

On Thu, Jul 26, 2018 at 1:58 PM Trishank Kuppusamy < trishank.kuppusamy@datadoghq.com> wrote:
When I get some downtime, I'll start writing some code for Warehouse + TUF... -- https://keybase.io/trishankdatadog

Hi Wes, Many members of the PyPA have been trying to obtain funding to try and work on this and other security features I believe. I don't believe, but could be wrong, anything has occurred in implementation here. But Ernest, Donald and Dustin would know best. Are you interested in helping? Cooper
participants (4)
-
Cooper Ry Lees
-
Sumana Harihareswara
-
Trishank Kuppusamy
-
Wes Turner