[Distutils] Immutability of Release Metadata in Warehouse

[Donald Stufft]

> On Dec 19, 2017, at 4:30 PM, Matthias Bussonnier <bussonniermatthias at gmail.com> wrote:
> 
> One alternative is have the ability to "yank" a package. Make it still available, but installable only when pinned explicitly. I believe that's what Rust does. 

The ability to yank a package is something I’d love to introduce, unfortunately it will require changes to the API that tools like pip use, because PyPI does not currently have any idea why you’re asking for a list of the releases, just that you’re asking for one. We’d also want to do it in a way that doesn’t introduce unnecessary overhead in terms of HTTP requests or caching (for instance, previously pip would look up /simple/foo/2.0/ if you did ``pip install foo==2.0`` but we removed that because in 99% of cases it wasn’t going to be more helpful than just requesting /simple/foo/ and it made it harder to cache our pages I Fastly for everything).

One possible solution is to simply add a ``data-yanked=true`` field on /simple/foo/. That would mean that much like requires-python you’d need to have a new enough pip to support yanked packages properly but it wouldn’t destroy our cacheability like the old method did. This wouldn’t require any real changes to any of the mirroring tooling and static mirrors would still be totally feature complete.

Another possible option is to just exclude the yanked listing by default, but have something like ?yanked=true added to the URL which would then include *all* the yanked releases. This is “safe” to do in the case of ==, because adding extraneous yanked releases isn’t going to matter, since the == will only allow it to install one of them anyways. It bloats our cache a bit (two versions of the /simple/<foo>/ url for each project instead of 1) but it doesn’t completely destroy it like the /simple/foo/<version>/ scheme did. This option would mean that older versions of tools simply wouldn’t see the yanked versions ever, even with ==. It also would probably preclude the ability to have a fully static mirror, since you’d need something to handle the ?yanked=true handling (although we could maybe get around that by using something like /simple/_yanked/foo/ or something).

There are possibly other ideas to handle this that I’m not thinking of offhand, but I think fundamentally the question will come down to whether we want older versions of the tooling to see yanked packages by default or not.

Ultimately this is probably a big enough feature that it would deserve a PEP to flesh out the various options and to figure out which path we go down. Unfortunately I don’t currently have time to handle that, but would gladly participate in such a discussion if someone were to lead it.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20171219/b592cb81/attachment.html>