[Distutils] Announcement: Pip 10 is coming, and will move all internal APIs

Alex Grönholm alex.gronholm at nextday.fi
Sat Oct 21 14:11:18 EDT 2017

Paul Moore kirjoitti 21.10.2017 klo 13:03:
> On 20 October 2017 at 23:53, Richard Jones <r1chardj0n3s at gmail.com> wrote:
>> Hiya Paul,
>> There's a bunch of tooling out there using pip's internals to extending
>> pip's functionality. Could you please provide a some reasoning as to why
>> they're all going to be broken at pip 10, and possibly some guidance on how
>> to get that functionality back?
> Hi Richard,
> There was a change to the pip docs that clarified the status of pip's
> internal code. The PR for that is at
> https://github.com/pypa/pip/pull/4743 but unfortunately it appears
> that the docs build has been failing so it hasn't yet made it to the
> formal pip docs site.
> To summarise, pip has *never* supported the use of its internal APIs
> by external code. Over time, we've had a steady trickle of people
> raising issues when their code broke because of doing so, and it
> usually turned out to be because they violated assumptions made by the
> pip code - such as that it's running in a single-threaded application,
> or it has sole control over the logging subsystem, or even just that
> you can run your own code after calling a pip API. We've explained
> this position regularly on those issues, but as is typical, people
> don't manage to find similar issues when raising new ones, so we spent
> a lot of time repeating ourselves.
> Coming up to pip 10, there's been a *lot* of internal work going on,
> and it's fairly likely that a decent chunk of code using pip's
> internal APIs will break anyway. We don't document internals changes,
> so we faced the possibility of an extended period of people raising
> issues saying "you broke my code" and us having no better response
> than "you shouldn't do that", which would likely hinder adoption of
> pip 10, and cause problems for the ecosystem as a whole. Rather than
> do this, we decided to make a clean compatibility break, where we
> could send out a clear message - "everything's moved, if that matters
> to you, then you were using unsupported functionality and you should
> find a better way". The breakage is still there (and certainly we made
> it affect more people, as there are no doubt some people who would
> have survived the pip 10 release unscathed if we hadn't done this) but
> at least it's clearly defined and contained.
> As to alternatives, we don't have all the answers here but I can offer
> the following suggestions:
> 1. There are a number of external packages that provide functionality
> equivalent to what pip does - packaging, wheel, distlib, pkg_resources
> are the ones I'm aware of. These are designed as libraries and so *do*
> provide supported APIs.
I need to correct you here: wheel does *NOT* have a public API!

It did previously have some documented functions but it was not really 
well thought out and we recently decided to remove all traces of API 
documentation so as to not cause any confusion about it.
> 2. We're making a strong push to standardise *all* of the external
> interfaces that pip uses, so you should be far more able to write your
> own code if that's necessary, without worrying that it'll work
> differently than pip does, or that things will suddenly change and
> break your code.
> 3. You can call pip as a subprocess - that's always been supported and
> will remain so. We've added some automation-friendly features there
> (such as json output format for "pip list") and we'd be happy to add
> more if people want to submit PRs.
> Likely the biggest problems will be for people who call into the pip
> resolver and build APIs, as I don't know of any alternatives out
> there. But they were *definitely* breaking anyway, as we've made major
> changes to that code (and will be making more).
> Also, I should note that we didn't take this decision lightly. We
> don't have any particular objection in principle to having a supported
> stable pip API, it's just that we don't have anything even close to
> the resources needed to define a supported API, maintain it with
> acceptable backward compatibility guarantees, and support users who
> will inevitably be using it in unexpected and creative ways (we don't
> even have the resources to support the limited use of pip's internals
> that we see today). Also, pip was never designed originally as a
> library, so we *would* have to design that API from scratch. As I
> alluded to above, the existing internals code makes some strong
> assumptions about how it's called - assumptions that would be
> unacceptable in library code, but are fine in an application's
> internal code.
> Paul
> PS People who want to, can of course hunt out the new equivalents of
> the code they were using, and just switch. It's not like we can stop
> them. But the new names make it clear that they shouldn't be doing
> this, so there's an obvious warning there.
> PPS Please disregard xoviat's response. This is something we've been
> considering for a while, and most definitely not a spur of the moment
> decision. It's unfortunate that he was the one most immediately
> affected by the change when we made it, but that was just bad timing
> (we didn't suddenly do this just because "someone complained").
> _______________________________________________
> Distutils-SIG maillist  -  Distutils-SIG at python.org
> https://mail.python.org/mailman/listinfo/distutils-sig

More information about the Distutils-SIG mailing list