On Tue, 25 Sep 2018 at 12:53, Chris Jerdonek
On Tue, Sep 25, 2018 at 3:47 AM, Tzu-ping Chung
wrote: We are using pip internals for things pip wasn’t implemented for. Specifically, Pipenv uses pip’s package-fetching functions to implement its platform-agnostic resolver. pip does not have this, so there’s no functional overlap here. Those utilities are used to build something that doesn’t exist in pip, so there’s no duplicated efforts.
My recent focus on making sense of packaging implementations and splitting out parts of pip is exactly to prevent potential duplicated efforts. If we can’t use pip internals, let’s make things we want to use *not* internal!
I don't see this practice of "splitting out parts of pip" actually being followed so far though, and I want to tell you how it's leading to duplicated efforts right now -- even for the PR's I happen to be working on today.
[...]
So by creating implementations of functions separate from pip instead of splitting them out of pip, it's leading to duplicated work for people working on pip's code base. A process of splitting something out of pip would I think involve a process more like what I'm following -- namely to refactor functionality in pip *first* so it can be used separately, and then pull that out so you only have one implementation at any point in time. Otherwise, you wind up creating two versions of similar functionality that not only need to be created twice, but also later reconciled if the plan is merge them back together.
This is a very good point, and thanks for the concrete example. For the record, I'm also a little annoyed that pipenv have been doing all this work in isolation. There's been very little communication between the pip and pipenv projects, and I think that's contributed to the problem. I, for one, wasn't even aware for a long time that they were embedding their own copy of pip. When we made pip's internals in private in pip 10, we got essentially no feedback from anyone saying that it would be a problem, and finding out that it had a sufficiently major impact on pipenv that they had to embed the old version of pip *after* the release was (to put it politely) suboptimal :-( Having said all this, this thread is the start of an attempt to work better together and I think we should be glad that it's happening and try to make it work. But I don't think we've got off to a particularly good start and it will need work. In particular, I'd like to see an attempt from *both* projects to clarify our goals - I've been speaking solely for myself so far, it would be good to get the other pip developers' views and set out a sort of roadmap, and for pipenv to do the same. Paul