On 11 October 2015 at 05:31, Nathaniel Smith <njs@pobox.com> wrote:
Do the relevant pip maintainers even read this mailing list? :-)
Donald and I are pip maintainers (not the only ones, but probably the ones who make the most noise on this list :-)). The comments I've made that you quoted are based mostly on how I recall pip works, it's unfortunately been some time since I looked into pip's dependency resolution and install code (it's a bit of a pit of snakes) and I haven't had time to check the details when commenting on this thread, so I don't claim my comments are authoritative. I think your description of the behaviour is right. As regards what pip could do, technically you are of course correct (it's possible, it just needs someone willing to make the code changes). I don't know, however, if the approach you're proposing fits with how we currently envisage pip developing. Specifically, my personal goal [1] is that we get to a point where pip can do all of the dependency resolution steps, getting to a point where it knows exactly what it will install, *before* it starts downloading packages, running build steps, etc. (And I don't see that sitting well with your proposal). The reason I want that is so that I would like to be able to see what pip plans on doing, and audit it before it begins (a "pip install --dry-run" type of step). Typically, this lets me avoid getting part-way through an install only to find that I need a project that doesn't have a wheel and I can't build from the sdist. That's a real problem I deal with a lot - at the moment I work with "pip list --outdated", check the listing to see what provides wheels, and update just those packages individually. But that's messy and manual, and not always 100% effective - and I'd like to be able to do better. I also don't think it's reasonable to focus solely on what *pip* requires. Just as important (IMO) is allowing PyPI to provide rich metadata. In particular, I write a lot of programs that query PyPI for package information. There are a lot of useful scripts I would like to be able to write that need to query PyPI for the dependency details of a package, and that simply isn't possible at the moment. Most of these are ad-hoc or personal-use scripts, and a lot of things I simply haven't done because I can't. So it's hard to quantify the benefits. But currently, the wonderful Metadata 2.0 ideal future promises me static dependency data on PyPI. That will only happen if we get enough people interested in implementing it, but I live in hope. If I understand your proposals, they are closing off that option, and explicitly stating that we'll never have dependency metadata on PyPI for projects that don't publish wheels. Maybe you're right, and it'll never be possible. But I don't want to give up that dream without being sure it's necessary - which is why I push back so hard on the idea. Paul [1] Which is not backed up by me having the time to write the code, so the only relevance here is that I'm inclined to support proposals that work towards that goal.