Improved dependency resolution in distlib
Following some changes to distlib, there have been improvements in how it can resolve dependencies. I ran a test on all source archives reachable through PyPI, and initial results seem to show that out of over 25,000 projects and 112,000 source archives, we can extract dependency metadata from all but 2,540 archives relating to 916 projects. The distlib locator code has been updated to access this metadata, and this makes it possible to resolve dependency graphs reasonably quickly, without downloading any archives. The dependency finder command-line script is at https://gist.github.com/3919026 The script, finddeps.py, when given a distribution name, will attempt to locate it and its dependencies, identify requirements which couldn't be satisfied, show a topological sort (advisory, given the existence of circular dependencies on many PyPI projects) and a download ordering. Example results are given in the same Gist for: Flask (3 dists in all, resolved in ~1 second) apycotbot (22 dists in all, resolved in ~5 seconds) collective.megaphone (242 dists in all, resolved in ~55 seconds) You can also specify constraints, e.g. $ python finddeps.py "pyramid (> 1.0, < 1.3)" I also included a separate script for testing the case described in https://github.com/pypa/pip/issues/174 as mentioned by Carl. In this case, we get the expected result, with the dependency tree looking like this: A 1.0 B 1.0 [B] D 0.9 [D (<= 0.9)] C 1.0 [C] D 0.9 [D (<= 1.1)] D 0.9 C 1.0 D 0.9 [D (<= 1.1)] B 1.0 D 0.9 [D (<= 0.9)] If anyone has the time to try out distlib, I'd appreciate some feedback on the dependency finder (or anything else for that matter) and any unexpected results you get in your trials. You should be able to run the dependency finder on most projects on PyPI, but not ones which don't have a reachable download (for example, "Goose". But there, "pip install Goose" fails, too). Thanks and regards, Vinay Sajip
participants (1)
-
Vinay Sajip