distlib updated with "locators" API
I've updated distlib with an experimental API for locating distributions. A locator is just something whose "get_project" method you invoke with a "project" name such as "flask" or "sqlalchemy", and it returns a dictionary whose keys are versions of the project which were located, and the corresponding value is a tuple whose first element is a Metadata object and whose second element is a list of dicts pointing to archives for that version of the project. I've implemented three locators with some basic tests and docs: * DirectoryLocator - looks in a file system directory tree for archives * PyPIRPCLocator - uses PyPI's XML-RPC interface to locate archives * SimpleScrapingLocator - uses PyPI's "simple" web scraping interface to locate archives - similar to pip's PackageFinder A very basic script showing how to use it is at https://gist.github.com/3886402 There's plenty more work to do - if there's mileage in this idea, instead of returning Metadata and lists of dicts, Distribution instances can be returned which can be queried for dependencies in a more natural way. At the moment, this (dependency querying) isn't fully doable because all the metadata is not declarative. However, the work I've been doing with the package.yaml metadata format indicates that setup()-kwarg metadata can be automatically pulled out of setup.py for many (most) of the projects on PyPI. Feedback welcome. Regards, Vinay Sajip
On Sat, Oct 13, 2012 at 8:16 PM, Vinay Sajip
There's plenty more work to do - if there's mileage in this idea, instead of returning Metadata and lists of dicts, Distribution instances can be returned which can be queried for dependencies in a more natural way. At the moment, this (dependency querying) isn't fully doable because all the metadata is not declarative. However, the work I've been doing with the package.yaml metadata format indicates that setup()-kwarg metadata can be automatically pulled out of setup.py for many (most) of the projects on PyPI.
When you are using setuptools, setup.py can be used to generate declarative metadata with the command "python setup.py egg_info". You can add --egg-base <dir> to put the static metadata in a different directory. Setuptools even makes it easy to add new metadata files to the .egg-info folder by using a plugin to write each file. The majority of the sdists on pypi contain pre-built versions of this static metadata in .egg-info folders. Unfortunately setup.py is used to build not one, but many different dists (the most common difference of course being different requirements) depending on any number of factors, so pip rebuilds the static metadata before using it. Metadata 1.2+ of course allow us to put the most common "reasons for different dependencies" conditions in the static metadata itself. Wheel reads requires-dist lines from setup.cfg's [metadata] section so you can actually include Metadata 1.2+ style conditional dependencies. Unfortunately this feature makes it very easy to write sdists that install different dependencies if they are installed from a wheel archive versus straight from the sdist.
participants (2)
-
Daniel Holth
-
Vinay Sajip