I have some unpublished work I should publish. Part of the point with what i'm trying to do is to define a standard for what is inside a package, but not really for how you take a particular set of files and turn it into that. So if you want to edit yaml you can have a yaml file and have a package creation tool that turns that yaml into the standard json that gets added to the package. The same can be said for a python file or a setup.cfg or whatever. Ideally the roles of package creation, building, and installation should be able to be completely separate. So my goal is to facilitate that by creating a standard way to describe all the data about a distribution, including extensible data in a way that any tool can serialize or deserialize it losslessly. On Saturday, September 29, 2012 at 5:54 AM, Vinay Sajip wrote:
Nick Coghlan
http://gmail.com)> writes: The document has changed since then, http://python-notes.boredomandlaziness.org/en/latest/pep_ideas/core_packagin...
I read from your page there that Donald Stufft is working on a JSON-based metadata format. I've been looking that the same thing - a more flexible metadata format which directly maps to dicts - but I used YAML, as looking at too much JSON gives me eye-strain from all the extraneous quotes and braces.
I believe that JSON is the right format to use at the moment, because PyYAML still has some bugs which I've run into while doing this work (also, of course, it's not in the stdlib). As the formats are readily interchangeable, there might be interest here in looking at the package.yaml that I've come up with.
Since the metadata needs to support both the existing metadata and the additional things that e.g. setuptools supports via additional kwargs to setup(), I put together an ugly hack where I essentially mocked parts of distutils and setuptools, including the setup() call. This allows me to generate the YAML format automatically from most distributions on PyPI, using their setup.py.
Here's a GIST with sample package.yaml files automatically generated from PyPI downloads of SQLAlchemy 0.7.8, Jinja2 2.6, Flask 0.9 and wheel 0.9.4.
https://gist.github.com/3803556
The JSON format of the metadata is actually appended as a comment on the last line of the YAML metadata (I use that to report YAML bugs).
I've not yet documented the schema for the metadata, as I'm still thinking about the details.
I ran my hack on around 18,000 PyPI releases (basically, all the latest releases which are hosted on PyPI). For all but around 1300, I was able to generate package.yaml files. The ones which failed are those where no setup.py is present, or it's present, but can't be imported because it assumes that some third-party package is available.
Comments welcome.
Regards,
Vinay Sajip
_______________________________________________ Distutils-SIG maillist - Distutils-SIG@python.org (mailto:Distutils-SIG@python.org) http://mail.python.org/mailman/listinfo/distutils-sig