I have some unpublished work I should publish.

Part of the point with what i'm trying to do is to define a standard for what is inside
a package, but not really for how you take a particular set of files and turn it into
that. So if you want to edit yaml you can have a yaml file and have a package
creation tool that turns that yaml into the standard json that gets added to the
package. The same can be said for a python file or a setup.cfg or whatever. Ideally
the roles of package creation, building, and installation should be able to be completely
separate. So my goal is to facilitate that by creating a standard way to describe
all the data about a distribution, including extensible data in a way that any tool
can serialize or deserialize it losslessly.


On Saturday, September 29, 2012 at 5:54 AM, Vinay Sajip wrote:

Nick Coghlan <ncoghlan <at> gmail.com> writes:


The document has changed since then,
http://python-notes.boredomandlaziness.org/en/latest/pep_ideas/core_packaging_api.html

I read from your page there that Donald Stufft is working on a JSON-based
metadata format. I've been looking that the same thing - a more flexible
metadata format which directly maps to dicts - but I used YAML, as looking at
too much JSON gives me eye-strain from all the extraneous quotes and braces.

I believe that JSON is the right format to use at the moment, because
PyYAML still has some bugs which I've run into while doing this work (also,
of course, it's not in the stdlib). As the formats are readily
interchangeable, there might be interest here in looking at the package.yaml
that I've come up with.

Since the metadata needs to support both the existing metadata and the
additional things that e.g. setuptools supports via additional kwargs to
setup(), I put together an ugly hack where I essentially mocked parts of
distutils and setuptools, including the setup() call. This allows me to
generate the YAML format automatically from most distributions on PyPI, using
their setup.py.

Here's a GIST with sample package.yaml files automatically generated from PyPI
downloads of SQLAlchemy 0.7.8, Jinja2 2.6, Flask 0.9 and wheel 0.9.4.

https://gist.github.com/3803556

The JSON format of the metadata is actually appended as a comment on the last
line of the YAML metadata (I use that to report YAML bugs).

I've not yet documented the schema for the metadata, as I'm still thinking
about the details.

I ran my hack on around 18,000 PyPI releases (basically, all the latest
releases which are hosted on PyPI). For all but around 1300, I was
able to generate package.yaml files. The ones which failed are those where
no setup.py is present, or it's present, but can't be imported because it
assumes that some third-party package is available.

Comments welcome.

Regards,

Vinay Sajip

_______________________________________________
Distutils-SIG maillist - Distutils-SIG@python.org
http://mail.python.org/mailman/listinfo/distutils-sig