[Distutils] [Python-Dev] distlib updated with resources API

[Donald Stufft]

I have some unpublished work I should publish. 

Part of the point with what i'm trying to do is to define a standard for what is inside
a package, but not really for how you take a particular set of files and turn it into
that. So if you want to edit yaml you can have a yaml file and have a package
creation tool that turns that yaml into the standard json that gets added to the
package. The same can be said for a python file or a setup.cfg or whatever. Ideally
the roles of package creation, building, and installation should be able to be completely
separate. So my goal is to facilitate that by creating a standard way to describe
all the data about a distribution, including extensible data in a way that any tool
can serialize or deserialize it losslessly.



On Saturday, September 29, 2012 at 5:54 AM, Vinay Sajip wrote:

> Nick Coghlan <ncoghlan <at> gmail.com (http://gmail.com)> writes:
> 
> 
> > > The document has changed since then,
> http://python-notes.boredomandlaziness.org/en/latest/pep_ideas/core_packaging_api.html
> 
> I read from your page there that Donald Stufft is working on a JSON-based
> metadata format. I've been looking that the same thing - a more flexible
> metadata format which directly maps to dicts - but I used YAML, as looking at
> too much JSON gives me eye-strain from all the extraneous quotes and braces.
> 
> I believe that JSON is the right format to use at the moment, because
> PyYAML still has some bugs which I've run into while doing this work (also,
> of course, it's not in the stdlib). As the formats are readily
> interchangeable, there might be interest here in looking at the package.yaml
> that I've come up with.
> 
> Since the metadata needs to support both the existing metadata and the
> additional things that e.g. setuptools supports via additional kwargs to
> setup(), I put together an ugly hack where I essentially mocked parts of
> distutils and setuptools, including the setup() call. This allows me to
> generate the YAML format automatically from most distributions on PyPI, using
> their setup.py.
> 
> Here's a GIST with sample package.yaml files automatically generated from PyPI
> downloads of SQLAlchemy 0.7.8, Jinja2 2.6, Flask 0.9 and wheel 0.9.4.
> 
> https://gist.github.com/3803556
> 
> The JSON format of the metadata is actually appended as a comment on the last
> line of the YAML metadata (I use that to report YAML bugs).
> 
> I've not yet documented the schema for the metadata, as I'm still thinking
> about the details.
> 
> I ran my hack on around 18,000 PyPI releases (basically, all the latest
> releases which are hosted on PyPI). For all but around 1300, I was
> able to generate package.yaml files. The ones which failed are those where
> no setup.py is present, or it's present, but can't be imported because it
> assumes that some third-party package is available.
> 
> Comments welcome.
> 
> Regards,
> 
> Vinay Sajip
> 
> _______________________________________________
> Distutils-SIG maillist - Distutils-SIG at python.org (mailto:Distutils-SIG at python.org)
> http://mail.python.org/mailman/listinfo/distutils-sig
> 
> 


-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20120930/89198bc9/attachment.html>