[Distutils] distlib and wheel metadata

[Nathaniel Smith]

On Feb 14, 2017 12:21, "Vinay Sajip" <vinay_sajip at yahoo.co.uk> wrote:



> I thought the current status was that it's called metadata.json
> exactly *because* it's not standardized, and you *shouldn't* look at
> it?


Well, it was work-in-progress-standardised according to PEP 426 (since
sometimes implementations have to work in parallel with working out the
details of specifications). Given that PEP 426 wasn't done and dusted
but being progressed, I would have thought it perfectly acceptable to
use "pydist.json", as the only things that would be affected would be
packaging tools working to the PEP.

> It's too bad that the JSON thing didn't work out, but I think we're
> better off working on better specifying the one source of truth
> everything already uses (METADATA) instead of bringing in *new*
> partially-incompatible-and-poorly-specified formats.

When you say "everything already uses", do you mean setuptools and wheel?
If nobody else is allowed to play, that's one thing. But otherwise, there
need to be standards for interoperability. The METADATA file, now - exactly
which standard does it follow? The one in the dateutil wheel that Jim
referred to doesn't appear to conform to any of the metadata PEPs. It was
rejected by old metadata code in distlib (which came of out the Python 3.3
era "packaging" package - not to be confused with Donald's of the same name
-
which is strict in its interpretation of those earlier PEPs).


That's why I said we need to fix the standards to bring them back in sync
with reality. I'm not arguing that there's no problem, I'm saying that
replacing one serialization format with another won't actually address the
problem, but does cause new complications.


The METADATA format (key-value) is not really flexible enough for certain
things which were in PEP 426 (e.g. dependency descriptions), and for these
JSON seems a reasonable fit. There's no technical reason why "the JSON thing
didn't work out", as far as I can see - it was just given up on for a more
incremental approach (which has got no new PEPs other than 440, AFAICT). I
understand that social reasons are often more important than technical
reasons
when it comes to success or failure of an approach; I'm just not sure that
in this case, it wasn't given up on too early.


The technical problem with PEP 426 is that unless you want to throw away
pypi and start over, all tools need to understand the old METADATA files
regardless. So it still needs to be specified, all the same code needs to
be kept around, etc. Plus the most pressing issues are like "what does the
field actually mean", which is totally independent of the serialization
format.

If there are particular fields that need more structured data, then there
are options: we could have fields in METADATA whose values are JSON, or a
sidecar file that supplements the main METADATA file with extra
information. But adding a new way to specify fields like Name and Version
really doesn't help anybody.

-n
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20170214/8f5309a8/attachment-0001.html>