[Distutils] Builders vs Installers

[Vinay Sajip]

Donald Stufft <donald <at> stufft.io> writes:

> I disagree.
> 
> - These files are used for tools to exchange data, so "hand editable"
shouldn't be a primary concern.

Right. Nobody hand-edits PKG-INFO now, do they?

> - There are a number of current fields where the current format is *not*
enough and one off psuedo formats
> have had to be added
>   - `Keywords: dog puppy voting election` - A list masquerading as a string,
this one needs field.split() to
> actually parse it
>   - `Project-URL: Bug, Issue Tracker,
http://bitbucket.org/tarek/distribute/issues/` - A dictionary
> masquerading as a list of strings, this one needs {key.strip(): value.strip()
for key, value in
> [x.rsplit(", ", 1) for x in field]}
>   - Any of the fields can contain arbitrary content, previously Description
had specialized handling for
> this which it has now been moved to the payload section, but all the same
issues there affect other fields.
> - The Extension field name using ExtensionName/ActualKey to kludge a nested
dictionary
> - The ExtensionName/json is a horrible kludge why are we nesting a format
inside of a format instead of just
> using a format that supports everything we could want?
> 
> As far as I can tell the only things that even use PKG-INFO is
setuptools/distribute and we want to phase them
> out of existence anyways. The only other thing I can think of is Wheel which
can either a) be updated to a
> different format it's new enough there's not much need to worry about legacy
support or b) generate the
> METADATA file just for Wheels.

Please note that:

* I already have a system working fairly well *now* (though it's early days,
  and needs more testing) where JSON is used for metadata.
* The metadata covers not just the index metadata (PKG-INFO) but also metadata
  covering how to build, install and test distributions.
* The metadata already exists for the vast bulk of distributions on PyPI and
  is derived from the setup.py in those distributions. So migration is not
  much of an issue.
* The "distil" tool demonstrates each of the Archiver, Builder and Installer
  roles reasonably well for its stage of development.

Donald's above analysis resonates with me - it seems pretty kludgy trying to
shoe-horn stuff into key-value format which doesn't fit it well. There don't
seem to be any valid technical arguments to keeping the key-value format, other
than "please let's not try to change too many things at once". If that's really
going to be accepted as the reason, it strikes me as being a little timid
(given what "distil" shows is possible). And that would be enough of a shame as
it is, without making things worse by introducing something like
ExtensionName/json.

To those people who would balk at editing JSON by hand - who's asking you
to? Why not just get the data into an appropriate dict, using any tools you
like, and then serialise it to JSON? That approach seems to be what JSON was
designed for. If any tools need PKG-INFO style metadata, that's easy enough to
generate from a JSON format, as distil's wheel building support demonstrates.

Regards,

Vinay Sajip