[Distutils] Updated drafts of metadata 2.0 (PEP 426 and PEP 440)

[Nick Coghlan]

New versions of PEP 426 and PEP 440 are up on python.org:

PEP 426 (metadata 2.0): http://www.python.org/dev/peps/pep-0426/
PEP 440 (version spec): http://www.python.org/dev/peps/pep-0440/

(as before, not including them inline due to sheer length)

The bulk of the changes are in this commit:
http://hg.python.org/peps/rev/de305af601fa
(there are a few minor tweaks in subsequent commits to the PEPs repo)

There have been several significant changes to the main metadata spec
in PEP 426:

* Added a basic "jsonschema" based description of the format
* The "abbreviated metadata" notion is gone, replaced by "included
documents". The metadata can now list the names of additional files
included in the distinfo directory for the long description, the
license and the change history. The contents are no longer embedded
directly in the metadata (but will be extracted and made available by
PyPI, with the markup format being determined from the file extension,
just as it is by sites like GitHub)
* The dependency system has been redesigned under the name "Semantic
dependencies" (as they now allow a distribution to better say *why* a
dependency is needed, rather than just saying "I need this" with
almost no capacity to say why). There are now five kinds of
dependencies (:meta:, :run:, :test:, :build: and :dev:) and they're
integrated into the extras system so you can easily say you want to
install some, none or all of them.
* The "*" notation is added to extras to make it easier to say
"install all dependencies", along with the "-extra_name" notation to
exclude the dependencies for specific extras
* The "-" notation is added to extras to make it easier to install
*just* a distribution's dependencies, without installing the
distribution itself.
* "Install hooks" are now a distinct concept from the
still-hypothetical "metabuild" system, and place more constraints on
their expected handling (installation tools are also explicitly
permitted to refuse to run them, but doing so is required to fail
noisily rather than silently appearing to succeed)

The most significant change to PEP 440 is to the handling of
pre-releases: whether or not pre-releases should be accepted by
default is now determined solely by whether or not there is a stable
release that *also* satisfies the version specifier. Reference a
pre-release (or not) now has no effect on whether pre-releases are
considered viable candidates. Pre-releases are now accepted if:
* they're already installed
* there's no other available option
* the installation tool is told specifically to consider them

Other less significant changes to PEP 426 include:

* Longer introduction giving more context for the nature and purpose
of the metadata
* Separated various other things out into appendices
* Various tweaks to definitions (including the "Release" tweak from
PEP 440, and switching "source archive" to refer to the original raw
source, while using "sdist" for the Python specific format with the
extra metadata)
* Blanket permission for distribution related online services to
impose additional restrictions to protect the integrity of the service
(such as additional length limits beyond those stated in the PEP).
* Explicitly require UTF-8 encoded JSON for serialisation
* build_label renamed to source_label
* version_url renamed to source_url
* tightened up the validation rules for various fields (including RFC
references where appropriate). Many of these "new" rules are things
projects already comply with (because not complying doesn't work
properly). Including them in the spec is about giving PyPI clear
guidance to enforce them at point of upload, rather than leaving it to
installation tools to try to sort out later.
* a few more additions to the "Rejected Features" appendix (notably,
my rationale for *not* requiring the install hooks to accept arbitrary
keyword arguments)

The other PEP 440 changes are also relatively minor:

- what were previously called releases are now "final releases",
freeing up "release" as a general term for any kind of release
(developmental release, pre-release, final release, post release).
- "source references" are now "direct references" and can also refer
to prebuilt wheel files
- automated tools (especially index servers) are given a lot of
freedom to be opinionated about the appropriate uses for direct
references
- a few tweaks to the security guidelines for direct references
- pytz/Olson database version translation is explicitly discussed (add
a leading 0., convert the trailing letter to an incrementing number)
- tighter rules for what characters are allowed in a source label

The only major remaining open item is the addition of guidelines in
Appendix A for converting legacy metadata to metadata 2.0. I consider
the rest of the spec stable at this point, unless anyone picks up on a
glaring hole in the latest draft that Daniel, Donald and I missed.

Cheers,
Nick.

--
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia