[Python-Dev] Fwd: PEP 426 is now the draft spec for distribution metadata 2.0

[Nick Coghlan]

On Tue, Feb 19, 2013 at 11:23 PM, M.-A. Lemburg <mal at egenix.com> wrote:
> On 19.02.2013 11:28, Nick Coghlan wrote:
>> On Tue, Feb 19, 2013 at 7:37 PM, M.-A. Lemburg <mal at egenix.com> wrote:
>>> On 17.02.2013 11:11, Nick Coghlan wrote:
>>> I'm not against modernizing the format, but given that version 1.2
>>> has been out for around 8 years now, without much following,
>>> I think we need to make the implementation bit a requirement
>>> before accepting the PEP.
>>
>> It is being implemented in distlib, and the (short!) appendix to the
>> PEP itself shows how to read the format using the standard library's
>> email module.
>
> Hmm, what is distlib and where does it live ?

As part of the post-mortem of packaging's removal from Python 3.3,
several subcomponents were identified as stable and useful. distlib is
those subcomponents extracted into a separate repository by Vinay
Sajip.

It will be proposed as the standard library infrastructure for
building packaging related tools, while distutils will become purely a
build system and have nothing to do with installing software directly
(except perhaps on developer machines).

> The PEP only shows how to parse the RFC822-style format used by the
> metadata. That's not what I was referring to.
>
> If a tools wants to support metadata 2.0, it has to support all
> the complicated stuff as well, i.e. handle the requires fields,
> the environment markers and version comparisons/sorting.

Which is what distutils2 can be used for now, and what distlib will
provide without the unwanted build system infrastructure in
distutils2.

>> v2.0 is designed to fix many of the issues that prevented the adoption
>> of v1.2, including tweaks to the standardised version scheme and the
>> addition of a formal extension mechanism to avoid the ad hoc
>> extensions that occurred with earlier metadata versions.
>
> Some more notes:
>
> * I find it confusing that we now have two version schemes,
>   one defined in PEP 426 (hidden in the middle of the document)
>   and one in PEP 386. It would be better to amend or replace
>   PEP 386, since that's where you look for Python version strings.

You can't understand version specifiers without understanding the sort
order defined for the version scheme, so documenting them separately
is just a recipe for confusion.

I plan to mark PEP 386 as Withdrawn when I accept this PEP, as the
sorting scheme it defines is broken, and the distutils changes
proposed in that PEP are never going to happen.

> * PEP 426 doesn't include any mention of the egg distribution format,
>   even though it's the most popular distribution format at the moment.
>   It should at least include the location of the metadata file
>   in eggs (EGG-INFO/PKG-INFO) and egg installations
>   (<eggdir>/EGG-INFO/PKG-INFO).

"Other tools involved in Python distribution may also use this format."

The egg format has never been, and never will be, officially endorsed
by python-dev. The wheel format is the standard format for binary
distribution, and PEP 376 defines the standard location for metadata
on installed distributions.

>
> Not sure whether related or not, I also think it would be a good idea
> to make the metadata file available on PyPI for download (could be sent
> there when registering the package release). The register command
> only posts the data as 1.0 metadata, but includes fields from
> metadata 1.1. PyPI itself only displays part of the data.

It's not related, but I plan to propose the adoption of TUF (with GPG
based signatures) for PyPI's end-to-end security solution, and the
conversion of the metadata files to JSON for distribution through
TUF's metadata support. (Donald Stufft already wrote  PEP 426 <-> JSON
bidirectional converter as part of an unrelated experiment)

> It would be useful to have the metadata readily available for
> inspection on PyPI without having to download one of the
> distribution files first.

Indeed, which is a large part of why TUF (aka The Update Framework:
https://www.updateframework.com/) is such an interesting security
solution.

Regards,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia