[Python-Dev] Edits to Metadata 1.2 to add extras (optional dependencies)

[Daniel Holth]

On Aug 28, 2012, at 1:20 AM, Nick Coghlan <ncoghlan at gmail.com> wrote:

> On Tue, Aug 28, 2012 at 6:29 AM, "Martin v. Löwis" <martin at v.loewis.de> wrote:
>> Am 27.08.12 16:56, schrieb Daniel Holth:
>>> Metadata 1.2 is nearly 8 years old and it's Accepted but not Final. Is
>>> it better to continue editing it, or create a new PEP for Metadata
>>> 1.3?
>> 
>> 
>> You can't add new fields to the format after the fact, unless the format had
>> provided for such additions (which it does not - there is
>> no mention of custom fields anywhere, and no elaboration on how
>> "unknown" fields should be processed).
>> 
>> So if you want to add new fields, you need to create a new version
>> of the metadata.
> 
> I agree with this point - the main reason the metadata PEP is still
> lingering at Accepted rather than Final is the tangled relationship
> between distutils and other projects that led to the complete
> distutils feature freeze. Until distutils2 makes it into the standard
> library as the packaging module, the standard library is going to be
> stuck at v1.1 of the metadata format.
> 
>> Prepare for a ten-year period of acceptance - so it
>> would be good to be sure that no further additions are desired within
>> the next ten years before seeking approval for the PEP.
> 
> However, this point I really don't agree with. The packaging ecosystem
> is currently evolving outside the standard library, but the
> standardisation process for the data interchange formats still falls
> under the authority of python-dev and the PEP process.
> 
> If there are things missing from v1.2 of the metadata spec, then
> define v1.3 to address those known problems. Don't overengineer it in
> an attempt to anticipate every possible need that might come in the
> next decade. Tools outside the standard library are then free to adopt
> the new standard, even while the stdlib itself continues to lag
> behind.
> 
> When the packaging module is finally added (hopefully 3.4, even if
> that means we have to temporarily cull the entire compiler
> subpackage), it will handle the most recent accepted version of the
> metadata format (as well as any previous versions). If more holes
> reveal themselves in the next 18 months, then it's OK if v1.4 is
> created when it becomes clear that it's necessary.
> 
> At the very least, something v1.3 should make explicit is that custom
> metadata should NOT be put into the .dist-info/METADATA (PEP 376
> location, PKG-INFO, in distutils terms) file. Instead, that data
> should be placed in a *separate* file in the .dist-info directory.
> Something that *may* be appropriate is a new field in METADATA that
> explicitly calls out such custom metadata files by naming the PyPI
> distribution that is the authority for the relevant format (e.g.
> "Custom-Metadata: wheel" to indicate that 'wheel' defined metadata is
> present)

Setuptools just uses path.exists() when it needs a particular file and will not bother parsing pkg-info at all if it can help it. The metadata edits for 1.2 fold some of those files into metadata.