[Distutils] distlib and wheel metadata

[Daniel Holth]

Wheel puts everything important in METADATA, except entry_points.txt. The
requirements expressed there under 'Requires-Dist' are reliable, and the
full METADATA format is documented in the pre-JSON revision of PEP 426. At
runtime, once pkg_resources parses it, *.egg-info and *.dist-info look
identical, because it's just a different way to represent the same data.
Wheel's version of METADATA exists as the simplest way to add the critical
'extras' feature to distutils2-era *.dist-info/METADATA, necessary to
losslessly represent setuptools packages in a more PEP-standard way. I
could have completely redesigned the METADATA format instead of extending
it, but then I would have run out of time and wheel would not exist.

This function converts egg-info metadata to METADATA
https://bitbucket.org/pypa/wheel/src/54ddbcc9cec25e1f4d111a142b8bfaa163130a61/wheel/metadata.py?at=default&fileviewer=file-view-default#metadata.py-240

This one converts to the JSON format. It looks like it might work with
PKG-INFO or METADATA.
https://bitbucket.org/pypa/wheel/src/54ddbcc9cec25e1f4d111a142b8bfaa163130a61/wheel/metadata.py?at=default&fileviewer=file-view-default#metadata.py-98


On Wed, Feb 15, 2017 at 8:27 AM Nick Coghlan <ncoghlan at gmail.com> wrote:

> On 15 February 2017 at 12:58, Nathaniel Smith <njs at pobox.com> wrote:
> > On Wed, Feb 15, 2017 at 3:33 AM, Nick Coghlan <ncoghlan at gmail.com>
> wrote:
> >> - "requires": list where entries are either a string containing a PEP
> >> 508 dependency specifier or else a hash map contain a "requires" key
> >> plus "extra" or "environment" fields as qualifiers
> >> - "integrates": replacement for "meta_requires" that only allows
> >> pinned dependencies (i.e. hash maps with "name" & "version" fields, or
> >> direct URL references, rather than a general PEP 508 specifier as a
> >> string)
> >
> > What's accomplished by separating these? I really think we should
> > strive to have fewer more orthogonal concepts whenever possible...
>
> It's mainly a matter of incorporating
> https://caremad.io/posts/2013/07/setup-vs-requirement/ into the core
> data model, as this distinction between abstract development
> dependencies and concrete deployment dependencies is incredibly
> important for any scenario that involves
> publisher-redistributor-consumer chains, but is entirely non-obvious
> to folks that are only familiar with the publisher-consumer case that
> comes up during development-for-personal-and-open-source-use.
>
> One particular area where this is problematic is in the widespread
> advice "always pin your dependencies" which is usually presented
> without the all important "for application or service deployment"
> qualifier. As a first approximation:
> pinning-for-app-or-service-deployment == good,
> pinning-for-local-testing == good,
> pinning-for-library-or-framework-publication-to-PyPI == bad.
>
> pipenv borrows the Ruby solution to modeling this by having Pipfile
> for abstract dependency declarations and Pipfile.lock for concrete
> integration testing ones, so the idea here is to propagate that model
> to pydist.json by separating the "requires" field with abstract
> development dependencies from the "integrates" field with concrete
> deployment dependencies.
>
> In the vast majority of publication-to-PyPi cases people won't need
> the "integrates" field, since what they're publishing on PyPI will
> just be their abstract dependencies, and any warning against using
> "==" will recommend using "~=" or ">=" instead. But there *are*
> legitimate uses of pinning-for-publication (like the PyObjC
> metapackage bundling all its subcomponents, or when building for
> private deployment infastructure), so there needs to be a way to
> represent "Yes, I'm pinning this dependency for publication, and I'm
> aware of the significance of doing so"
>
> Cheers,
> Nick.
>
> --
> Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
> _______________________________________________
> Distutils-SIG maillist  -  Distutils-SIG at python.org
> https://mail.python.org/mailman/listinfo/distutils-sig
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20170215/83d6602a/attachment-0001.html>