[Distutils] New PEP : dependency specification

[Robert Collins]

On 7 November 2015 at 18:03, Nathaniel Smith <njs at pobox.com> wrote:

> Thanks, this is really great!
>
> I made a bunch of fiddly comments inline on the PR, but some more
> general comments:
>
> 1) Also mentioned this in the PR, but it's probably worth putting up
> for more general discussion here: do we want to define some graceful
> degradation for how to handle unrecognized variable names in the
> environment marker syntax, so as to allow more easily for future
> extensions? In the current PEP draft, an unrecognized variable name is
> simply a syntax error (all the variable names are effectively
> keywords).

I don't think we can, because all the implementations today error. My
goal is to document what is, rather than what might be, except where
variation will be safe or gated behind other checks (such as urls -
they are not legal in PyPI even in PEP-440, so including them as a
desired feature with the PEP-440 syntax does not impede adoption).

> 2) The PEP seems a little uncertain about whether it wants to talk
> about the "framing protocol" or not -- like whether there's a
> higher-level structure defining the edges of each individual
> requirement or not. In setup.py and in existing METADATA files, you
> have some higher level list-of-strings syntax and then each string is
> parsed as a single individual requirement, and comments don't make
> much sense; in requirements.txt then newlines are meaningful and
> comments are important. The PEP worries about newlines and newlines,
> but doesn't quite want to come out and say that it's defining
> requirements.txt -- it wants to be more general.

Being lower level was an explicit goal. I've dropped comments and
newline continuations and said embedding it should include such
features if desired.

> 3) The way extras are specified in METADATA files currently (ab)uses
> the environment marker syntax. E.g. here's the METADATA files from two
> popular packages:
>
>   https://gist.github.com/njsmith/ed74851c0311e858f0f7
>
> (Nice to see Metadata-Version: 2.0 getting some real-world use! I
> guess??? I thought I was starting to understand what is going on with
> python packaging standards but now I am baffled again. Anyway!)
>
> So apparently this is how you say that there is an extra called "doc"
> and that this extra adds a dependency on Sphinx:
>
> Provides-Extra: doc
> Requires-Dist: Sphinx (>=1.1); extra == 'doc'
>
> (https://gist.github.com/njsmith/ed74851c0311e858f0f7#file-ipython-4-0-0-wheel-metadata-L39)
>
> And here's how you say that the "terminal" extra adds a dependency on
> pyreadline (but only on windows):
>
> Requires-Dist: pyreadline (>=2); sys_platform == "win32" and extra == 'terminal'
>
> (https://gist.github.com/njsmith/ed74851c0311e858f0f7#file-ipython-4-0-0-wheel-metadata-L64)
>
> I'm not sure this is the syntax that I would have come up with, but I
> guess it's too late to put the genie back in the bottle, so this PEP
> should have some way to cope with these things? Currently they are
> simply syntax errors according to the PEP's grammar.

As we discussed, I've made the specification support this, but limited
to specific contexts such as wheel, not present-by-default.

This happens to make markerlib instant-buggy (because it defaults
extra to None, but on Python3 it will blow up anyhow in various
situations because e.g. "foo" < None :)

-Rob

-- 
Robert Collins <rbtcollins at hp.com>
Distinguished Technologist
HP Converged Cloud