On unknown names, current pkg_resources raises SyntaxError. So I think we need to update the spec from that perspective to be clear. All PEP 426 defined variables are handled without error by current pkg_resources/markerlib implementation. Perhaps the new variables should raise rather than evaluating to '' / 0 ? Some discussion / thought would be good. Certainly when used and evaluated by an existing pkg_resources they will error - so perhaps we should not add any new variables at this point?
So, if we don't unify this with the wheel encoding of extras, it will require multiple parsers indefinitely. I need to think more about whether it makes sense or not.
Wheel certainly needs a way to say 'this distribution exports extras X, Y, Z (and their respective dependencies)'. flit and other tools producing wheels need the same facility.
https://www.python.org/dev/peps/pep-0427 doesn't define this use of markers; but pip and wheel have collaborated on it. PEP-345 doesn't describe Provides-Extra, which pkg_resources uses when parsing .dist-info directories as well (it determines which extra variables get substituted into the set of requires to determine the values of the extras...). So there's basically still a bunch of underspecified behaviours out there in the wild, and since my strategy is to minimal variation vs whats there, we need to figure out the right places to split things to layer this well.
Specifying a new variable of 'extra' is fairly easy: we need to decide on the values it will take, and thats well defined but layer crossing: when processing a dependency with one or more extras, you need to loop over all the dependency specifications once with each extra defined (including I suspect for completeness '' for the non-extras) and then union together the results.
So at this layer I think we could say that:
This allows a single implementation to handle .dist-info directories as it does today, while specifying it more fully. It leaves it open in future for us to continue modelling exported extras as marker-filtered-specifications + a Provides-Extra, or to move to exported extras as something in a hash in a richer serialisation format, or some third thing. This is good I think.
I do like the idea of comments and line continuations being removed. We can then explicitly say that this DSL is going to be embedded in a larger context such as requirements.txt files, requires headers etc, and that those contexts may provide multi line handling as desired.
I'll apply Nathaniel's excellent review details + this on Monday and issue an update.
On 8 November 2015 at 05:08, Marcus Smith email@example.com wrote:
I'm not sure this is the syntax that I would have come up with, but I guess it's too late to put the genie back in the bottle, so this PEP should have some way to cope with these things?
why would this PEP deal with this? the higher level PEP that builds on top of this would bump the wheel metadata version (probably to 3.0, due to the jump to 2.0 already) then tools will cope with this based on the metadata version
-- Robert Collins firstname.lastname@example.org Distinguished Technologist HP Converged Cloud