On Sat, Nov 7, 2015 at 12:32 PM, Robert Collins robertc@robertcollins.net wrote:
On unknown names, current pkg_resources raises SyntaxError. So I think we need to update the spec from that perspective to be clear. All PEP 426 defined variables are handled without error by current pkg_resources/markerlib implementation. Perhaps the new variables should raise rather than evaluating to '' / 0 ? Some discussion / thought would be good. Certainly when used and evaluated by an existing pkg_resources they will error - so perhaps we should not add any new variables at this point?
I just thought of one possible strategy for allowing future extensions without opening the door wide to every typo: add a variable which contains the version number of the environment marker evaluation system, define the and/or operators as being short-circuiting, and keep the rule that trying to access an unknown variable name raises an error. Then you could use write things like:
# 'foo is only really needed if some_new_attr > "2", but doesn't hurt otherwise.
# So when using an old installation tool that doesn't know about some_new_attr,
# install it unconditionally.
Requires-Dist: foo; marker_handling >= "2" and some_new_attr > 2 Requires-Dist: foo; marker_handling < "2"
So, if we don't unify this with the wheel encoding of extras, it will require multiple parsers indefinitely. I need to think more about whether it makes sense or not.
Wheel certainly needs a way to say 'this distribution exports extras X, Y, Z (and their respective dependencies)'. flit and other tools producing wheels need the same facility.
https://www.python.org/dev/peps/pep-0427 doesn't define this use of markers; but pip and wheel have collaborated on it. PEP-345 doesn't describe Provides-Extra, which pkg_resources uses when parsing .dist-info directories as well (it determines which extra variables get substituted into the set of requires to determine the values of the extras...). So there's basically still a bunch of underspecified behaviours out there in the wild, and since my strategy is to minimal variation vs whats there, we need to figure out the right places to split things to layer this well.
Specifying a new variable of 'extra' is fairly easy: we need to decide on the values it will take, and thats well defined but layer crossing: when processing a dependency with one or more extras, you need to loop over all the dependency specifications once with each extra defined (including I suspect for completeness '' for the non-extras) and then union together the results.
So at this layer I think we could say that:
- extra is only valid if the context that is interpreting the specification defines it
- when invalid it will raise SyntaxError
This allows a single implementation to handle .dist-info directories as it does today, while specifying it more fully. It leaves it open in future for us to continue modelling exported extras as marker-filtered-specifications + a Provides-Extra, or to move to exported extras as something in a hash in a richer serialisation format, or some third thing. This is good I think.
This seems like a good strategy to me.
-n
-- Nathaniel J. Smith -- http://vorpus.org