[Distutils] Environment marker expression types

[Nick Coghlan]

On Fri, Apr 26, 2013 at 4:05 PM, PJ Eby <pje at telecommunity.com> wrote:
> On Thu, Apr 25, 2013 at 9:50 PM, Nick Coghlan <ncoghlan at gmail.com> wrote:
>> 5. The installer evaluates the conditional metadata as appropriate
>> before writing the metadata out to the installation database and
>> invoking the post-install hook. Post-installation, the top level
>> metadata includes both the unconditional metadata *and* any
>> conditional metadata that applies to the current system.
>
> ISTM that this does away with some of the benefits of e.g. fat wheels
> and the ability to share source across multiple Python versions (as
> distros sometimes like to do).  It also seems to have very little
> gain, since if you are reading the metadata, you've probably already
> imported distlib anyway.  ;-)

OK, I'm convinced - no install time transforms, if you need the full
set of dependencies at run time, you have to poke around in the
conditionals field to see if there is anything relevant.

> In general, I favor the minimum possible transformations between
> wheels and on-disk format, simply because it means fewer conditional
> paths in APIs that need to support both.  If you're going to resolve
> the dependencies in a wheel, you have to parse the conditionals, so
> again it doesn't save much to not parse them.
>
> If you want to increase parsing efficiency in the presence of
> conditionals, I would suggest nesting the conditionals in individual
> fields, or at least using a structure like:
>
>     { 'conditionals': {'requires': [....], 'other-field': [...]}}
>
> As this means that you only have to parse the conditions for fields
> you're actually accessing.  That keeps parsing to a minimum if a given
> query is only looking at unconditional fields.

Yes, I think I prefer this layout for runtime consumption. Getting the
full value for a field would then just be something like:

    # Core dependencies
    deps = meta["requires"]
    # Check dependencies for any specified optional components
    for extra_name in extras_of_interest:
        extra = meta["extras"].get(extra_name)
        if not extra:
            continue
        deps.extend(meta["extras"].get("requires", ()))
    # Check conditional dependencies
    maybe_deps = meta["conditional"].get("requires")
    if maybe_deps:
        for dep in maybe_deps:
            if some_marker_evaluation_api(dep["condition"]):
                deps.extend(dep["entry"])

Good enough for a first draft, anyway :)

distlib would probably evolve some metadata abstraction that hides all
that fiddling behind a property.

> So, my suggestion would be to drop the overloading of extras as
> markers, and structure all conditional data such that only the
> conditionals needed to satisfy a particular inquiry ever need to be
> parsed.  This should optimize the common case (infrequent dependency
> parsing and infrequent conditionals in a simple unpacked structure)
> while still enabling advanced cases such as cross-version or
> cross-platform sharing to work correctly.

Yes, I'm definitely sold on that part.

Cheers,
Nick.

--
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia