[Distutils] Towards a simple and standard sdist format that isn't intertwined with distutils

[Marcus Smith]

>
> But the different builds for the different configurations end up with
> different metadata. If I'm understanding right, the whole point of
> "source wheels" is that they have all the static metadata that pip
> needs in order to make decisions, and this has to match the resulting
> wheels -- right?
>

I think we're largely talking about variances in "external" non-python
system dependencies (and their build settings).

PEP426 currently doesn't cover this in the core metadata, so as it stands,
any 2.0 sdist couldn't exhaust these build variances in it's core metadata.

There has been some discussion on how to represent external dependencies.
In brief, I think the going idea is that it would be through extensions (
https://www.python.org/dev/peps/pep-0426/#metadata-extensions), not in the
core python metadata, and the various groups (distro folks, science folks,
etc..) would implement these themselves to fulfill their needs...

Assuming they did implement such an extension, it would exist in the sdist,
and for cases like numpy likely support some notion of "build options", and
hence allow for a 1 to many mapping between sdist and binary wheels.

Marcus


> The way I'm imagining it is that there are multiple levels of metadata
> staticness:
>
> package name, author, description, ...
>   static in: VCS checkouts, source releases, source wheels, wheels
> package version
>   static in: source releases, source wheels, wheels
> package dependencies
>   static in: source wheels, wheels
> environment tag
>   static in: wheels
>
> > Of course, there *is* an unsolved issue here, which is how we manage
> > compatibility for wheels at the level needed for numpy. But I thought
> > the discussion on that was ongoing? I'm concerned that this proposal
> > is actually about bypassing that discussion, and instead trying to
> > treat incompatibly linked wheels as "different" in terms of project
> > metadata, which I think is the wrong way of handling things. I note
> > that Christoph Gohlke's numpy builds are tagged with a "+mkl" local
> > version modifier - that's presumably intended to mark the fact that
> > they are built with an incompatible runtime - but that's a misuse of
> > local versions (and I've found it causes niggling issues with how pip
> > recognises upgrades, etc).
>
> Yeah, that's not a good long term solution -- it needs to be moved
> into the metadata (probably by creating an MKL wheel and then making
> the numpy wheel depend on it). That's exactly the problem :-)
>
> > So, in summary: Your points above don't seem to me to in any way
> > preclude having a single numpy source wheel, and a number of (mutually
> > incompatible, but the same in terms of project and version) binary
> > wheels.
>
> Maybe I have misunderstood: does it actually help pip at all to have
> static access to name and version, but not to anything else? I've been
> assuming not, but I don't think anyone's pointed to any examples yet
> of the problems that pip is encountering due to the lack of static
> metadata -- would this actually be enough to solve them?
>
> -n
>
> --
> Nathaniel J. Smith -- http://vorpus.org
> _______________________________________________
> Distutils-SIG maillist  -  Distutils-SIG at python.org
> https://mail.python.org/mailman/listinfo/distutils-sig
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20151008/e413187a/attachment-0001.html>