[Distutils] [Python-ideas] Add processor generation to wheel metadata

Wes Turner wes.turner at gmail.com
Tue Oct 31 12:23:40 EDT 2017

Maybe the anaconda team has some insight on a standard way to capture (&
configure) compiler versions and flags in package metadata?


> The Anaconda 5.0 release used very modern compilers to rebuild almost
everything (~99.5%) provided in the installers for x86 Linux and MacOS.
This enables Anaconda users to get the benefits of the latest compilers—
still allowing support for older operating systems—back to MacOS 10.9 and
CentOS 6. Our own builds of GCC 7.2 (Linux) and Clang 4.0.1 (MacOS) are
used, and every reasonable security flag has been enabled. CFLAGS and
CXXFLAGS are no longer managed by each package; instead compiler activation
sets them globally.
> The packages built with the new compilers are in a different channel from
packages built the old way, and as we build out this new channel, we will
eventually be able to change the default experience to only using these
packages. Interested in using this approach to build your own conda
packages? Stay tuned for a more developer-focused blog post!

On Tuesday, October 31, 2017, Ivan Pozdeev via Distutils-SIG <
distutils-sig at python.org> wrote:

> Missed the fact that Nathaniel didn't quote the entire original message.
> Here it is:
> ------------------------------
> Generally, packages are compiled for the same processor generation as the
> corresponding Python.
> But not always -- e.g. NumPy opted for SSE2 even for Py2 to work around
> some compiler bug
> (https://github.com/numpy/numpy/issues/6428).
> I was bitten by that at an old machine once and found out that there is no
> way for `pip' to have checked for that.
> Besides, performance-oriented packages like the one mentioned could
> probably benefit from newer instructions.
> Regarding identifiers:
> gcc, cl and clang all have their private directories of generation
> identifiers:
> https://gcc.gnu.org/onlinedocs/gcc-4.7.1/gcc/i386-
> and-x86_002d64-Options.html
> https://msdn.microsoft.com/en-us/library/7t5yh4fd.aspx
> https://clang.llvm.org/doxygen/Driver_2ToolChains_
> 2Arch_2X86_8cpp_source.html
> Linux packages typically use gcc's ones. Clang generally follows in gcc's
> footsteps and accepts cl's IDs, too, as aliases.
> So, using the IDs of whatever compiler is used to build the package (i.e.
> most likely the canonical compiler for CPython for that platform) looks
> like the simple&stupid(r) way - we can just take the value of the "march"
> argument.
> The tricky part is mapping the system's processor to an ID when checking
> compatibility: the logic will have to keep a directory (that's the job of
> `wheel' package maintainers though, I guess).
> I also guess that there are cases where there's no such thing as _the_
> system's processor.
> --
> Regards,
> Ivan
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20171031/317a5bbf/attachment.html>

More information about the Distutils-SIG mailing list