[Distutils] PEP425 - Wheel Binary Package Compatibility

[Ned Deily]

In article 
<CAFvE7K4U7L4+8n3U72eHER4fa8m-fCANv0q75BhM1AaLfmA2nw at mail.gmail.com>,
 Olivier Grisel <olivier.grisel at ensta.org> wrote:
> Homebrew-installed Python and the system Python that comes by default
> on OSX are binary compatible with wheels generated with Python from
> the official OSX installer from python.org.
> 
> In current versions of pip you "just" have to rename the wheel
> filenames to get them picked up from PyPI on those platforms.
> 
> This explains the strange names of the OSX wheels for numpy for instance:
> 
> numpy-1.9.0-cp27-none-macosx_10_6_intel.macosx_10_9_intel.macosx_10_9_x86_64.w
> hl
> numpy-1.9.0-cp33-cp33m-macosx_10_6_intel.macosx_10_9_intel.macosx_10_9_x86_64.
> whl
> numpy-1.9.0-cp34-cp34m-macosx_10_6_intel.macosx_10_9_intel.macosx_10_9_x86_64.
> whl
> 
> https://pypi.python.org/pypi/numpy
> 
> macosx_10_9_x86_64 is the platform tag used by Homebrew-installed
> Python while macosx_10_6_intel is the platform tag of the python.org
> Python.

To expand on this, the platform tag is generated by Python's Distutils 
and wheels use and expand on the pattern established by it.  For OS X, 
the "10_x" part is based on the minimum OS X deployment target of the 
Python build, normally set by the compiler -mmacosx-version-min argument 
or the MACOSX_DEPLOYMENT_TARGET env variable at configure time (these 
are standard options supported by the Apple-supplied buildtool chains).  
That value is saved and used by Distutils in subsequent extension module 
builds, although you can override it to a higher (not lower) value for a 
particular build.  The arch values in the tag also come from the 
configured universal architectures.  Any Python configured the same way 
(wrt MACOSX_DEPLOYMENT_TARGET and archs) will supply the same platform 
tag.  So there's nothing special about Homebrew or MacPorts or 
python.org or the Apple-supplied system Pythons; it is assumed that any 
identically configured Python will provide a compatible ABI for C 
extension modules.  We also assume that Apple will provide backward 
compatibility on newer releases of OS X, such that, for example, an 
extension module compiled for 10.6 will run just fine on later OS X 
releases *and* will run with a Python configured for a later OS X 
release, assuming there is a non-zero intersection of universal archs 
between the interpreter and the extension module.  In practice, those 
assumptions seems to have worked fairly well on OS X, starting from the 
beginning of universal arch support and at least up to now although they 
may break down in the future.  It's still not perfect. For example, the 
platform string doesn't take into account the Python 2 
--enable-unicode=ucs2/ucs4 build option; but ucs4 builds aren't widely 
used on OS X and Python 3 no longer has this problem.  (For C++ 
extension modules, there also may be issues with Apple's change in 
default c++ libraries in OS X 10.9.)

On Linux systems, it can be dangerous to make similar simplifying 
assumptions, with many more variables in play, much less reflected in 
the platform string.

-- 
 Ned Deily,
 nad at acm.org