[Distutils] RFC: package multiple platforms into a single binary package

[Daniel Holth]

I've been trying to work out a filename convention and specification
for packaging multiple platforms worth of Python into a single
archive. The feature would reduce the number of mostly-redundant
archives required for a package that is mostly Python but has a small
extension module, or express that a package contains fat binaries a-la
OS X.


Now that Python includes the __pycache__ directory and ABI tags for
extension modules, it will often be possible to copy installations of
the same package compiled for different Python ABIs on top of each
other without clobbering any files. For example, a directory could
include .pyc files for Python 3.2 and 3.3, or extension modules for
win32 and Linux at the same time. When copied into the same location,
the only files clobbered by the second architecture should be the
identical .py, .dist-info/METADATA and so forth, while the second
platform's shared libraries will have a unique name.

The proposed filename is {distribution}-{version}-{python
tag}-{platform tag}.{archive extension}

distribution: Distribution name, e.g. ‘django’, ‘pyramid’
version: PEP 386-compliant version, e.g. 1.0
Python implementation tag
Platform tag or 'noarch'

For example, package-1.0-py27-noarch.rar is compatible with Python 2.7
(any Python 2.7 implementation) on any CPU architecture.

The Python implementation is abbreviated to save space. Each
implementation has a two-letter code:

py: Generic Python
cp: CPython
ip: IronPython
pp: PyPy
jy: Jython

concatenated with the ABI tag: SOABI.split(‘-’)[-1] “33m”, or, for
“pure Python” packages, py_version_nodot “27”.

The platform tag is distutils.util.get_platform() with all periods and
hyphens replaced with underscore, or the string ‘noarch’.

The {python tag} field and {platform tag} field are actually
.-delimited sets of tags. The filename indicates compatibility with
the cartesian product of the two sets.

For example, you would be able to combine a win32 and linux_x86_64 archive into

package-1.0-py33-linux_x86_64.win32.tar.bz2


The part I'm unsure about is that 'noarch' and 'py27' don't quite fit
with 'cp33dmu' (which says something about both the compiled code and
the version of Python required by the .py files).

Thanks,

Daniel Holth