[Python-ideas] bdist naming scheme (compatibility tags) PEP

[Daniel Holth]

On Aug 9, 2012 4:07 AM, "M.-A. Lemburg" <mal at egenix.com> wrote:

> Daniel Holth wrote:
> > Platform Tag
> > ------------
> >
> > The platform tag is simply `distutils.util.get_platform()` with all
> > hyphens `-` and periods `.` replaced with underscore `_`.
>
> This part is going to cause problems. distutils is good at identifying
> Linux and Windows and giving them sensible platform names, but it
> doesn't do a good job for other OSes.
>
> For e.g. FreeBSD it adds too much detail, for Mac OS X it doesn't have
> enough detail and it also has a tendency to change even for Python
> dot releases (esp. for Mac OS X which constantly causes problems).
>

egg does something a little more specific for OS X. I should probably copy
that. egg is obviously a big influence on this work.

I downloaded 862 eggs in May, 72 of which were platform-specific:

Counter({'win32': 48,
'linux-x86_64': 9,
'macosx-10.6-fat': 8,
'linux-i686': 2,
'macosx-10.7-intel': 2,
'macosx-10.4-x86_64': 1,
'macosx-10.5-intel': 1,
'macosx-10.5-i386': 1})

I think your naming scheme ought to focus more on the platform
> part, as the other parts (Python version and implementation) are
> well understood.
>

OK


> For the platform, the installer would have to detect whether a
> package is compatible with the platform. This often requires intimate
> knowledge about the platform.
>
> Things to consider:
>
>  * OS name
>  * OS version, if that matters for compatibility
>  * C lib version, if that matters for compatibility
>  * ABI version, if that matters for compatibility
>  * architecture (Intel, PowerPC, Sparc, ARM, etc)
>  * bits (32, 64, 128, etc.)
>  * fat builds which include multiple variants in a single
>    archive
>
> and probably some more depending on OS.
>
> In some cases, a package will also have external requirements such
> as specific versions of a library (e.g. 0.9.8 vs. 1.0.0 OpenSSL
> library, or 2.2 vs. 2.3 unixODBC). These quickly get complicated
> up to the point where you need to run a script in order to determine

whether a platform is compatible with the package or not.
>

The external library requirements are out of scope for these tags. There is
a suitable Metadata 1.2 tag for external requirements.

Putting all that information into a tag is going to be difficult,
> so an installer will either have to access more meta information
> about the package from some other resource than the file name
> (which is what PyPI is heading at), or download all variants
> that fit the target platform and then look inside the files
> for more meta information.
>
> So the tag name format will have to provide a set of basic
> "dimensions" for the platform (e.g. OS name, architecture, bits),
> but also needs to provide user defined additions that can
> be used to differentiate between all the extra variants which
> may be needed, and which can easily be parsed by a human with
> more background knowledge about the target system and his/her
> needs to select the right file.
>

I don't want anyone to manually download packages. It just doesn't work
when you have a lot of dependencies.

I am interested in an 80% solution to this problem. Like the people who
have uploaded eggs to pypi, I use Windows, Mac, and Linux. If someone can
provide a good get_platform() for other platforms, great. I don't have that
knowledge.

Why don't I add the platform tag "local". Pre-built binary packages on pypi
are most-necessary for Windows where it is hard to install the compiler,
then Mac, and then Linux where you usually do have a compiler. If you are
on a less common platform that always compiles everything from source
anyway then you might compile a local cache of -local tagged binary
packages. The tools will know not to upload these to pypi.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20120809/e06a26c2/attachment.html>