Incorrect detection of 32/64-bit arch on Linux for old 32-bit VMs migrated to 64-bit kernels
Here's an interesting bug I just discovered on the old and poorly maintained VM that I keep for running random junk: # 64-bit kernel $ uname -a Linux roberts 4.1.5-x86_64-linode61 #7 SMP Mon Aug 24 13:46:31 EDT 2015 x86_64 x86_64 x86_64 GNU/Linux # 32-bit userland $ file /usr/bin/python2.7 /usr/bin/python2.7: ELF 32-bit LSB executable, Intel 80386, version 1 (SYSV), dynamically linked (uses shared libs), for GNU/Linux 2.6.24, BuildID[sha1]=0xf481c2b1f8b4328b2f56b642022cc35b4ad91b61, stripped $ /usr/bin/python2.7 # Yep, definitely 32-bit
import sys; sys.maxint 2147483647
# Uh oh.
import distutils.util; distutils.util.get_platform() 'linux-x86_64'
# Yeah, this is bad
import distlib.wheel; distlib.wheel.COMPATIBLE_TAGS set([(u'cp21', u'none', u'any'), (u'py21', u'none', u'any'), (u'py23', u'none', u'any'), (u'cp24', u'none', u'any'), (u'py2', u'none', u'any'), (u'cp27', u'none', u'any'), (u'cp20', u'none', u'any'), (u'py22', u'none', u'any'), (u'cp26', u'none', u'any'), (u'cp27', u'cp27mu', u'linux_x86_64'), (u'py25', u'none', u'any'), (u'cp23', u'none', u'any'), (u'py24', u'none', u'any'), (u'cp25', u'none', u'any'), (u'py27', u'none', u'any'), (u'cp22', u'none', u'any'), (u'cp2', u'none', u'any'), (u'cp27', u'none', u'linux_x86_64'), (u'py26', u'none', u'any'), (u'py20', u'none', u'any')])
In the past this has never mattered, because there were no linux wheels on pypi, so even if pip was using the wrong platform tag it still wouldn't download the wrong thing. But once manylinux1 goes live, any systems configured like this will start downloading and installing totally wheels that will crash on import. The problem seems to be that distutils.util.get_platform() assumes that the architecture is whatever uname's "machine" field says (equivalent to uname -m). Unfortunately, this gives the architecture of the kernel, not of the Python interpreter. I think the fix is that we should add some check like if osname == "linux" and machine == "x86_64" and sys.maxsize == 2147483647: machine = "i686" to distutils (in the branches where it's still maintained), and also need to add a similar workaround to distlib? (With the later needing to land before or together with manylinux1 enablement.) -n -- Nathaniel J. Smith -- https://vorpus.org
Nathaniel Smith wrote:
The problem seems to be that distutils.util.get_platform() assumes that the architecture is whatever uname's "machine" field says (equivalent to uname -m). Unfortunately, this gives the architecture of the kernel, not of the Python interpreter.
I think the fix is that we should add some check like
if osname == "linux" and machine == "x86_64" and sys.maxsize == 2147483647: machine = "i686"
If you fix this, please do it for other platforms as well. I encountered the same problem on MacOSX, where for historical reasons I have installed Python and all my compiled-from-source libraries as 32 bit. The result is that py2app includes the wrong version of the stub executable in the apps it builds. Conceivably a similar thing could happen on Windows, although I haven't tested it. -- Greg
participants (2)
-
Greg Ewing
-
Nathaniel Smith