[New-bugs-announce] [issue44009] Provide "python3.x-intel64" command with macOS universal2 framework builds

Ned Deily report at bugs.python.org
Sun May 2 03:23:23 EDT 2021

New submission from Ned Deily <nad at python.org>:

On certain macOS configurations, it is possible to have a universal build (one that contains executables and libraries with binaries for more than one CPU architecture combined into "fat" files) where running with a choice of archs. Examples in the past included Intel-64 (x86_64) vs Intel-32 (i386) vs (in macOS 10.5 and 10.6) ppc (using Rosetta emulation. While it is possible to use the macOS arch utility to select a non-default arch for execution:
   arch -i386 /usr/local/bin/python3
this method does not handle the common case of a Python program spawning another Python subprocess (to run a test, say) by calling the subprocess module to execute the interpreter binary recorded in sys.executable, which only contains the interpreter executable file name, not the arch command.  Thus the subprocess attempts to launch in the default arch for the configuration rather than the overriden arch, which may cause program failures or incorrect results.  To get around this where appropriate, framework universal builds on macOS provided an additional 32-bit-only python3.x executable file, python3.x-32, that solves this problem: by invoking Python with python3.x-32, the main interpreter process runs in 32-bit mode and any spawned subprocesses using sys.executable will also run in 32-bit mode.

With the introduction of Apple Silicon Macs and their support for running x86_64 Intel-64 binaries with Rosetta 2 emulation, the need to be able to force Python to run a non-default arch reliably is again important for the transition period while third-party packages are being ported or tested on the new arm64-based Macs. For example, there are some popular packages on PyPI that do not yet provide universal2 or just arm64 wheels but, by forcing Python to run in Intel mode, existing wheels can be tested and used.

To that end, the PR for this issue adds a "python3.x-intel64" and "python3-intel64" executable or symlink, as needed, when installing a macOS "universal2" framework build, similar to the "python3.x-32" and "python3-32" links installed for macOS "intel" framework builds.

An example:

$ sw_vers 
ProductName:	macOS
ProductVersion:	11.3
BuildVersion:	20E232
$ uname -a
Darwin pyb20 20.4.0 Darwin Kernel Version 20.4.0: Fri Mar  5 01:14:02 PST 2021; root:xnu-7195.101.1~3/RELEASE_ARM64_T8101 arm64
$ python3.8 -m pip install --only-binary ':all:' numpy
ERROR: Could not find a version that satisfies the requirement numpy (from versions: none)
ERROR: No matching distribution found for numpy
$ python3.8-intel64 -m pip install --only-binary ':all:' numpy 
Collecting numpy
  Using cached numpy-1.20.2-cp38-cp38-macosx_10_9_x86_64.whl (16.0 MB)
Installing collected packages: numpy
Successfully installed numpy-1.20.2

Of course, for this to be useful assumes that running under Rosetta 2 emulation provides the correct results. Testing is advised!

assignee: ned.deily
components: macOS
messages: 392669
nosy: ned.deily, ronaldoussoren
priority: normal
severity: normal
status: open
title: Provide "python3.x-intel64" command with macOS universal2 framework builds
versions: Python 3.10, Python 3.8, Python 3.9

Python tracker <report at bugs.python.org>

More information about the New-bugs-announce mailing list