[New-bugs-announce] [issue22490] Using realpath for __PYVENV_LAUNCHER__ makes Homebrew installs fragile

Tim Smith report at bugs.python.org
Thu Sep 25 07:30:45 CEST 2014

New submission from Tim Smith:

Homebrew, the OS X package manager, distributes python3 as a framework build. We like to be able to control the shebang that gets written to scripts installed with pip. [1]

The path we prefer for invoking the python3 interpreter is like /usr/local/opt/python3/bin/python3.4, which is symlinked to the framework stub launcher at /usr/local/Cellar/python3/3.4.1_1/Frameworks/Python.framework/Versions/3.4/bin/python3.4. For Python 2.x, we discovered that assigning "/usr/local/opt/python/bin/python2.7" to sys.executable in sitecustomize.py resulted in correct shebangs for scripts installed by pip. The same approach doesn't work with Python 3.

A very helpful conversation with Vinay Sajip [2] led us to consider how the python3 stub launcher sets __PYVENV_LAUNCHER__, which distlib uses in preference to sys.executable to discover the intended interpreter when pip writes shebangs.

Roughly, __PYVENV_LAUNCHER__ is set from argv[0], so it mimics the invocation of the stub, except that symlinks in the directory component of the path to the identified interpreter are resolved to a "real" path. For us, this means that __PYVENV_LAUNCHER__ (and therefore the shebangs of installed scripts) always points to the Cellar path, not the preferred opt path, even when python is invoked via the opt path.

Avoiding this symlink resolution would allow us to control pip's shebang (which sets the shebangs of all pip-installed scripts) by controlling the way we invoke python3 when we use ensurepip during installation.

Building python3 with the attached diff removes the symlink resolution.

[1]  This is important to Homebrew because packages are physically installed to versioned prefixes, like /usr/local/Cellar/python3/3.4.1_1/. References to these real paths are fragile and break when the version number changes or when the revision number ("_1") changes, when can happen when e.g. openssl is released and Python needs to be recompiled against the new library. To avoid this breakage, Homebrew maintains a version-independent symlink to each package, like /usr/local/opt/python3, which points to the .../Cellar/python3/3.4.1_1/ location.

[2] https://github.com/pypa/pip/issues/2031

assignee: ronaldoussoren
components: Macintosh
files: dont-realpath-venv-dirname.diff
keywords: patch
messages: 227505
nosy: ned.deily, ronaldoussoren, tdsmith, vinay.sajip
priority: normal
severity: normal
status: open
title: Using realpath for __PYVENV_LAUNCHER__ makes Homebrew installs fragile
type: behavior
versions: Python 3.4, Python 3.5
Added file: http://bugs.python.org/file36718/dont-realpath-venv-dirname.diff

Python tracker <report at bugs.python.org>

More information about the New-bugs-announce mailing list