[Distutils] PEX at Twitter (re: PEX - Twitter's multi-platform executable archive format for Python)

[Nick Coghlan]

On 1 February 2014 18:50, Noah Kantrowitz <noah at coderanger.net> wrote:
> On Feb 1, 2014, at 12:43 AM, Nick Coghlan <ncoghlan at gmail.com> wrote:
>> That said, something I mentioned to the OpenStack folks a while ago
>> (and I think on this list, but potentially not), is that I have now
>> realised the much-reviled (for good reason) *.pth files actually have
>> a legitimate use case in allowing API compatible versions of packages
>> to be shared between multiple virtual environments - you can trade
>> reduced isolation for easier upgrades on systems containing multiple
>> virtual environments by adding a suitable *.pth file to the venv
>> rather than the package itself. While there's currently no convenient
>> tooling around that, they're a feature CPython has supported for as
>> long as I can remember, so tools built on that idea would comfortably
>> work on all commonly supported Python versions.
>
> In all but a tiny number of cases, you could use a symlink for this. Much less magic :-)

Unfortunately, that has two problems:

- the PEP 376 installation database directories are versioned so you
would need at least *two* symlinks (and potentially more for packages
that include multiple top level entries) for the installation metadata
to be properly available, and the metadata one would still have to be
updated in every virtual environment whenever the shared version
changed
- Windows symlinks are still broken and hard to use compared to POSIX symlinks

If it wasn't for the "support execution of arbitrary Python code
during interpreter startup" "feature" *.pth files likely wouldn't
worry so many people, but we're stuck with that now for backwards
compatibility reasons :P (Well, and because nobody thought to suggest
reducing its power to just adding new sys.path entries in Python 3...)

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia