[Python-Dev] PEP-582 and multiple Python installations

[Victor Stinner]

Hi,

Le mar. 2 avr. 2019 à 17:20, Calvin Spealman <cspealma at redhat.com> a écrit :
> While the PEP does show the version number as part of the path to the actual packages, implying support for multiple versions, this doesn't seem to be spelled out in the actual text. Presumably __pypackages__/3.8/ might sit beside __pypackages__/3.9/, etc. to keep future versions capable of installing packages for each version, the way virtualenv today is bound to one version of Python.
>
>
> I'd like to raise a potential edge case that might be a problem, and likely an increasingly common one: users with multiple installations of the *same* version of Python.

Hum, I don't know if it's relevant to support multiple Python binaries
of the same Python version, but just in case, let me share my
experience with that in the pyperformance project.

The pyperformance project uses virtual environment for two binaries of
the exact Python version (and usually the same path!): one unpatched
"reference" and one "patched" binary, to experiment an optimization. I
needed a way to build a short text identifier to still be able to get
a "cached" virtual environment per Python binary. I wrote a short code
to generate the identifier using:

* pyperformance version
* requirements.txt
* sys.executable
* sys.version
* sys.version_info
* sys.implementation.name of platform.python_implementation()

The script builds a long string using these info, hash it with SHA1
and take first 12 characters of the hexadecimal format of the hash.

Script:
---
import hashlib
import platform
import sys

performance_version = sys.argv[1]
requirements = sys.argv[2]

data = performance_version + sys.executable + sys.version

pyver= sys.version_info

if hasattr(sys, 'implementation'):
    # PEP 421, Python 3.3
    implementation = sys.implementation.name
else:
    implementation = platform.python_implementation()
implementation = implementation.lower()

if not isinstance(data, bytes):
    data = data.encode('utf-8')
with open(requirements, 'rb') as fp:
    data += fp.read()
sha1 = hashlib.sha1(data).hexdigest()

name = ('%s%s.%s-%s'
        % (implementation, pyver.major, pyver.minor, sha1[:12]))
print(name)
---

Examples:

$ touch requirements.txt # empty file
$ python3.7 x.py version requirements.txt
cpython3.7-502d35b8e005
$ python3.6 x.py version requirements.txt
cpython3.6-7f4febbec0be

$ python3 x.py version requirements.txt
cpython3.7-59ab636dfacb
$ file /usr/bin/python3
/usr/bin/python3: symbolic link to python3.7

Hum, python3 and python3.7 produce the different hash whereas it's the
same binary. Maybe os.path.realpath() should be called on
sys.executable :-)

Victor

--
Night gathers, and now my watch begins. It shall not end until my death.