On 29 March 2014 07:23, Paul Moore email@example.com wrote:
On 28 March 2014 21:06, Nick Coghlan firstname.lastname@example.org wrote:
So consider me in the school that suggests this be a standard installer feature that can be applied to any entry point script, rather than something that varies by project.
Doing that also implies (detecting and) deliberately ignoring any versioned entry points created by setup.py. I'd suggest going for a rule that if a project defines two entry points with the same definitions, where one ends in the pattern -?\d(.\d)? (in other words, -X, -X.Y, X or X.Y) but they are otherwise the same, then ignore the versioned one. If we don't do this, we'll likely end up with abominations like virtualenv-2.7-3.4.exe.
We need to be careful with that - scripts may end with a version number unrelated to the Python version. If we can get agreement on what we want the *default* idiom to look like (more on that below), then we can start considering the special cases where that idiom breaks down (like those where the base script name already ends with a number, perhaps by inserting a leading "-py" into the suffix in those cases).
I'm not 100% sure whether you're saying that the installer should mandate a versioning policy, or if the user can specify. The altinstall case suggests we need a user override option.
I'm biased because there are actually two places where this matters. It's not just the user scripts directory (everywhere), but also the system binary directories on Linux distros.
(Note: I haven't run the ideas below by Slavek or Toshio yet, so take the finer details with a grain of salt - but the general principles should be right. I don't believe Slavek is making it to PyCon, but Toshio is, so we should be able to thrash out some more specifics there)
From a Linux distro perspective, these versioned scripts should
arguably have different shebang lines - ignoring the parts that would remain the same, "pip" would correspond to "python", "pip3" to "python3" and "pip3.4" to "python3.4". Otherwise you could change the "python" symlink to point somewhere else, and then have to figure out why "pip" appeared to succeed, but you couldn't import things you just installed (because they were installed into the wrong version).
For system wide installs, *which* ones to install should also be possible to largely automate by following a convention based on the final name element in sys.executable (and that approach would also fix the problem of potential conflicts in the user scripts directory).
There's also the fact that for universal (or otherwise cross-version) wheels, which scripts get installed should be governed by the version of Python used for the *installation*, rather than the one used to do the build.
If we put this under the control of the installer, then there is some hope of eventually bringing order to the chaos and getting some kind of consistency in naming schemes and alignment with the shebang lines. If we leave it up to the individual projects? Not a chance (especially since we *can't* get it right if the versioned scripts are embedded in wheel files at build time).
There's also the question of if I do "pip install virtualenv --use-versioned X,X.Y" and then "pip install -U virtualenv" will the installer remember my versioning choices for the upgrade? Doing so needs some complex logic. Not doing so means that "pip install -U pip" could change the executables ensurepip had chosen to create.
Yeah, I think driving it based on sys.executable is a better idea, with a fallback of "-<interpretername>" as the suffix if the final component of sys.executable doesn't follow one of the recognised patterns (python, pythonX, pythonX.Y). The reason for using sys.executable is because that is the thing that can tell us the interpreter's role *with respect to the larger system*.
Under that model, on a current Fedora system, we would see the following behaviour when run under the following interpreters:
python -> pip, pip2, pip2.7 python2 -> pip2, pip2.7 python2.7 -> pip2.7 python3 -> pip3, pip3.3 python3.3 -> pip3.3 pypy -> pip-pypy
That avoids collisions not only in /usr/bin (or /usr/local/bin if we change the install directory for scripts on POSIX), but also in ~/.local/bin (and the per-user directory on Windows).
On Windows system wide installs, since the Scripts directory path already includes the Python version, I'd just install all 3 in all cases. Ditto for virtual environments (even when using an alternate interpreter implementation).
That's essentially using ensurepip's behaviour as precedent for how we *want* Python scripts to behave.
You may ask "why would you ever do that?", and from my perspective, the answer is because Fedora is migrating over to using "pip" in the RPM build process for Python packages, rather than calling setup.py directly. This means we can more easily adapt to changes in upstream conventions and automatically take advantage of the upcoming metadata 2.0 enhancements and related installation database changes. However, it also means we're relying on being able to get pip to play nice with Fedora's dual stack python2/python3 setup, even as we start migrating more and more components over to Python 3
The one "user" (i.e. "specfile author") controlled behaviour we would likely want is the ability to decide whether Python 2 or Python 3 was considered the default for a given module - when pure applications that just happen to be written in Python (rather than Python libraries or Python specific tools) cut over to Python 3, they'll likely just switch entirely, rather than going the dual stack Python 2/3 approach.