[Python-Dev] PEP-582 and multiple Python installations

[Matthias Klose]

On 02.04.19 18:10, Steve Dower wrote:
> On 02Apr2019 0817, Calvin Spealman wrote:
>> (I originally posted this to python-ideas, where I was told none of this PEP's
>> authors subscribe so probably no one will see it there, so I'm posting it here
>> to raise the issue where it can get seen and hopefully discussed)
> 
> Correct, thanks for posting. (I thought we had a "discussions-to" tag with
> distutils-sig on it, but apparently not.)
> 
>> While the PEP does show the version number as part of the path to the actual
>> packages, implying support for multiple versions, this doesn't seem to be
>> spelled out in the actual text. Presumably __pypackages__/3.8/ might sit
>> beside __pypackages__/3.9/, etc. to keep future versions capable of installing
>> packages for each version, the way virtualenv today is bound to one version of
>> Python.
>>
>> I'd like to raise a potential edge case that might be a problem, and likely an
>> increasingly common one: users with multiple installations of the *same*
>> version of Python. This is actually a common setup for Windows users who use
>> WSL, Microsoft's Linux-on-Windows solution, as you could have both the Windows
>> and Linux builds of a given Python version installed on the same machine. The
>> currently implied support for multiple versions would not be able to separate
>> these and could create problems if users pip install a Windows binary package
>> through Powershell and then try to run a script in Bash from the same
>> directory, causing the Linux version of Python to try to use Windows python
>> packages.
>>
>> I'm not actually sure what the solution here is. Mostly I wanted to raise the
>> concern, because I'm very keen on WSL being a great entry path for new
>> developers and I want to make that a better experience, not a more confusing
>> one. Maybe that version number could include some other unique identify, maybe
>> based on Python's own executable. A hash maybe? I don't know if anything like
>> that already exists to uniquely identify a Python build or installation.
> 
> Yes, this is a situation we're aware of, and it's caught in the conflict of "who
> is this feature meant to support".

This smells the same like mixing system installed python packages (deb/rpm) with
one managed by pip, and pip touching system installed packages.

> Since all platforms have a unique extension module suffix (e.g.
> "module.cp38-win32.pyd"), it would be possible to support this with "fat"
> packages that include all binaries (or some clever way of merging wheels for
> multiple platforms).

unfortunately not. The Android developers opted out of that, reverting that
change.  Also how would you differentiate win32 builds for different
architectures? But maybe this is already done.

> And since this is already in CPython itself, it leads to about the only
> reasonable solution - instead of "3.8", use the extension module suffix
> "cp38-win32". (Wheel tags are not in core CPython, so we can't use those.)
> 
> But while this seems obvious, it also reintroduces problems that this has the
> potential to fix - suddenly, just like installing into your global environment,
> your packages are not project-specific anymore but are Python-specific. Which is
> one of the major confusions people run into ("I pip installed X but now can't
> import it in python").
> 
> So the main points of discussion right now are "whose problem does this solve"
> and "when do we tell people they need a full venv". And that discussion is
> mostly happening at
> https://discuss.python.org/t/pep-582-python-local-packages-directory/963/
> 
> Cheers,
> Steve
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: https://mail.python.org/mailman/options/python-dev/doko%40ubuntu.com