Thanks folks for the feedback (here and on the draft PR). This is now PEP 668, and there's a Discourse thread for feedback: https://discuss.python.org/t/graceful-cooperation-between-external-and-pytho... Replies to a couple of things inline: On Tue, 8 Jun 2021, Petr Viktorin wrote:
On 02. 06. 21 16:45, Matthias Klose wrote:
On 6/1/21 3:37 PM, Petr Viktorin wrote:
Hi!
FWIW, we plan for Fedora's system tools to have `-s` in Python shebangs by default, meaning that the user’s Python packages (whether installed by `pip install --user` or just placed in the current directory) don’t interfere with the RPM installed software. Extensible software like Ansible or Sphinx should leave off the `-s`, and plugins for them should be installed using `pip install --user`. Does that seem like a reasonable general suggestion?
Is this good enough? We shortly discussed to have sys.path having *only* the distro "site-packages" on sys.path, which would be 'python -s' additionally without any locally pip installed library/module. So maybe it's worth to implement a python -D which is exactly doing this?
Yes, I think that, along with an option to skip adding the current directory, would be good additions.
I think these approaches (which I am in favor of personally) are basically equivalent to what the PEP already talks about in the section about "System Python." Basically, - These are (or were, in May) new ideas, they'll probably work, but it's premature to recommend them to other distros. "Install things in virtualenv and stay out of your distro Python's sys.path" is something that a lot of people already do, so it's a much safer recommendation. - You have to make a package-by-package judgment call on "Is this like Ansible or Sphinx, or not." - Commands like Ansible and Sphinx remain prone to being broken, and this PEP is still worthwhile because it reduces the risk of breaking them - the less stuff people use with "pip install --user", the less likely it is that some of Ansible or Sphinx's (many) dependencies gets accidentally shadowed. There's some debate among the coauthors about whether "System Python" is a good idea (and what variants of it are good ideas), so we didn't endorse it in this proposal, but we said that it's definitely worth exploring and experimenting with.
One more note for the Alternatives section, to go along with the INSTALLER discussion: PEP 627 mentions one more way to mark an installed package as not modifiable by Python-specific package manager: removing the RECORD file (https://www.python.org/dev/peps/pep-0627/#optional-record-file). It seems that this PEP obsolete one use case for missing RECORD. The other use case remains -- it needs to be kept it in sync with distro actually installs, which can be fragile.
sorry, I fail to understand/parse this section.
It is possible to mark individual installed packages so that Python tools (pip) won't uninstall them. This is done by removing (or not including) the RECORD file from the metadata. So, distros can already prevent pip from uninstalling (but not shadowing) their packages. I think this fact is as relevant to the draft PEP as the other discussion in "Alternatives".
Miro Hrončok also brought this up on the GitHub draft, so it's worth mentioning. I think we should add this to the "Recommendations for Distros" section, namely, that distros should continue not to ship a RECORD file, even though that's no longer the tool that prevents pip from removing distro packages.
A few nitpicks on terminology:
For practical reasons, I suggest using "distro" rather than "distribution" to avoid a conflict with the meaning from https://packaging.python.org/glossary/ While this document disambiguates quite clearly, standardizing on "distro" could make future discussions clearer.
Done, and thanks Matthias for sending me a patch to do so - I got a few more spots and so now it consistently avoids "distribution".
PyPI meets the definition of such a distro; it might be good to exclude it explicitly.
I'm fine with that change, as long as people understand that e.g. Conda and HomeBrew are considered a distro as well.
Yes, I'd mention that explicitly. The current definition already does that.
Hmmm, I'm not sure exactly how to phrase that PyPI is not a distro but Conda is. Both use Python-aware package management tools (pip / conda). And while Conda is well-known for having non-Python software (R, curl, etc.), so does PyPI (patchelf, ninja, etc.). And we expect that Conda will _not_ provide an EXTERNALLY-MANAGED file, though Homebrew will. So maybe it's okay to say that technically both PyPI and Conda are distros, but since they use Python-specific package managers as their primary package manager, it's okay for them to choose not to implement EXTERNALLY-MANAGED? -- Geoffrey Thomas https://ldpreload.com geofft@ldpreload.com