Where it is decided whether "lib" or "lib64" ends up in sys.path on a POSIX (non MacOS) system?
Hi everyone! First of all, sorry for bringing up, perhaps, a trivial matter, but since it's about the sources I guess it's better to ask it here. I'm investigating a curious problem caused by the fact that in a user's virtualenv, created with the standard "venv" module, "lib64" directory, traditionally symlinked to "lib", gets included in sys.path instead of "lib" itself. ``` $ python -msite sys.path = [ '/current/working/directory' '/usr/lib64/python36.zip', '/usr/lib64/python3.6', '/usr/lib64/python3.6/lib-dynload', '/path/to/venv/lib64/python3.6/site-packages', ] ``` The user is on Gentoo, and, though the venv's layout is exactly the same, it's different from the typical result on a Debian-based distribution where it's the original "lib" in sys.path, not its "lib64" link. I'm wondering what's controlling this behavior. Modules/getpath.c doesn't mention architecture-dependent lib directories anywhere, at least at first glance. Could someone please give me a hint about where to look in the sources or, maybe, which build options are affecting this? Thanks a lot!
The sysconfig & site modules cover the addition of non-stdlib directories to sys.path, but be aware that many Linux distros patch that logic to better align with the conventions of that particular distro. So your discrepancy is likely coming from either build process differences between the Debian & Gentoo packages, or else patches in one distro but not the other have actually altered the behaviour. Cheers, Nick. On Sun., 4 Oct. 2020, 12:41 am Mikhail Golubev via Python-Dev, < python-dev@python.org> wrote:
Hi everyone!
First of all, sorry for bringing up, perhaps, a trivial matter, but since it's about the sources I guess it's better to ask it here.
I'm investigating a curious problem caused by the fact that in a user's virtualenv, created with the standard "venv" module, "lib64" directory, traditionally symlinked to "lib", gets included in sys.path instead of "lib" itself.
``` $ python -msite
sys.path = [ '/current/working/directory' '/usr/lib64/python36.zip', '/usr/lib64/python3.6', '/usr/lib64/python3.6/lib-dynload', '/path/to/venv/lib64/python3.6/site-packages', ] ```
The user is on Gentoo, and, though the venv's layout is exactly the same, it's different from the typical result on a Debian-based distribution where it's the original "lib" in sys.path, not its "lib64" link. I'm wondering what's controlling this behavior. Modules/getpath.c doesn't mention architecture-dependent lib directories anywhere, at least at first glance. Could someone please give me a hint about where to look in the sources or, maybe, which build options are affecting this?
Thanks a lot! _______________________________________________ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-leave@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/5T72JDQG... Code of Conduct: http://python.org/psf/codeofconduct/
FYI Python 3.9 has a new sys.platlibdir to choose between "lib" and "lib64". Fedora and OpenSUSE use /usr/lib64 directory on 64-bit systems rather than /usr/lib. On 64-bit Fedora, sys.platlibdir is set to lib64: $ python3.9 -c 'import sys; print(sys.platlibdir)' lib64 Commented (simplified) sys.path: $ python3.9 -m site sys.path = [ (...) '/usr/lib64/python3.9', # stdlib (pure Python) '/usr/lib64/python3.9/lib-dynload', # stdlib (C extensions) (...) '/usr/lib64/python3.9/site-packages', '/usr/lib/python3.9/site-packages', # RPM packages installed by dnf ] (...) Fedora also uses sub-directories in /usr/local for packages installed by pip (here are directories of Python 3.8): '/usr/local/lib64/python3.8/site-packages', '/usr/local/lib/python3.8/site-packages', For curious people, see https://bugs.python.org/issue1294959 which took 15 years to be fixed! Fedora and OpenSUSE maintained a large downstream stream for at least 10 years :-) I added sys.platlibdir to remove this downstream change. Victor Le sam. 3 oct. 2020 à 16:46, Mikhail Golubev via Python-Dev <python-dev@python.org> a écrit :
Hi everyone!
First of all, sorry for bringing up, perhaps, a trivial matter, but since it's about the sources I guess it's better to ask it here.
I'm investigating a curious problem caused by the fact that in a user's virtualenv, created with the standard "venv" module, "lib64" directory, traditionally symlinked to "lib", gets included in sys.path instead of "lib" itself.
``` $ python -msite
sys.path = [ '/current/working/directory' '/usr/lib64/python36.zip', '/usr/lib64/python3.6', '/usr/lib64/python3.6/lib-dynload', '/path/to/venv/lib64/python3.6/site-packages', ] ```
The user is on Gentoo, and, though the venv's layout is exactly the same, it's different from the typical result on a Debian-based distribution where it's the original "lib" in sys.path, not its "lib64" link. I'm wondering what's controlling this behavior. Modules/getpath.c doesn't mention architecture-dependent lib directories anywhere, at least at first glance. Could someone please give me a hint about where to look in the sources or, maybe, which build options are affecting this?
Thanks a lot! _______________________________________________ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-leave@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/5T72JDQG... Code of Conduct: http://python.org/psf/codeofconduct/
-- Night gathers, and now my watch begins. It shall not end until my death.
Oh, wow. Thanks for all the insights! Since there just wasn't a standard solution for that in the form of sys.platlibdir before 3.9, I guess I should better check the downstream version of CPython sources used by Gentoo directly. The fact that Fedora and OpenSUSE use the same layout is also very helpful. On Mon, Oct 5, 2020 at 11:57 AM Victor Stinner <vstinner@python.org> wrote:
FYI Python 3.9 has a new sys.platlibdir to choose between "lib" and "lib64". Fedora and OpenSUSE use /usr/lib64 directory on 64-bit systems rather than /usr/lib.
On 64-bit Fedora, sys.platlibdir is set to lib64:
$ python3.9 -c 'import sys; print(sys.platlibdir)' lib64
Commented (simplified) sys.path:
$ python3.9 -m site sys.path = [ (...) '/usr/lib64/python3.9', # stdlib (pure Python) '/usr/lib64/python3.9/lib-dynload', # stdlib (C extensions) (...) '/usr/lib64/python3.9/site-packages', '/usr/lib/python3.9/site-packages', # RPM packages installed by dnf ] (...)
Fedora also uses sub-directories in /usr/local for packages installed by pip (here are directories of Python 3.8):
'/usr/local/lib64/python3.8/site-packages', '/usr/local/lib/python3.8/site-packages',
For curious people, see https://bugs.python.org/issue1294959 which took 15 years to be fixed! Fedora and OpenSUSE maintained a large downstream stream for at least 10 years :-) I added sys.platlibdir to remove this downstream change.
Victor
Le sam. 3 oct. 2020 à 16:46, Mikhail Golubev via Python-Dev <python-dev@python.org> a écrit :
Hi everyone!
First of all, sorry for bringing up, perhaps, a trivial matter, but
since it's about the sources I guess it's better to ask it here.
I'm investigating a curious problem caused by the fact that in a user's
instead of "lib" itself.
``` $ python -msite
sys.path = [ '/current/working/directory' '/usr/lib64/python36.zip', '/usr/lib64/python3.6', '/usr/lib64/python3.6/lib-dynload', '/path/to/venv/lib64/python3.6/site-packages', ] ```
The user is on Gentoo, and, though the venv's layout is exactly the same, it's different from the typical result on a Debian-based distribution where it's the original "lib" in sys.path, not its "lib64" link. I'm wondering what's controlling this behavior. Modules/getpath.c doesn't mention architecture-dependent lib directories anywhere, at least at first glance. Could someone please give me a hint about where to look in the
virtualenv, created with the standard "venv" module, "lib64" directory, traditionally symlinked to "lib", gets included in sys.path sources or, maybe, which build options are affecting this?
Thanks a lot! _______________________________________________ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-leave@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at
https://mail.python.org/archives/list/python-dev@python.org/message/5T72JDQG...
Code of Conduct: http://python.org/psf/codeofconduct/
-- Night gathers, and now my watch begins. It shall not end until my death.
-- Mikhail Golubev Software Developer JetBrains http://www.jetbrains.com The drive to develop
participants (3)
-
Mikhail Golubev -
Nick Coghlan -
Victor Stinner