On 20.05.2021 11:47, Victor Stinner wrote:
On Wed, May 19, 2021 at 3:35 PM Marc-Andre Lemburg <mal@egenix.com> wrote:
On 19.05.2021 12:45, Ronald Oussoren via capi-sig wrote:
This bug is a major issue, because C extensions are no longer linked to libpython since Python 3.8. Python must export all symbols that it pretends to export, especially if it built without libpython dynamic library (libpythonXY.so).
It has been a problem longer than that, if I recall correctly framework builds on macOS have never linked extensions to libpython. That said, as I wrote above this is currently not that large of an issue due to the way the code is structured and the way linkers work.
I believe it's the same on Linux.
There is no such thing as "Linux". There are "Linux distributions" :-) It depends on how Python provided by your Linux distribution was built.
Fedora builds Python with --enable-shared: "python" is linked to "libpython". Debian/Ubuntu builds Python without --enable-shared. If I recall correctly: C extensions are no linked to libpython if Python is not linked to libpython.
Since Python 3.8, distutils no longer links C extensions to libpython (in any case). Fedora fixed packages which linked C extensions to libpython, extensions usually not built by distutils. We also had to modify some applications embedding Pyhon to use python-3.9-embed to link them to libpython. Example on Fedora 34:
$ pkg-config python-3.9 --libs # empty result $ pkg-config python-3.9-embed --libs # link to libpython -lpython3.9
I had checked a larger number of pip downloaded wheel files and none of the .so files link to libpythonXX.so.
I now checked the OS Python 3.6 packages on my SUSE system and they do indeed also link against libpythonXX.so, just like the main Python 3.6 binary on the system.
So in conclusion, it really all depends on how you install Python to begin with.
The default configure settings of Python don't use a shared libpythonXX.so, so packages compiled against such installation will also never use libpythonXX.so, but instead get the symbols from the main executable.
Given that the libpythonXX.so is not the default, the problem is indeed a lot older than Python 3.8. How old depends a lot on the default optimization settings of the linker and compilers.
In any case, you are right: care has to be taken to make sure that we don't accidentally lose exports simply because Python itself no longer uses a function and the symbol gets optimized away from the main executable.
-- Marc-Andre Lemburg eGenix.com
Professional Python Services directly from the Experts (#1, May 20 2021)
Python Projects, Coaching and Support ... https://www.egenix.com/ Python Product Development ... https://consulting.egenix.com/
::: We implement business ideas - efficiently in both time and costs :::
eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 https://www.egenix.com/company/contact/ https://www.malemburg.com/