https://github.com/python/cpython/commit/8c3ecc6bacc8d0cd534f2b5b53ed962dd13... commit: 8c3ecc6bacc8d0cd534f2b5b53ed962dd1368c7b branch: master author: Victor Stinner <vstinner@redhat.com> committer: GitHub <noreply@github.com> date: 2019-04-25T20:13:10+02:00 summary: bpo-21536: C extensions are no longer linked to libpython (GH-12946) On Unix, C extensions are no longer linked to libpython. It is now possible to load a C extension built using a shared library Python with a statically linked Python. When Python is embedded, libpython must not be loaded with RTLD_LOCAL, but RTLD_GLOBAL instead. Previously, using RTLD_LOCAL, it was already not possible to load C extensions which were not linked to libpython, like C extensions of the standard library built by the "*shared*" section of Modules/Setup. distutils, python-config and python-config.py have been modified. files: A Misc/NEWS.d/next/Build/2019-04-25-01-51-52.bpo-21536.ACQkiC.rst M Doc/distutils/apiref.rst M Doc/whatsnew/3.8.rst M Lib/distutils/command/build_ext.py M Makefile.pre.in M Misc/python-config.in M Misc/python-config.sh.in diff --git a/Doc/distutils/apiref.rst b/Doc/distutils/apiref.rst index a825efc1a672..c3cdfc8a0a8e 100644 --- a/Doc/distutils/apiref.rst +++ b/Doc/distutils/apiref.rst @@ -277,6 +277,10 @@ the full reference. | | simply skip the extension. | | +------------------------+--------------------------------+---------------------------+ + .. versionchanged:: 3.8 + + On Unix, C extensions are no longer linked to libpython. + .. class:: Distribution diff --git a/Doc/whatsnew/3.8.rst b/Doc/whatsnew/3.8.rst index 344656b9288f..2270334a281b 100644 --- a/Doc/whatsnew/3.8.rst +++ b/Doc/whatsnew/3.8.rst @@ -851,16 +851,19 @@ Changes in the Python API Changes in the C API -------------------- +* On Unix, C extensions are no longer linked to libpython. When Python is + embedded, ``libpython`` must not be loaded with ``RTLD_LOCAL``, but + ``RTLD_GLOBAL`` instead. Previously, using ``RTLD_LOCAL``, it was already not + possible to load C extensions which were not linked to ``libpython``, like C + extensions of the standard library built by the ``*shared*`` section of + ``Modules/Setup``. + * Use of ``#`` variants of formats in parsing or building value (e.g. :c:func:`PyArg_ParseTuple`, :c:func:`Py_BuildValue`, :c:func:`PyObject_CallFunction`, etc.) without ``PY_SSIZE_T_CLEAN`` defined raises ``DeprecationWarning`` now. It will be removed in 3.10 or 4.0. Read :ref:`arg-parsing` for detail. (Contributed by Inada Naoki in :issue:`36381`.) - -Changes in the C API --------------------------- - * Instances of heap-allocated types (such as those created with :c:func:`PyType_FromSpec`) hold a reference to their type object. Increasing the reference count of these type objects has been moved from diff --git a/Lib/distutils/command/build_ext.py b/Lib/distutils/command/build_ext.py index 0428466b00c9..1672d02acf1f 100644 --- a/Lib/distutils/command/build_ext.py +++ b/Lib/distutils/command/build_ext.py @@ -714,20 +714,5 @@ def get_libraries(self, ext): # don't extend ext.libraries, it may be shared with other # extensions, it is a reference to the original list return ext.libraries + [pythonlib] - else: - return ext.libraries - elif sys.platform == 'darwin': - # Don't use the default code below - return ext.libraries - elif sys.platform[:3] == 'aix': - # Don't use the default code below - return ext.libraries - else: - from distutils import sysconfig - if sysconfig.get_config_var('Py_ENABLE_SHARED'): - pythonlib = 'python{}.{}{}'.format( - sys.hexversion >> 24, (sys.hexversion >> 16) & 0xff, - sysconfig.get_config_var('ABIFLAGS')) - return ext.libraries + [pythonlib] - else: - return ext.libraries + + return ext.libraries diff --git a/Makefile.pre.in b/Makefile.pre.in index cd7098cac72b..68ac7723556e 100644 --- a/Makefile.pre.in +++ b/Makefile.pre.in @@ -1460,7 +1460,7 @@ libinstall: build_all $(srcdir)/Modules/xxmodule.c -PYTHONPATH=$(DESTDIR)$(LIBDEST) $(RUNSHARED) \ $(PYTHON_FOR_BUILD) -m lib2to3.pgen2.driver $(DESTDIR)$(LIBDEST)/lib2to3/PatternGrammar.txt -python-config: $(srcdir)/Misc/python-config.in Misc/python-config.sh +python-config: $(srcdir)/Misc/python-config.in $(srcdir)/Misc/python-config.sh @ # Substitution happens here, as the completely-expanded BINDIR @ # is not available in configure sed -e "s,@EXENAME@,$(BINDIR)/python$(LDVERSION)$(EXE)," < $(srcdir)/Misc/python-config.in >python-config.py diff --git a/Misc/NEWS.d/next/Build/2019-04-25-01-51-52.bpo-21536.ACQkiC.rst b/Misc/NEWS.d/next/Build/2019-04-25-01-51-52.bpo-21536.ACQkiC.rst new file mode 100644 index 000000000000..5e1e717b1ea8 --- /dev/null +++ b/Misc/NEWS.d/next/Build/2019-04-25-01-51-52.bpo-21536.ACQkiC.rst @@ -0,0 +1,12 @@ +On Unix, C extensions are no longer linked to libpython. + +It is now possible to load a C extension built using a shared library Python +with a statically linked Python. + +When Python is embedded, ``libpython`` must not be loaded with ``RTLD_LOCAL``, +but ``RTLD_GLOBAL`` instead. Previously, using ``RTLD_LOCAL``, it was already +not possible to load C extensions which were not linked to ``libpython``, like +C extensions of the standard library built by the ``*shared*`` section of +``Modules/Setup``. + +distutils, python-config and python-config.py have been modified. diff --git a/Misc/python-config.in b/Misc/python-config.in index 714415222798..31ad55822e55 100644 --- a/Misc/python-config.in +++ b/Misc/python-config.in @@ -47,9 +47,7 @@ for opt in opt_flags: print(' '.join(flags)) elif opt in ('--libs', '--ldflags'): - libs = ['-lpython' + pyver + sys.abiflags] - libs += getvar('LIBS').split() - libs += getvar('SYSLIBS').split() + libs = getvar('LIBS').split() + getvar('SYSLIBS').split() # add the prefix/lib/pythonX.Y/config dir, but only if there is no # shared library in prefix/lib/. if opt == '--ldflags': diff --git a/Misc/python-config.sh.in b/Misc/python-config.sh.in index a3c479ce571f..ac1a467678e4 100644 --- a/Misc/python-config.sh.in +++ b/Misc/python-config.sh.in @@ -41,7 +41,7 @@ LIBM="@LIBM@" LIBC="@LIBC@" SYSLIBS="$LIBM $LIBC" ABIFLAGS="@ABIFLAGS@" -LIBS="-lpython${VERSION}${ABIFLAGS} @LIBS@ $SYSLIBS" +LIBS="@LIBS@ $SYSLIBS" BASECFLAGS="@BASECFLAGS@" LDLIBRARY="@LDLIBRARY@" OPT="@OPT@"