[New-bugs-announce] [issue34814] makesetup: must link C extensions to libpython when compiled in shared mode

[STINNER Victor]

New submission from STINNER Victor <vstinner at redhat.com>:

Python can be compiled in "shared" mode: "./configure --enable-shared", Py_ENABLE_SHARED is defined in pyconfig.h. Most Linux distributions use this configuration.

By default, Python builds most C extensions using setup.py which is based on distutils. The get_libraries() method of Lib/distutils/command/build_ext.py explicity add a dependency to libpythonX.Y if Py_ENABLE_SHARED is defined.

But it is possible to use Modules/Setup to build some C extensions using Makefile rather than setup.py. If "*shared*" is in Modules/Setup, following modules will be compiled as libraries (".so" files on Linux). For example, RHEL and Fedora use this configuration for many C extensions. Problem: C extensions compiled like are not linked to libpython.


Example of the issue on Fedora 28 with Python 2.7:

$ ldd $(python2 -c 'import _struct; print(_struct.__file__)')
	linux-vdso.so.1 (0x00007ffeedf38000)
	libpthread.so.0 => /lib64/libpthread.so.0 (0x00007fb4da876000)
	libc.so.6 => /lib64/libc.so.6 (0x00007fb4da4b7000)
	/lib64/ld-linux-x86-64.so.2 (0x00007fb4daca1000)

=> notice the lack of libpython

Python 3.6 is fine:

$ ldd $(python3 -c 'import _struct; print(_struct.__file__)')
	linux-vdso.so.1 (0x00007ffd493dd000)
	libpython3.6m.so.1.0 => /lib64/libpython3.6m.so.1.0 (0x00007f47b9160000)
	...


Patch used by Fedora to build _struct (and other modules) using Makefile:

https://src.fedoraproject.org/rpms/python2/blob/f27/f/python-2.7.1-config.patch



Another example of patch, to build _contextvars as a shared library:

diff --git a/Modules/Setup b/Modules/Setup
index a0622cc8c6..975aeff70d 100644
--- a/Modules/Setup
+++ b/Modules/Setup
@@ -148,7 +148,7 @@ _symtable symtablemodule.c
 # modules are to be built as shared libraries (see above for more
 # detail; also note that *static* or *disabled* cancels this effect):
 
-#*shared*
+*shared*
 
 # GNU readline.  Unlike previous Python incarnations, GNU readline is
 # now incorporated in an optional module, configured in the Setup file
@@ -166,7 +166,7 @@ _symtable symtablemodule.c
 #array arraymodule.c   # array objects
 #cmath cmathmodule.c _math.c # -lm # complex math library functions
 #math mathmodule.c _math.c # -lm # math library functions, e.g. sin()
-#_contextvars _contextvarsmodule.c  # Context Variables
+_contextvars _contextvarsmodule.c  # Context Variables
 #_struct _struct.c     # binary structure packing/unpacking
 #_weakref _weakref.c   # basic weak reference support
 #_testcapi _testcapimodule.c    # Python C API test module


Attached PR fixes Modules/makesetup to:

* (1) Add a dependency on the Makefile target to libpython: to make sure that the parallel compilation works as expected
* (2) Add a dependency to libpythonX.Y on the compiled shared library (".so" file on Linux)

----------
components: Build
messages: 326486
nosy: vstinner
priority: normal
severity: normal
status: open
title: makesetup: must link C extensions to libpython when compiled in shared mode
versions: Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue34814>
_______________________________________