Autotools integration / How to determine .so filename extension
Hi, I am currently using CPython 2 extensions and code in a project that's mostly written in C. I've been moving the extensions from the CPython API to PyPy/CFFI; but now I'm struggling with integrating the CFFI extensions into the Autotools build process. With CPython, there is the autoconf macro `AM_PATH_PYTHON' for determining the interpreter path and module directories[0]. Is there anything similar for PyPy? In order to build and install the extension modules correctly, I need to find out the appropriate (1) .so filename extension, (2) compiler flags, (3) linker flags, and (4) installation directory. Finding out the flags works the same as it does in CPython 3,
from distutils.ccompiler import new_compiler from distutils.sysconfig import customize_compiler c = new_compiler() customize_compiler(c) c.compiler_so[1:] ['-pthread', '-DNDEBUG', '-O2', '-fPIC'] c.linker_so[1:] ['-pthread', '-shared']
but finding out the .so filename
import _imp _imp.extension_suffixes()[0] '.cpython-37m-x86_64-linux-gnu.so'
doesn't work with PyPy:
import _imp Traceback (most recent call last): File "<stdin>", line 1, in <module> ImportError: No module named _imp
How do I determine the correct filename extension for PyPy extension modules? Roland [0] https://www.gnu.org/software/automake/manual/html_node/Python.html
On Sat, Jul 10, 2021 at 7:18 AM Roland Lutz <rlutz@hedmen.org> wrote:
Hi,
Hi. I am currently using CPython 2 extensions and code in a project that's
mostly written in C. I've been moving the extensions from the CPython API to PyPy/CFFI;
CFFI is probably good, but did you evaluate HPY? In order to build and install the extension modules correctly, I need to
find out the appropriate
(1) .so filename extension, (2) compiler flags, (3) linker flags, and (4) installation directory.
Does libtool handle some of this for you?
On Sat, 10 Jul 2021, Dan Stromberg wrote:
On Sat, Jul 10, 2021 at 7:18 AM Roland Lutz <rlutz@hedmen.org> wrote: I've been moving the extensions from the CPython API to PyPy/CFFI
CFFI is probably good, but did you evaluate HPY?
HPy looks like it may be exactly what I've been looking for, but unfortunately, according to the website, "is still in an early stage of development and there is no official release yet"[0]. [0] https://hpyproject.org/
On 10/7/21 4:44 pm, Roland Lutz wrote:
How do I determine the correct filename extension for PyPy extension modules?
The canonical way is to use sysconfig pypy -c "import sysconfig;print(sysconfig.get_config_var('EXT_SUFFIX'))" .pypy37-pp73-x86_64-linux-gnu.so On python2.7 use 'SO' ~/oss/pypy2.7-HEAD/bin/pypy -c "import sysconfig;print(sysconfig.get_config_var('SO'))" .pypy-73.so If you wish to cross-compile or do not want to run python to find the values, pypy7.3.5+ on linux now ship a CPython compatible file with all the values pre-computed. The file is <pypy-directory>/lib_pypy/_sysconfigdata__linux*.py Matti
On Sat, 10 Jul 2021, Matti Picus wrote:
The canonical way is to use sysconfig
pypy -c "import sysconfig;print(sysconfig.get_config_var('EXT_SUFFIX'))"
.pypy37-pp73-x86_64-linux-gnu.so
On python2.7 use 'SO'
~/oss/pypy2.7-HEAD/bin/pypy -c "import sysconfig;print(sysconfig.get_config_var('SO'))"
.pypy-73.so
This is exactly what I have been looking for. Thenk you very much! In case anyone is interested, here is my resulting autoconf macro: https://gist.github.com/rlutz/334b995462e8d52ae2ae102334b45170 This is an extended version of the standard `AM_PATH_PYTHON' macro. It takes two arguments: a minimum Python 2 version, and a minimum Python 3 version. AM_PATH_PYTHON_EXT([2.7], [3.5]) In case your package supports either only Python 2 or only Python 3, leave the other argument blank. AM_PATH_PYTHON_EXT([2.7], []) AM_PATH_PYTHON_EXT([], [3.5]) This macro chooses the newest available version of PyPy matching the arguments, falling back to CPython if PyPy isn't available. It defines the following output values: PYTHON - the name of the Python executable PYTHON_VERSION - the Python version in the form MAJOR.MINOR PYTHON_PLATFORM - the value of sys.platform PYTHON_EXT_SUFFIX - filename extension for extension modules PYTHON_CFLAGS - CFLAGS for compiling extension modules PYTHON_LDFLAGS - LDFLAGS for compiling extension modules PYTHON_PREFIX, PYTHON_EXEC_PREFIX - the correct prefixes pythondir, pkgpythondir - where to install pure-Python packages pyexecdir, pkgpyexecdir - where to install packages that contain extension modules The macro can be used as a drop-in replacement for AM_PATH_PYTHON (but beware of the changed arguments). If you are using Autotools for building Python extensions, you are probably also calling PKG_CHECK_MODULES for obtaining the PYTHON_CFLAGS; this call isn't necessary any more and should be removed. Roland
participants (3)
-
Dan Stromberg -
Matti Picus -
Roland Lutz