How to force export of a particular symbol from python.exe?
Hi all,
I'm having some trouble making some bits of the Python core code available to extension modules. Specifically, I'm trying to add a function 'Py_force_to_memory' to Python/pymath.c and then use it (via a macro) from Modules/cmathmodule.c. But importing of the cmath module fails with a 'Symbol not found' error. The function is declared with a 'PyAPI_FUNC' in Python/pymath.h.
Here's the relevant portion of the make output:
*** WARNING: renaming "cmath" since importing it failed: dlopen(build/lib.macosx-10.3-i386-2.7/cmath.so, 2): Symbol not found: _Py_force_to_memory Referenced from: /Users/dickinsm/python_source/branches/trunk/build/lib.macosx-10.3-i386-2.7/cmath.so Expected in: dynamic lookup
This is a non-debug trunk build, on OS X (10.5.5), with all the defaults. I'm using Apple's standard toolchain (gcc 4.0.1, Darwin linker). The patch I'm building with can be seen at:
http://bugs.python.org/issue4575
(It's the first of the two patches there, called 'force_to_memory.patch'.)
I think I understand the cause of this problem; I just don't know how to fix it. The cause seems to be that none of the symbols in pymath.o is used in the Python executable; they're used only in the extension modules. So while the '_Py_force_to_memory' symbol appears in libpython2.7.a, it doesn't appear in the python.exe executable; hence the above error.
If I move the definition of Py_force_to_memory from Python/pymath.c to Objects/floatobject.c then everything works as expected.
Questions:
(1) Is this an OS X only problem?
(2) Is there an easy way to force a particular symbol (or all the symbols from a particular object file) to be exported in the Python executable, so that it's available to a dynamically loaded extension module?
I've found the -u option to gcc, but this doesn't seem like a particularly portable solution. Of course, if this problem exists only on OS X, then the solution doesn't need to be portable.
Thanks,
Mark
(1) Is this an OS X only problem?
Probably not. If nothing of pymath.c is actually needed when linking the python executable, pymath.o will be excluded by the linker.
(2) Is there an easy way to force a particular symbol (or all the symbols from a particular object file) to be exported in the Python executable, so that it's available to a dynamically loaded extension module?
That's not the issue. Had pymath.o been linked into python, it's symbols would have been exported (is that proper use of English tenses?)
To fix this, I see three solutions
1. Explicitly link the module to extensions which are known to require it, e.g. by explicitly adding it to the sources in setup.py. That might cause duplications, but would IMO be the cleanest solution (python.exe has no business in exporting standard math functions, IMO)
2. Explicitly link pymath.o to python.exe, instead of integrating it into libpythonxy.a. If the symbols need to be exposed through python.exe (for whatever reason), this is the clean way to do it.
3. Implicitly force linkage, by adding a dummy symbol to pymath.o which gets referenced from an object known to be linked into the interpreter. This has the least impact on the build process, but is the most hackish approach (IMO).
Regards, Martin
On Sun, Dec 14, 2008 at 9:06 PM, "Martin v. Löwis" martin@v.loewis.de wrote:
That's not the issue. Had pymath.o been linked into python, it's symbols would have been exported (is that proper use of English tenses?)
Sounds right to me.
To fix this, I see three solutions
[...]
Thanks for this; this gives me a clearer idea of how things might be solved.
(python.exe has no business in exporting standard math functions, IMO)
It's a little bit messy: some bits of pymath.c (hypot, and possibly copysign) are needed in the core, but only on platforms whose math libraries haven't caught up with C99. The rest is only (possibly) needed in the math and cmath modules. In fact, on OS X none of pymath.c is needed at all, which results in lots of "ranlib: file: libpython2.7.a(pymath.o) has no symbols" in the build output...
I'll try to find a non-hackish solution.
Mark
It's a little bit messy: some bits of pymath.c (hypot, and possibly copysign) are needed in the core, but only on platforms whose math libraries haven't caught up with C99.
It would be possible to only build the module if it defines any functions; that should be checked in configure.
Alternatively, I believe that autoconf offers a mechanism to have fallback functions in files named like the function; autoconf will then build itself a list of all additional source files. Using that would require to split pymath.c into multiple files.
Regards, Martin
participants (2)
-
"Martin v. Löwis" -
Mark Dickinson