Solaris and distutils: Need to pass LIBDIR explicitly with -L when building extensions?
I am trying to build a number of projects that use Python extensions on Solaris 10 and I've discovered that nothing with extensions will link unless I explicitly pass in a '-L/path/to/python/lib/dir' because libpython2.5.so is not otherwise found when the '-lpython2.5' argument is specified during linking. This doesn't seem right as I don't have to specify this explicitly on Linux or Windows. Admittedly I'm using a custom built Python, so perhaps I missed some configure argument? Here's my Python build process: ./configure --prefix=/home/dpeterson/py/build --enable-shared make make test make install It is built with the gcc provided by Sun in a default install of Solaris 10, which means the gcc compiler and the sun linker. "gcc -v" gives: Reading specs from /usr/sfw/lib/gcc/i386-pc-solaris2.10/3.4.3/specs Configured with: /builds/sfw10-gate/usr/src/cmd/gcc/gcc-3.4.3/configure --prefix=/usr/sfw --with-as=/usr/sfw/bin/gas --with-gnu-as --with-ld=/usr/ccs/bin/ld --without-gnu-ld --enable-languages=c,c++ --enable-shared Thread model: posix gcc version 3.4.3 (csl-sol210-3_4-branch+sol_rpath) If that all seems correct, then it appears the issue is the finalize_options() source in lib/distutils/commands/build_ext.py. There are a number of "if" blocks that explicitly append the value of distutils.sysconfig.get_config_vars('LIBDIR') to the list of library_dirs used to link built extensions with. However, there doesn't seem to be one of these for Solaris / sunos. Is this just an oversite? I've verified that adding a simple block with a 'sunos' value gate solves the problem and I no longer need to be explicit about providing '-L'. Am I missing something or should this patch be submitted as a bugfix on the issue tracker? -- Dave
Hi Dave On Mon, Jan 26, 2009 at 06:48:06PM -0600, Dave Peterson wrote:
I am trying to build a number of projects that use Python extensions on Solaris 10 and I've discovered that nothing with extensions will link unless I explicitly pass in a '-L/path/to/python/lib/dir' because libpython2.5.so is not otherwise found when the '-lpython2.5' argument is specified during linking. [...] If that all seems correct, then it appears the issue is the finalize_options() source in lib/distutils/commands/build_ext.py. There are a number of "if" blocks that explicitly append the value of distutils.sysconfig.get_config_vars('LIBDIR') to the list of library_dirs used to link built extensions with. However, there doesn't seem to be one of these for Solaris / sunos.
Could you point to one of the projects you're having trouble with? I haven't had any such problems with Python 2.5 on Solaris 10, distutils always finds the right things when I'm building extension modules. Regards Floris -- Debian GNU/Linux -- The Power of Freedom www.debian.org | www.gnu.org | www.kernel.org
On 2009-01-30 17:08, Floris Bruynooghe wrote:
Hi Dave
On Mon, Jan 26, 2009 at 06:48:06PM -0600, Dave Peterson wrote:
I am trying to build a number of projects that use Python extensions on Solaris 10 and I've discovered that nothing with extensions will link unless I explicitly pass in a '-L/path/to/python/lib/dir' because libpython2.5.so is not otherwise found when the '-lpython2.5' argument is specified during linking. [...] If that all seems correct, then it appears the issue is the finalize_options() source in lib/distutils/commands/build_ext.py. There are a number of "if" blocks that explicitly append the value of distutils.sysconfig.get_config_vars('LIBDIR') to the list of library_dirs used to link built extensions with. However, there doesn't seem to be one of these for Solaris / sunos.
Could you point to one of the projects you're having trouble with? I haven't had any such problems with Python 2.5 on Solaris 10, distutils always finds the right things when I'm building extension modules.
It's not specific to any one project. Mostly, it seems to be tied to using a custom --prefix where $prefix/lib is not on the default library search path. Are you using such a custom --prefix? -- Robert Kern "I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth." -- Umberto Eco
Floris Bruynooghe wrote:
Hi Dave
On Mon, Jan 26, 2009 at 06:48:06PM -0600, Dave Peterson wrote:
I am trying to build a number of projects that use Python extensions on Solaris 10 and I've discovered that nothing with extensions will link unless I explicitly pass in a '-L/path/to/python/lib/dir' because libpython2.5.so is not otherwise found when the '-lpython2.5' argument is specified during linking.
[...]
If that all seems correct, then it appears the issue is the finalize_options() source in lib/distutils/commands/build_ext.py. There are a number of "if" blocks that explicitly append the value of distutils.sysconfig.get_config_vars('LIBDIR') to the list of library_dirs used to link built extensions with. However, there doesn't seem to be one of these for Solaris / sunos.
Could you point to one of the projects you're having trouble with? I haven't had any such problems with Python 2.5 on Solaris 10, distutils always finds the right things when I'm building extension modules.
Pretty much everything I've tried that uses an extension has this problem. Cython, Numpy, Traits, etc. As Robert Kern pointed out, I'm using a custom built Python (Python 2.5.4) built and installed into a custom location via '--prefix'. What Python are you using? Just for grins, I've tried building Python with a number of other compilers, but all with a --prefix setting, and they've all exhibited this problem. I'm leaning more and more toward this is actually a bug with the distutils source on Solaris. -- Dave
On Fri, Jan 30, 2009 at 05:45:26PM -0600, Dave Peterson wrote:
Floris Bruynooghe wrote:
On Mon, Jan 26, 2009 at 06:48:06PM -0600, Dave Peterson wrote:
I am trying to build a number of projects that use Python extensions on Solaris 10 and I've discovered that nothing with extensions will link unless I explicitly pass in a '-L/path/to/python/lib/dir' because libpython2.5.so is not otherwise found when the '-lpython2.5' argument is specified during linking.
[...]
If that all seems correct, then it appears the issue is the finalize_options() source in lib/distutils/commands/build_ext.py. There are a number of "if" blocks that explicitly append the value of distutils.sysconfig.get_config_vars('LIBDIR') to the list of library_dirs used to link built extensions with. However, there doesn't seem to be one of these for Solaris / sunos.
Could you point to one of the projects you're having trouble with? I haven't had any such problems with Python 2.5 on Solaris 10, distutils always finds the right things when I'm building extension modules.
Pretty much everything I've tried that uses an extension has this problem. Cython, Numpy, Traits, etc. As Robert Kern pointed out, I'm using a custom built Python (Python 2.5.4) built and installed into a custom location via '--prefix'. What Python are you using?
Ah, I get it now. I did cut out the important part: you build with --enable-shared. I'm also using a custom build Python with --prefix but don't use --enable-shared so don't get this problem.
I'm leaning more and more toward this is actually a bug with the distutils source on Solaris.
Yes, I agree now that it is a bug in distutils and I agree with your fix. The if statement should check both that it is SunOS and that it is using a shared python, just like the linux one. If fact the linux one could just be modified: if (sys.paltform.startswith('linux') \ or sys.platform.startswith('gnu') \ or sys.platform.startswith('sunos')) \ and sysconfig.get_config_var('Py_ENABLE_SHARED'): ... I'll leave the honours of reporting the bug to you if you agree with this. Lastly out of curiosity: why --enable-shared? Do you embed python? Or are there other reasons to use it? Regards Floris -- Debian GNU/Linux -- The Power of Freedom www.debian.org | www.gnu.org | www.kernel.org
2009/1/31 Floris Bruynooghe <floris.bruynooghe@gmail.com>:
I'm leaning more and more toward this is actually a bug with the distutils source on Solaris.
Yes, I agree now that it is a bug in distutils and I agree with your fix. The if statement should check both that it is SunOS and that it is using a shared python, just like the linux one. If fact the linux one could just be modified:
if (sys.paltform.startswith('linux') \ or sys.platform.startswith('gnu') \ or sys.platform.startswith('sunos')) \ and sysconfig.get_config_var('Py_ENABLE_SHARED'): ...
I'll leave the honours of reporting the bug to you if you agree with this.
Yes thanks to report this, I'll apply your fix, Regards Tarek -- Tarek Ziadé - Directeur Technique INGENIWEB (TM) - SAS 50000 Euros - RC B 438 725 632 Bureaux de la Colline - 1 rue Royale - Bâtiment D - 9ème étage 92210 Saint Cloud - France Phone : 01.78.15.24.00 / Fax : 01 46 02 44 04 http://www.ingeniweb.com - une société du groupe Alter Way
Tarek Ziade wrote:
2009/1/31 Floris Bruynooghe <floris.bruynooghe@gmail.com>:
I'm leaning more and more toward this is actually a bug with the distutils source on Solaris.
Yes, I agree now that it is a bug in distutils and I agree with your fix. The if statement should check both that it is SunOS and that it is using a shared python, just like the linux one. If fact the linux one could just be modified:
if (sys.paltform.startswith('linux') \ or sys.platform.startswith('gnu') \ or sys.platform.startswith('sunos')) \ and sysconfig.get_config_var('Py_ENABLE_SHARED'): ...
I'll leave the honours of reporting the bug to you if you agree with this.
Yes thanks to report this, I'll apply your fix,
Regards Tarek
Hi Tarek, It isn't clear to me if you fixed it already or were waiting for my patch, but I just added issue 5132 to the Python issue tracker to document the issue and provide a patch file. -- Dave
On Mon, Feb 2, 2009 at 7:29 PM, Dave Peterson <dpeterson@enthought.com> wrote:
Hi Tarek,
It isn't clear to me if you fixed it already or were waiting for my patch, but I just added issue 5132 to the Python issue tracker to document the issue and provide a patch file.
I was waiting for that, thanks :) Tarek
-- Dave
_______________________________________________ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
-- Tarek Ziadé | Association AfPy | www.afpy.org Blog FR | http://programmation-python.org Blog EN | http://tarekziade.wordpress.com/
Tarek Ziadé wrote:
On Mon, Feb 2, 2009 at 7:29 PM, Dave Peterson <dpeterson@enthought.com> wrote:
Hi Tarek,
It isn't clear to me if you fixed it already or were waiting for my patch, but I just added issue 5132 to the Python issue tracker to document the issue and provide a patch file.
I was waiting for that, thanks :)
Tarek
I noticed you removed the Python 2.5 version specification. I wasn't aware there were no further planned maintenance releases for the 2.5 branch. A few of the libs we use still seem to have problems building on 2.6. Guess it is just a matter of time. -- Dave
On Mon, Feb 2, 2009 at 8:01 PM, Dave Peterson <dpeterson@enthought.com> wrote:
Tarek Ziadé wrote:
On Mon, Feb 2, 2009 at 7:29 PM, Dave Peterson <dpeterson@enthought.com> wrote:
Hi Tarek,
It isn't clear to me if you fixed it already or were waiting for my patch, but I just added issue 5132 to the Python issue tracker to document the issue and provide a patch file.
I was waiting for that, thanks :)
Tarek
I noticed you removed the Python 2.5 version specification. I wasn't aware there were no further planned maintenance releases for the 2.5 branch. A few of the libs we use still seem to have problems building on 2.6. Guess it is just a matter of time.
Yeah, 2.5 is only security bugfixes now. So it will be available in the next 2.6.x I guess Regards Tarek
-- Dave
-- Tarek Ziadé | Association AfPy | www.afpy.org Blog FR | http://programmation-python.org Blog EN | http://tarekziade.wordpress.com/
On Feb 2, 2009, at 12:01 PM, Dave Peterson wrote:
I noticed you removed the Python 2.5 version specification. I wasn't aware there were no further planned maintenance releases for the 2.5 branch. A few of the libs we use still seem to have problems building on 2.6. Guess it is just a matter of time.
I maintain a few libraries with compiled C code. I don't know how to build binaries for Python 2.6, because apparently the method that I've been using until now to build binary extension modules (mingw) isn't able to produce 2.6-compatible extension modules yet. For now, I'm just not distributing binaries for Python 2.6 (nor, of course, doing a proper job of testing with Python 2.6). I've been kind of hoping that either mingw would catch up or that all my users would give up on Python >= 2.6 and go back to using Python 2.5, but perhaps I should start making new plans... Meanwhile I'm watching with interest this ticket: http:// bugs.python.org/issue3871 Maybe in the future I can just offer my users a Python executable which is compatible with the Free Software tools that I know and use, instead of figuring out how to make modules which are compatible with the proprietary compiler used to build the official Python 2.6! Regards, Zooko P.S. Whoops, sorry. This message was total flame-bait and off-topic to boot. But it has been bugging me, and I feel better for having gotten it off my chest.
zooko wrote:
On Feb 2, 2009, at 12:01 PM, Dave Peterson wrote:
I noticed you removed the Python 2.5 version specification. I wasn't aware there were no further planned maintenance releases for the 2.5 branch. A few of the libs we use still seem to have problems building on 2.6. Guess it is just a matter of time.
I maintain a few libraries with compiled C code. I don't know how to build binaries for Python 2.6, because apparently the method that I've been using until now to build binary extension modules (mingw) isn't able to produce 2.6-compatible extension modules yet.
Building python extensions with mingw on 2.6 is possible, at least on 32 bits arch. We can build numpy with mingw on 2.6; given numpy's reliance on a lot of distutils features/C code, I think most python softwares should build fine. cheers, David
participants (7)
-
Dave Peterson -
David Cournapeau -
Floris Bruynooghe -
Robert Kern -
Tarek Ziade -
Tarek Ziadé -
zooko