[lxml-dev] install lxml 2.0.5 on Mac OS X Leopard - why is it so hard?
I know this has been discussed over and over but I'm writing to see if anyone has made a breakthrough yet. The problem of course is that Leopard's builtin libxml2 and libxslt are too old for lxml 2.0. You have to build libxml2 either from source or use a port. There is currently a problem with the libxml2 port, but the workaround is going fine for me: http://trac.macports.org/ticket/15230 (I know because postgres built just fine and I have some tests exerising psycopg2 as well) So after updating my libxml2 to 2.6.31 and libxslt to 1.1.23 and updating my $PATH so that the new xml2-config and xslt-config can be found, I can build lxml *without errors* but I see these warnings: $ sudo easy_install lxml-2.0.5.tgz Processing lxml-2.0.5.tgz Running lxml-2.0.5/setup.py -q bdist_egg --dist-dir /tmp/easy_install-3azY8e/lxml-2.0.5/egg-dist-tmp-t80esG Building lxml version 2.0.5. NOTE: Trying to build without Cython, pre-generated 'src/lxml/etree.c' needs to be available. Using build configuration of libxslt 1.1.23 ld: warning in /opt/local/lib/libxslt.dylib, file is not of required architecture ld: warning in /opt/local/lib/libexslt.dylib, file is not of required architecture ld: warning in /opt/local/lib/libxml2.dylib, file is not of required architecture [... and more like this ...] ... Finished processing dependencies for lxml==2.0.5 What doesn't make sense is these files seem fine to me: $ file -L /opt/local/lib/libxslt.dylib /opt/local/lib/libxslt.dylib: Mach-O dynamically linked shared library i386 $ file -L /opt/local/lib/libexslt.dylib /opt/local/lib/libexslt.dylib: Mach-O dynamically linked shared library i386 I was having similar trouble like this on Tiger and I had test cases in my own test suite that would consistently segfault. On Leopard, those same test cases were *not* segfaulting but now I have some different test cases that are consistently segfaulting. The segfault looks like this in the crash log: Exception Type: EXC_BAD_ACCESS (SIGBUS) Exception Codes: KERN_PROTECTION_FAILURE at 0x0000000000000008 Crashed Thread: 0 Thread 0 Crashed: 0 libxml2.2.dylib 0x90d39419 xmlDictLookup + 360 1 libxml2.2.dylib 0x025626e4 xmlXPathCompExprAdd + 212 2 libxml2.2.dylib 0x025709c6 xmlXPathCompPathExpr + 3910 [etc....] Setting my dyld path (like suggested in the docs, export DYLD_LIBRARY_PATH=/opt/local/lib:/usr/lib) *does* make my test cases run without segfault so I'm assuming what's happening is lxml is using the older dylibs at runtime. This is a really lame way to fix the problem! Specifically, my svn binaries do not like this dylib setting, producing errors like: $ svn ls dyld: lazy symbol binding failed: Symbol not found: _iconv_open Referenced from: /usr/lib/libaprutil-1.0.dylib Expected in: /opt/local/lib/libiconv.2.dylib [etc] (This is slightly odd since I included /usr/lib but whatever.) *sigh* Next, I tried doing a static build of lxml by setting STATIC_LIBRARY_DIRS = ['/opt/local/lib'] in setup.py and running: python setup.py bdist_egg --static --with-xml2-config=/opt/local/bin/xml2-config --with-xslt-config=/opt/local/bin/xslt-config I had to fiddle with gcc to get this to build but otherwise it built fine and installed ok but I did not see any difference. Still consistent segfaults that are fixed by setting the dyld path. Now I'm out of ideas. Does anyone have another suggestion? Until then I have a stupid bash file that I have to source anytime I want to work on lxml. -Kumar
Hi Kumar, you ask why this is so hard? Simple answer: because no-one has contributed a way so far to make it easier. We had lots of reports about stuff not working and almost as many work-arounds, but no-one came up with a patch that would allow building lxml reliably at least on a subset of Mac-OS systems. And I just cannot believe that there is no-one amongst the Mac-OS-X users who knows how to use distutils to build a binary extension. Or at least someone who knows how to build C code statically against a C library.
From my POV, Mac-OS seems to lack three things that make this problem non-trivial. It doesn't have a standard package management system. Neither does it have something like the Linux Standard Base, which dictates where newly installed things belong. And it doesn't seem to support "rpath", which would allow a binary to say "I know where my dependencies come from". Or at least distutils don't support that on Mac. So everything I could try here on Linux to make it work better is bound to fail.
Kumar McMillan wrote:
I know this has been discussed over and over but I'm writing to see if anyone has made a breakthrough yet. The problem of course is that Leopard's builtin libxml2 and libxslt are too old for lxml 2.0. You have to build libxml2 either from source or use a port. [lots of important details skipped to keep this at a higer level for now] Next, I tried doing a static build of lxml by setting STATIC_LIBRARY_DIRS = ['/opt/local/lib'] in setup.py and running:
python setup.py bdist_egg --static --with-xml2-config=/opt/local/bin/xml2-config --with-xslt-config=/opt/local/bin/xslt-config
I had to fiddle with gcc to get this to build but otherwise it built fine and installed ok but I did not see any difference. Still consistent segfaults that are fixed by setting the dyld path.
This is because the --static switch was made specifically for static building on Windows, which has even less support for package management or even half-decent software installation. It just doesn't support Mac-OS as no-one ever told me how to support it. If you want this to run, let's make a deal. Here is a patch (against the trunk, but should work with 2.0.x) that lets --static require setting the STATIC_*_DIRS variables only on Windows, which should result in reading the directories from xml2-config/xslt-config if the hard-coded setup is not provided. Given your above example, this should be the right thing to do. Now, please look at the function "libraries()" in setupinfo.py and fix it up for Mac-OS-X (and for whatever sys.platform calls it) to find the correct static libraries in these directories. If you get it to run reliably on your system, just with your above command line, I'll make sure it gets into 2.0.6. Stefan
On Sun, 11 May 2008 09:01:01 +0200 Stefan Behnel <stefan_ml@behnel.de> wrote:
you ask why this is so hard? Simple answer: because no-one has contributed a way so far to make it easier.
Gee, I had no trouble at all doing this last week (the release of Oracle library bits for Intel OS-X means it's now desirable). I installed macports, did a self-update, then installed py25-lxml. It installed python2.5.2 and the versions of libxml2 and libxslt that were in macports as part of the process. Installing cx_Oracle after that was more work.
We had lots of reports about stuff not working and almost as many work-arounds, but no-one came up with a patch that would allow building lxml reliably at least on a subset of Mac-OS systems. And I just cannot believe that there is no-one amongst the Mac-OS-X users who knows how to use distutils to build a binary extension. Or at least someone who knows how to build C code statically against a C library.
I'm sorry, but my experience is that binary distributions make the problems *worse*, not better - at least if you require multiple different components to be installed. You have to make sure the components all agree about the builds of any libraries they have in common, and unless you have a coordinated build, that just doesn't happen. After all, I could build a binary distribution of lxml from macports, but to use it, you'd have to have the macports versions of python, libxml2 and libxslt. If you've got that, it's probably easier to install the macports version than it is to download and install whatever I might build. I could use ports to build a binary package with all those things in it - is there anyone who really wants that? I started working with lxml last year, when the latest version was 1.3.3. Since updating the software after deployment would be a traumatic operation (a single "instance" of the application uses about 40 cores spread across 10 systems and two SANs, and we typically run three instances), I wanted the latest stable versions of everything. I looked at five different systems, and on only two was getting that combination sane: OSX using macports, and FreeBSD, mostly because they had the latest versions in the ports system when I went to look. The three GNU/Linux systems either had old versions of Python, of the xml libraries, and lxml was either old or missing. So I wound up doing initial development on OSX and FreeBSD while we dealt with the GNU/Linux platform we were speced to run on. Grabbing binaries only half worked. Replacing the installed system tools on GNU/linux is a recipe for disaster, and our sysadmins correctly refused to do so. Which means the LSB is no help at all. The sysadmins found a binary build of python 2.5, and installed that. We then grabbed the lxml rpm from PyPI, and installed that - only it wouldn't run, because it had been built against a version of Python that was compiled with a python shared library, and the version we had hadn't been. I eventually wound up building everything - Python, libxml2, libxslt, lxml and cx_Oracle - by hand to run our installation on, and providing a carefully tailored environment to run things in. Which is what I did on OSX and FreeBSD, except their ports systems makes building from sources trivial, and FreeBSD doesn't need the tailored environment. Updating those two systems is trivial. Updating the GNU/Linux systems is several days worth of work just to get to the point where I have something to give to operations.
From my POV, Mac-OS seems to lack three things that make this problem non-trivial. It doesn't have a standard package management system. Neither does it have something like the Linux Standard Base, which dictates where newly installed things belong. And it doesn't seem to support "rpath", which would allow a binary to say "I know where my dependencies come from". Or at least distutils don't support that on Mac. So everything I could try here on Linux to make it work better is bound to fail.
Providing a binary distribution for *any* system that includes libraries that are "to old" in the base system is going to be a major pain. Everyone runs into this on OSX, because they didn't update those libraries in 10.5 for some unknown reason (I've filed bug ID #5926693 at adc about this). Those of us building for corporate environments - where we always run on out of date platforms, because we can't get corporate approval to use a new one before it becomes out of date - run into it all the time. <mike -- Mike Meyer <mwm@mired.org> http://www.mired.org/consulting.html Independent Network/Unix/Perforce consultant, email for more information. O< ascii ribbon campaign - stop html mail - www.asciiribbon.org
On Sun, May 11, 2008 at 1:48 PM, Mike Meyer <mwm@mired.org> wrote:
On Sun, 11 May 2008 09:01:01 +0200 Stefan Behnel <stefan_ml@behnel.de> wrote:
you ask why this is so hard? Simple answer: because no-one has contributed a way so far to make it easier.
Gee, I had no trouble at all doing this last week (the release of Oracle library bits for Intel OS-X means it's now desirable). I installed macports, did a self-update, then installed py25-lxml. It installed python2.5.2 and the versions of libxml2 and libxslt that were in macports as part of the process.
The build of lxml doesn't fail and you probably won't see any errors unless you are using xpath. In fact, running selftest.py after building passes for me (I'm not sure if that runs all tests or not) but I do get a consistent segfault in my program. Looking at the macport of py25-lxml I don't see any flags that would indicate they have accomplished statically linking the new libxml libs. I don't like to use ports of python modules because /opt/local/bin/python doesn't mix well with a Framework python installation from my experience.
On Sun, 11 May 2008 18:00:26 -0500 "Kumar McMillan" <kumar.mcmillan@gmail.com> wrote:
On Sun, May 11, 2008 at 1:48 PM, Mike Meyer <mwm@mired.org> wrote:
On Sun, 11 May 2008 09:01:01 +0200 Stefan Behnel <stefan_ml@behnel.de> wrote:
you ask why this is so hard? Simple answer: because no-one has contributed a way so far to make it easier.
Gee, I had no trouble at all doing this last week (the release of Oracle library bits for Intel OS-X means it's now desirable). I installed macports, did a self-update, then installed py25-lxml. It installed python2.5.2 and the versions of libxml2 and libxslt that were in macports as part of the process.
The build of lxml doesn't fail and you probably won't see any errors unless you are using xpath. In fact, running selftest.py after building passes for me (I'm not sure if that runs all tests or not) but I do get a consistent segfault in my program.
Well, we make fairly heavy use of xpath (we use it to extract millions of records/minute in our ETL system, plus provide default attributes in the xml config file), so if it's a problem, I'm sure I'll see it. The few tests I've run so far worked fine. Care to provide an example that breaks?
Looking at the macport of py25-lxml I don't see any flags that would indicate they have accomplished statically linking the new libxml libs. I don't like to use ports of python modules because /opt/local/bin/python doesn't mix well with a Framework python installation from my experience.
That's always a problem when you start building your version of languages in the base system - you probably can't use the platform-specific modules that are in the base systems language installation. I can't get to any of the rpm-related python modules on RHEL with my custom python installed. Fortunately, I don't need access to either the rpm libraries or the mac Python frameworks in my applications. <mike -- Mike Meyer <mwm@mired.org> http://www.mired.org/consulting.html Independent Network/Unix/Perforce consultant, email for more information. O< ascii ribbon campaign - stop html mail - www.asciiribbon.org
On Sun, May 11, 2008 at 6:26 PM, Mike Meyer <mwm@mired.org> wrote:
Well, we make fairly heavy use of xpath (we use it to extract millions of records/minute in our ETL system, plus provide default attributes in the xml config file), so if it's a problem, I'm sure I'll see it. The few tests I've run so far worked fine.
huh, yeah it does seem like you'd see a crash. Maybe the py25-lxml port gains some advantages from getting built within the macports environment somehow.
Care to provide an example that breaks?
unfornately, I don't think I have one, not something that is decoupled from the app I'm working on anyway. The app I'm working on makes heavy use of lxml.html to spider through the web, uses xpath() here and there, and the test cases use xpaths for assertions. However, I see the segfault in strange places. For example, if I run all tests at once (I'm using nose) then I usually don't see a segfault. But if I run test cases by themselves I will generally see a segfault. And if I do, it is a consistent segfault. Looking at the crash log I can see that it's on an xpath lookup (I posted this earlier). However, to make matters worse, the test cases I can trigger segfaults in generally do not seem to touch any of the xpath code :/ Nonetheless, all the workarounds I've mentioned stop the segfaults.
Hi, Mike Meyer wrote:
On Sun, 11 May 2008 09:01:01 +0200 Stefan Behnel <stefan_ml@behnel.de> wrote:
you ask why this is so hard? Simple answer: because no-one has contributed a way so far to make it easier.
Gee, I had no trouble at all doing this last week (the release of Oracle library bits for Intel OS-X means it's now desirable). I installed macports, did a self-update, then installed py25-lxml. It installed python2.5.2 and the versions of libxml2 and libxslt that were in macports as part of the process. Installing cx_Oracle after that was more work.
We had lots of reports about stuff not working and almost as many work-arounds, but no-one came up with a patch that would allow building lxml reliably at least on a subset of Mac-OS systems. And I just cannot believe that there is no-one amongst the Mac-OS-X users who knows how to use distutils to build a binary extension. Or at least someone who knows how to build C code statically against a C library.
I'm sorry, but my experience is that binary distributions make the problems *worse*, not better
I wasn't talking about distributing binaries. I meant: someone has to provide a way to configure the compiler so that it builds lxml statically on Mac-OS. Stefan
Stefan, thanks for all the info On Sun, May 11, 2008 at 2:01 AM, Stefan Behnel <stefan_ml@behnel.de> wrote:
From my POV, Mac-OS seems to lack three things that make this problem non-trivial. It doesn't have a standard package management system. Neither does it have something like the Linux Standard Base, which dictates where newly installed things belong. And it doesn't seem to support "rpath", which would allow a binary to say "I know where my dependencies come from". Or at least distutils don't support that on Mac. So everything I could try here on Linux to make it work better is bound to fail.
From more googling it does appear however that setting this var might confuse some apps that do rely on two-level namespacing. So far *my*
I don't have experience building native OS X applications but I've done a little more research into the problem and I think it is specifically this: "/usr/lib/libxml2.2.dylib uses two-level namespace, meaning that the Foundation framework will always use this one instead of yours" -- from http://0xced.blogspot.com/2006/07/dealing-with-outdated-open-source-libs.htm... What is two-level namespacing? Good question. I haven't quite figured that out yet but as the blog post suggests, you can "flatten" it at runtime by setting DYLD_FORCE_FLAT_NAMESPACE=1 And, by golly, this actually works -- that is, setting it in my shell and running my test cases that would otherwise segfault run smoothly. Also, this doesn't screw up my lib paths like setting DYLD_LIBRARY_PATH does (the conflict with subversion went away!). problems have gone away (well, besides this being a kludge) but I guess I'll have to keep an eye on it. The dyld manual didn't help me understand this any better: http://developer.apple.com/documentation/Darwin/Reference/ManPages/man1/dyld...
If you want this to run, let's make a deal. Here is a patch (against the trunk, but should work with 2.0.x) that lets --static require setting the STATIC_*_DIRS variables only on Windows, which should result in reading the directories from xml2-config/xslt-config if the hard-coded setup is not provided. Given your above example, this should be the right thing to do. Now, please look at the function "libraries()" in setupinfo.py and fix it up for Mac-OS-X (and for whatever sys.platform calls it) to find the correct static libraries in these directories. If you get it to run reliably on your system, just with your above command line, I'll make sure it gets into 2.0.6.
I'm willing to do whatever I can to contribute a better Mac OS X build process for lxml. However, I'm not experienced with using ext_modules in python and am having a hard time following your suggestions. You say your patch removed the enforcement of STATIC_*_DIRS but that was never a problem. in fact, that seems to confuse gcc when building with --static since it produces orphaned -I args (no directory attached) Next, you suggest to adjust the sys.platform checks. sys.platform always equals "darwin" on OS X but where would I want to make adjustments? I don't understand what this is doing in libraries() : if sys.platform in ('win32',): libs = ['%s_a' % lib for lib in libs] if I add "darwin" to the list, I get the error: ld: library not found for -lxslt_a whereas -lxslt is the correct arg (just like on linux). In my /opt/local/lib dir I have libxslt.dylib, libxslt.la, libexslt.dylib, libexslt.la, etc. I tried changing the above list comprehension to generate .la names but that didn't work either (still said library not found). I'm still not clear on how to statically link the libxml libraries and that's the first step to solving the problem. If anyone has done this, please let me know and I'll have another go at it. Maybe I need to use libtool to produce static versions then link to those. More googling suggests it *is* possible ;) http://lists.apple.com/archives/Unix-porting/2006/Aug/msg00012.html gcc man pages are not helping me. -Kumar
Hi, Kumar McMillan wrote:
I don't have experience building native OS X applications
See? That seems to be a general problem amongst Mac-OS users. If no-one using that platform knows how to build a C program, how am I supposed to know it?
What is two-level namespacing?
*shrug*, I prefer an automatic static build on Mac-OS anyway.
You say your patch removed the enforcement of STATIC_*_DIRS but that was never a problem.
It was, as it requires manual interaction by users that should only be required in stupid "who-needs-a-system-compiler-anyway" environments like Windows.
in fact, that seems to confuse gcc when building with --static since it produces orphaned -I args (no directory attached)
It just disables the requirement for setting the variables. It doesn't configure anything so far. The config has to come from xml2-config and xslt-config.
Next, you suggest to adjust the sys.platform checks. sys.platform always equals "darwin" on OS X
Ok, then the function will likely look something like this: def libraries(): if sys.platform in ('win32', 'darwin'): libs = ['libxslt', 'libexslt', 'libxml2', 'iconv'] else: libs = ['xslt', 'exslt', 'xml2', 'z', 'm'] if OPTION_STATIC: if sys.platform in ('win32',): libs = ['%s_a' % lib for lib in libs] elif sys.platform in ('darwin',): libs = ['%s.a' % lib for lib in libs] if sys.platform in ('win32',): libs.extend(['zlib', 'WS2_32']) return libs Minus some changes for libiconv and libz.
but where would I want to make adjustments? I don't understand what this is doing in libraries() :
if sys.platform in ('win32',): libs = ['%s_a' % lib for lib in libs]
if I add "darwin" to the list, I get the error: ld: library not found for -lxslt_a
The static libraries are called xxx_a in Windows. If someone can figure out what they are called on Mac-OS, I can fill it in myself.
whereas -lxslt is the correct arg (just like on linux). In my /opt/local/lib dir I have libxslt.dylib, libxslt.la, libexslt.dylib, libexslt.la, etc. I tried changing the above list comprehension to generate .la names but that didn't work either (still said library not found).
Hmmm, on Linux, the static libraries are called "libxml2.a" etc. Can you find anything like that on your system? Stefan
On Mon, May 12, 2008 at 4:04 AM, Stefan Behnel <stefan_ml@behnel.de> wrote:
Kumar McMillan wrote:
I don't have experience building native OS X applications
See? That seems to be a general problem amongst Mac-OS users.
Most people "use" computers they don't build them. Your work is greatly appreciated! :)
What is two-level namespacing?
*shrug*, I prefer an automatic static build on Mac-OS anyway.
me too, I think that would be the right solution.
in fact, that seems to confuse gcc when building with --static since it produces orphaned -I args (no directory attached)
It just disables the requirement for setting the variables. It doesn't configure anything so far. The config has to come from xml2-config and xslt-config.
something is going wrong then with --static because I get "Python.h not found" errors and the gcc command looked something like this: gcc ... -I -I/path/to/python/headers notice the orphaned -I call where, afaict, STATIC_INCLUDE_DIRS was previously getting inserted. Just a theory.
Hmmm, on Linux, the static libraries are called "libxml2.a" etc. Can you find anything like that on your system?
OK, I dug up some more dirt. The problem with the macport of libxml2 is that it doesn't build static libraries. From the port file itself, I now see: --disable-static, doh! But, yeah, I think if I build my own with --disable-shared and then point to that dir as an include this might work. And I assume I will probably get a libxml2.a file out of that build. But is that a feasible end user solution? That is, I'm not convinced this will make great strides in solving the lxml runtime problem where it uses the wrong version of libxml2 / libxslt. Kumar
Hi, Kumar McMillan wrote:
something is going wrong then with --static because I get "Python.h not found" errors and the gcc command looked something like this:
gcc ... -I -I/path/to/python/headers
That's a bug. Here is a patch. Stefan === setupinfo.py ================================================================== --- setupinfo.py (revision 4206) +++ setupinfo.py (local) @@ -15,8 +15,11 @@ PACKAGE_PATH = "src/lxml/" def env_var(name): - value = os.getenv(name, '') - return value.split(os.pathsep) + value = os.getenv(name) + if value: + return value.split(os.pathsep) + else: + return [] def ext_modules(static_include_dirs, static_library_dirs, static_cflags): if CYTHON_INSTALLED:
On Tue, May 13, 2008 at 12:25 AM, Stefan Behnel <stefan_ml@behnel.de> wrote:
gcc ... -I -I/path/to/python/headers
That's a bug. Here is a patch.
closer! thanks for that fix. That got all the -I includes in order. Next up, I'm pretty sure I need to pass -static to libtool so that it honors the -lxml2.a (without -static, it says xml2.a -- lib not found). My idea for this was: export LDFLAGS='-static' and I got: gcc -arch i386 -arch ppc -isysroot /Developer/SDKs/MacOSX10.4u.sdk -g -bundle -undefined dynamic_lookup -static build/temp.macosx-10.3-i386-2.5/src/lxml/lxml.etree.o -L/Users/kumar/src/lxml-2.0/parts/libxml2/lib -L/Users/kumar/src/lxml-2.0/parts/libxslt/lib -lxslt.a -lexslt.a -lxml2.a -lz.a -lm.a -o build/lib.macosx-10.3-i386-2.5/lxml/etree.so ld_classic: incompatible flag -bundle used (must specify "-dynamic" to be used) so ... how do I stop it from adding -bundle? Ideas for another approach?
Hi, Kumar McMillan wrote:
On Tue, May 13, 2008 at 12:25 AM, Stefan Behnel <stefan_ml@behnel.de> wrote:
gcc ... -I -I/path/to/python/headers
That's a bug. Here is a patch.
closer! thanks for that fix. That got all the -I includes in order.
Next up, I'm pretty sure I need to pass -static to libtool so that it honors the -lxml2.a (without -static, it says xml2.a -- lib not found).
It's not "-lxml2.a" but a plain "/path/to/libxml2.a" as parameter to link it in just like the normal lxml.etree.o object file that was just compiled. Stefan
Hello again On Wed, May 14, 2008 at 1:01 AM, Stefan Behnel <stefan_ml@behnel.de> wrote:
Next up, I'm pretty sure I need to pass -static to libtool so that it honors the -lxml2.a (without -static, it says xml2.a -- lib not found).
It's not "-lxml2.a" but a plain "/path/to/libxml2.a" as parameter to link it in just like the normal lxml.etree.o object file that was just compiled.
when I tried the plain paths it says library cannot be found. But I've discovered that building with -static is a dead end. It seems that Apple all but disallows static linking completely: http://developer.apple.com/qa/qa2001/qa1118.html HOWEVER after blood, sweat, and some tears (kidding) this is *all* I needed, it seems: export CFLAGS="-flat_namespace" ...no static builds libxml2 libs, no buildout recipe. I just set that and ran: python setup.py bdist_egg --with-xml2-config=/opt/local/bin/xml2-config --with-xslt-config=/opt/local/bin/xslt-config which uses the libxml2 and etc. installed by ports. In fact, as long as /opt/local/bin is on my path that should work without having to set paths (i.e. from easy_install). All my tests that were segfaulting are now passing. This appears to be the exact same behavior I got by setting DYLD_FORCE_FLAT_NAMESPACE at runtime but without the side affect of applying itself to anything else running in my shell ;) so, I'm thinking this is just two lines of code added to cflags() ... if sys.platform in ('darwin',): result.append('-flat_namespace') Do you want a patch that also includes the adjustments to --static when not windows? I don't think they are necessary anymore. Actually, using --static on darwin should probably raise an error "Apple says no" ;) -Kumar
Hi Kumar, Kumar McMillan wrote:
so, I'm thinking this is just two lines of code added to cflags() ...
if sys.platform in ('darwin',): result.append('-flat_namespace')
That's cool, thanks. I added it to the trunk and to the 2.0 branch. Let's see if Mac users get along with 2.0.6 then... Thanks for the effort! Stefan
On Thu, May 15, 2008 at 6:03 AM, Stefan Behnel <stefan_ml@behnel.de> wrote:
if sys.platform in ('darwin',): result.append('-flat_namespace')
That's cool, thanks. I added it to the trunk and to the 2.0 branch.
excellent
Let's see if Mac users get along with 2.0.6 then...
Thanks for the effort!
sure, no problem. I researched this a bit more. It seems that people generally consider -flat_namespace a bad "hack," something to keep in mind. However, this seems to be because a few libraries take advantage of -twolevel_namespace (the default gcc behavior as of OS X 10.3 or something) so your binaries may cause other linked libs to behave wrong. The only specific example I could find of one that uses two level namespaces was OpenGL, but maybe there are others. Anyway, for lxml's purposes *I think* it is OK to use -flat_namespace since there aren't many other libs involved. Let's roll with it. This is what etree links to : $ otool -l path/to/lxml/etree.so [snip] Load command 7 cmd LC_LOAD_DYLIB cmdsize 56 name /opt/local/lib/libxslt.1.dylib (offset 24) time stamp 2 Wed Dec 31 18:00:02 1969 current version 3.23.0 compatibility version 3.0.0 Load command 8 cmd LC_LOAD_DYLIB cmdsize 56 name /opt/local/lib/libexslt.0.dylib (offset 24) time stamp 2 Wed Dec 31 18:00:02 1969 current version 9.13.0 compatibility version 9.0.0 Load command 9 cmd LC_LOAD_DYLIB cmdsize 56 name /opt/local/lib/libxml2.2.dylib (offset 24) time stamp 2 Wed Dec 31 18:00:02 1969 current version 9.32.0 compatibility version 9.0.0 Load command 10 cmd LC_LOAD_DYLIB cmdsize 52 name /opt/local/lib/libz.1.dylib (offset 24) time stamp 2 Wed Dec 31 18:00:02 1969 current version 1.2.3 compatibility version 1.0.0 Load command 11 cmd LC_LOAD_DYLIB cmdsize 52 name /usr/lib/libSystem.B.dylib (offset 24) time stamp 2 Wed Dec 31 18:00:02 1969 current version 88.3.6 compatibility version 1.0.0 ... so unless libSystem.B.dylib somehow would be tripped up by -flat_namespace I think all should be good. BTW, when I add those two lines all tests pass for me (they passed before but, hey, still a good sign) : Index: setupinfo.py =================================================================== --- setupinfo.py (revision 54771) +++ setupinfo.py (working copy) @@ -136,6 +136,8 @@ for possible_cflag in possible_cflags: if not possible_cflag.startswith('-I'): result.append(possible_cflag) + if sys.platform in ('darwin',): + result.append('-flat_namespace') return result def define_macros(): -Kumar
participants (3)
-
Kumar McMillan -
Mike Meyer -
Stefan Behnel