[lxml-dev] build lxml against libxml2 from $HOME
Hi
I've got a small issue with lxml's gcc call when I use libxml2 installed to $HOME. The problem is that the "used" libxml2 is determined by the dynamic linker and defaults to /usr/lib/libxml2.so if I don't set LD_LIBRARY_PATH to point to $HOME/libxml2/lib. Now this is cumbersome and there's a way around: rpath.
libxml2's python bindings use the following command for linkage of the .so: gcc -shared .libs/libxml.o .libs/types.o .libs/libxml2-py.o -Wl,--rpath -Wl,/home/andreas/compiling/python/xml/libxml2-2.6.26/.libs -Wl,--rpath -Wl,/home/andreas/libxml2/lib -L/home/andreas/compiling/python/xml/libxml2-2.6.26/.libs ../.libs/libxml2.so -L/usr/lib -Wl,-soname -Wl,libxml2mod.so -o .libs/libxml2mod.so
the important path is the -Wl and --rpath stuff.
The drawback with rpath is that it needs relinking upon installation, for libxml2's python binding this looks like: gcc -shared .libs/libxml.o .libs/types.o .libs/libxml2-py.o -Wl,--rpath -Wl,/home/andreas/libxml2/lib -L/home/andreas/compiling/python/xml/libxml2-2.6.26/.libs -L/home/andreas/libxml2/lib -lxml2 -L/usr/lib -Wl,-soname -Wl,libxml2mod.so -o .libs/libxml2mod.so
So to conclude: I'd be _extremely_ happy if lxml could adapt that and link the etree.so using rpath. Is that possible?
Thanks, Andreas
Hi Andreas,
Andreas Pakulat wrote:
I'd be _extremely_ happy if lxml could adapt that and link the etree.so using rpath. Is that possible?
The easiest way to prove that it's possible would be a patch against setup.py that implements it. However, given the fact that this is a rather specific requirement that will not touch some 96% of lxml's users, I don't think we will change the now-working-for-linux+windows+macos build process to integrate it.
I don't think there's anything wrong with LD_LIBRARY_PATH for this kind of requirement. Note that you can always build etree.so statically against the library of choice and then use setuptools to distinguish between them from Python.
Stefan
On 11.06.06 09:37:18, Stefan Behnel wrote:
Andreas Pakulat wrote:
I'd be _extremely_ happy if lxml could adapt that and link the etree.so using rpath. Is that possible?
The easiest way to prove that it's possible would be a patch against setup.py that implements it. However, given the fact that this is a rather specific requirement that will not touch some 96% of lxml's users, I don't think we will change the now-working-for-linux+windows+macos build process to integrate it.
I had a quick try yesterday, but I wasn't really sucessful. Maybe I'll try again later. Would you maybe include a patch if using rpath is an option (turned off by default)? Else I'm not really that eager to find a way ;-)
I don't think there's anything wrong with LD_LIBRARY_PATH for this kind of requirement.
Well, rpath is just more convenient.
Note that you can always build etree.so statically against the library of choice and then use setuptools to distinguish between them from Python.
I don't want that, even though libxml2 is not really large.
Andreas
On 11.06.06 09:37:18, Stefan Behnel wrote:
Andreas Pakulat wrote:
I'd be _extremely_ happy if lxml could adapt that and link the etree.so using rpath. Is that possible?
The easiest way to prove that it's possible would be a patch against setup.py that implements it.
That one was rather easy, because distutils have an option for this. The patch adds an option --rpath, which will "inspect" the arguments returned from xslt-config --libs and provide a different Extension instance. It uses the runtime_library_dirs option to Extension objects and sets this to the list of all directories that do not start with /usr.
Andreas
On 12.06.06 00:04:08, Andreas Pakulat wrote:
On 11.06.06 09:37:18, Stefan Behnel wrote:
Andreas Pakulat wrote:
I'd be _extremely_ happy if lxml could adapt that and link the etree.so using rpath. Is that possible?
The easiest way to prove that it's possible would be a patch against setup.py that implements it.
That one was rather easy, because distutils have an option for this. The patch adds an option --rpath, which will "inspect" the arguments returned from xslt-config --libs and provide a different Extension instance. It uses the runtime_library_dirs option to Extension objects and sets this to the list of all directories that do not start with /usr.
Stefan, any chance of getting this into lxml? I mean I'm fine with applying it here locally but maybe others find this useful. And it doesn't change the current behaviour if setup.py at all...
Andreas
Hi Andreas,
Andreas Pakulat wrote:
On 12.06.06 00:04:08, Andreas Pakulat wrote:
On 11.06.06 09:37:18, Stefan Behnel wrote:
Andreas Pakulat wrote:
I'd be _extremely_ happy if lxml could adapt that and link the etree.so using rpath. Is that possible?
The easiest way to prove that it's possible would be a patch against setup.py that implements it.
That one was rather easy, because distutils have an option for this. The patch adds an option --rpath, which will "inspect" the arguments returned from xslt-config --libs and provide a different Extension instance. It uses the runtime_library_dirs option to Extension objects and sets this to the list of all directories that do not start with /usr.
Stefan, any chance of getting this into lxml? I mean I'm fine with applying it here locally but maybe others find this useful. And it doesn't change the current behaviour if setup.py at all...
Here is a somewhat cleaner patch that implements this (against the trunk). Please tell me if it works for you.
Stefan
On 22.06.06 21:44:40, Stefan Behnel wrote:
Andreas Pakulat wrote:
On 12.06.06 00:04:08, Andreas Pakulat wrote:
On 11.06.06 09:37:18, Stefan Behnel wrote:
Andreas Pakulat wrote:
I'd be _extremely_ happy if lxml could adapt that and link the etree.so using rpath. Is that possible?
The easiest way to prove that it's possible would be a patch against setup.py that implements it.
That one was rather easy, because distutils have an option for this. The patch adds an option --rpath, which will "inspect" the arguments returned from xslt-config --libs and provide a different Extension instance. It uses the runtime_library_dirs option to Extension objects and sets this to the list of all directories that do not start with /usr.
Stefan, any chance of getting this into lxml? I mean I'm fine with applying it here locally but maybe others find this useful. And it doesn't change the current behaviour if setup.py at all...
Here is a somewhat cleaner patch that implements this (against the trunk). Please tell me if it works for you.
Well, it didn't because flags() didn't return a list anymore but a plain string. I changed this back to a list and now it works perfectly.
Attached patch is against 1.0 branch, I don't have trunk around ATM (Note: your patch didn't apply against 1.0 setup.py)
Andreas
Hi Andreas,
Andreas Pakulat wrote:
On 22.06.06 21:44:40, Stefan Behnel wrote:
Here is a somewhat cleaner patch that implements this (against the trunk). Please tell me if it works for you.
Well, it didn't because flags() didn't return a list anymore but a plain string.
What? Not on my machine. It still ends with .split() which (for strings) is guaranteed to return a list of substrings.
I changed this back to a list and now it works perfectly.
Hmmm, could you please check again that the original patch had the above problem? All I changed in flags() was this:
- return rf.read().strip().split(' ') + return rf.read().split()
The second line is more general than the original line and according to the Python docs, it does what is intended here.
Note: your patch didn't apply against 1.0 setup.py
I guess that was the problem then.
Anyway, if it works, it should also work with the above change. I'll apply it to the trunk for now and see if I can get it into 1.0.2 also.
Stefan
On 23.06.06 13:36:32, Stefan Behnel wrote:
Andreas Pakulat wrote:
On 22.06.06 21:44:40, Stefan Behnel wrote:
Here is a somewhat cleaner patch that implements this (against the trunk). Please tell me if it works for you.
Well, it didn't because flags() didn't return a list anymore but a plain string.
I changed this back to a list and now it works perfectly.
Hmmm, could you please check again that the original patch had the above problem? All I changed in flags() was this:
- return rf.read().strip().split(' ')
- return rf.read().split()
Dooh, my mistake. I had a too quick look at that change. As I said the patch didn't apply against 1.0 branch out of the box, so I applied it "by hand" and I mixed strip and split on that line. So I had return rf.read().strip() and not rf.read().split().
Sorry for the confustion. Your patch works perfect then also on 1.0 branch. Here's what I get when trying your patch against 1.0 branch: andreas@morpheus:~/compiling/python/xml/lxml-1.0>patch --dry-run -p0 <../lxml_official_rpath.patch patching file setup.py Hunk #2 FAILED at 23. Hunk #3 succeeded at 91 (offset -6 lines). Hunk #4 succeeded at 105 (offset -6 lines). Hunk #5 succeeded at 126 (offset -6 lines). 1 out of 5 hunks FAILED -- saving rejects to file setup.py.rej
HTH.
Andreas
participants (2)
-
Andreas Pakulat -
Stefan Behnel