[lxml-dev] build lxml against libxml2 from $HOME

Hi I've got a small issue with lxml's gcc call when I use libxml2 installed to $HOME. The problem is that the "used" libxml2 is determined by the dynamic linker and defaults to /usr/lib/libxml2.so if I don't set LD_LIBRARY_PATH to point to $HOME/libxml2/lib. Now this is cumbersome and there's a way around: rpath. libxml2's python bindings use the following command for linkage of the .so: gcc -shared .libs/libxml.o .libs/types.o .libs/libxml2-py.o -Wl,--rpath -Wl,/home/andreas/compiling/python/xml/libxml2-2.6.26/.libs -Wl,--rpath -Wl,/home/andreas/libxml2/lib -L/home/andreas/compiling/python/xml/libxml2-2.6.26/.libs ../.libs/libxml2.so -L/usr/lib -Wl,-soname -Wl,libxml2mod.so -o .libs/libxml2mod.so the important path is the -Wl and --rpath stuff. The drawback with rpath is that it needs relinking upon installation, for libxml2's python binding this looks like: gcc -shared .libs/libxml.o .libs/types.o .libs/libxml2-py.o -Wl,--rpath -Wl,/home/andreas/libxml2/lib -L/home/andreas/compiling/python/xml/libxml2-2.6.26/.libs -L/home/andreas/libxml2/lib -lxml2 -L/usr/lib -Wl,-soname -Wl,libxml2mod.so -o .libs/libxml2mod.so So to conclude: I'd be _extremely_ happy if lxml could adapt that and link the etree.so using rpath. Is that possible? Thanks, Andreas -- You get along very well with everyone except animals and people.

Hi Andreas, Andreas Pakulat wrote:
I'd be _extremely_ happy if lxml could adapt that and link the etree.so using rpath. Is that possible?
The easiest way to prove that it's possible would be a patch against setup.py that implements it. However, given the fact that this is a rather specific requirement that will not touch some 96% of lxml's users, I don't think we will change the now-working-for-linux+windows+macos build process to integrate it. I don't think there's anything wrong with LD_LIBRARY_PATH for this kind of requirement. Note that you can always build etree.so statically against the library of choice and then use setuptools to distinguish between them from Python. Stefan

On 11.06.06 09:37:18, Stefan Behnel wrote:
I had a quick try yesterday, but I wasn't really sucessful. Maybe I'll try again later. Would you maybe include a patch if using rpath is an option (turned off by default)? Else I'm not really that eager to find a way ;-)
I don't think there's anything wrong with LD_LIBRARY_PATH for this kind of requirement.
Well, rpath is just more convenient.
Note that you can always build etree.so statically against the library of choice and then use setuptools to distinguish between them from Python.
I don't want that, even though libxml2 is not really large. Andreas -- Cold hands, no gloves.

On 11.06.06 09:37:18, Stefan Behnel wrote:
That one was rather easy, because distutils have an option for this. The patch adds an option --rpath, which will "inspect" the arguments returned from xslt-config --libs and provide a different Extension instance. It uses the runtime_library_dirs option to Extension objects and sets this to the list of all directories that do not start with /usr. Andreas -- Expect a letter from a friend who will ask a favor of you.

On 12.06.06 00:04:08, Andreas Pakulat wrote:
Stefan, any chance of getting this into lxml? I mean I'm fine with applying it here locally but maybe others find this useful. And it doesn't change the current behaviour if setup.py at all... Andreas -- In the stairway of life, you'd best take the elevator.

On 22.06.06 21:44:40, Stefan Behnel wrote:
Well, it didn't because flags() didn't return a list anymore but a plain string. I changed this back to a list and now it works perfectly. Attached patch is against 1.0 branch, I don't have trunk around ATM (Note: your patch didn't apply against 1.0 setup.py) Andreas -- There is a fly on your nose.

Hi Andreas, Andreas Pakulat wrote:
What? Not on my machine. It still ends with .split() which (for strings) is guaranteed to return a list of substrings.
I changed this back to a list and now it works perfectly.
Hmmm, could you please check again that the original patch had the above problem? All I changed in flags() was this: - return rf.read().strip().split(' ') + return rf.read().split() The second line is more general than the original line and according to the Python docs, it does what is intended here.
Note: your patch didn't apply against 1.0 setup.py
I guess that was the problem then. Anyway, if it works, it should also work with the above change. I'll apply it to the trunk for now and see if I can get it into 1.0.2 also. Stefan

On 23.06.06 13:36:32, Stefan Behnel wrote:
Dooh, my mistake. I had a too quick look at that change. As I said the patch didn't apply against 1.0 branch out of the box, so I applied it "by hand" and I mixed strip and split on that line. So I had return rf.read().strip() and not rf.read().split(). Sorry for the confustion. Your patch works perfect then also on 1.0 branch. Here's what I get when trying your patch against 1.0 branch: andreas@morpheus:~/compiling/python/xml/lxml-1.0>patch --dry-run -p0 <../lxml_official_rpath.patch patching file setup.py Hunk #2 FAILED at 23. Hunk #3 succeeded at 91 (offset -6 lines). Hunk #4 succeeded at 105 (offset -6 lines). Hunk #5 succeeded at 126 (offset -6 lines). 1 out of 5 hunks FAILED -- saving rejects to file setup.py.rej HTH. Andreas -- Your depth of comprehension may tend to make you lax in worldly ways.

Hi Andreas, Andreas Pakulat wrote:
I'd be _extremely_ happy if lxml could adapt that and link the etree.so using rpath. Is that possible?
The easiest way to prove that it's possible would be a patch against setup.py that implements it. However, given the fact that this is a rather specific requirement that will not touch some 96% of lxml's users, I don't think we will change the now-working-for-linux+windows+macos build process to integrate it. I don't think there's anything wrong with LD_LIBRARY_PATH for this kind of requirement. Note that you can always build etree.so statically against the library of choice and then use setuptools to distinguish between them from Python. Stefan

On 11.06.06 09:37:18, Stefan Behnel wrote:
I had a quick try yesterday, but I wasn't really sucessful. Maybe I'll try again later. Would you maybe include a patch if using rpath is an option (turned off by default)? Else I'm not really that eager to find a way ;-)
I don't think there's anything wrong with LD_LIBRARY_PATH for this kind of requirement.
Well, rpath is just more convenient.
Note that you can always build etree.so statically against the library of choice and then use setuptools to distinguish between them from Python.
I don't want that, even though libxml2 is not really large. Andreas -- Cold hands, no gloves.

On 11.06.06 09:37:18, Stefan Behnel wrote:
That one was rather easy, because distutils have an option for this. The patch adds an option --rpath, which will "inspect" the arguments returned from xslt-config --libs and provide a different Extension instance. It uses the runtime_library_dirs option to Extension objects and sets this to the list of all directories that do not start with /usr. Andreas -- Expect a letter from a friend who will ask a favor of you.

On 12.06.06 00:04:08, Andreas Pakulat wrote:
Stefan, any chance of getting this into lxml? I mean I'm fine with applying it here locally but maybe others find this useful. And it doesn't change the current behaviour if setup.py at all... Andreas -- In the stairway of life, you'd best take the elevator.

On 22.06.06 21:44:40, Stefan Behnel wrote:
Well, it didn't because flags() didn't return a list anymore but a plain string. I changed this back to a list and now it works perfectly. Attached patch is against 1.0 branch, I don't have trunk around ATM (Note: your patch didn't apply against 1.0 setup.py) Andreas -- There is a fly on your nose.

Hi Andreas, Andreas Pakulat wrote:
What? Not on my machine. It still ends with .split() which (for strings) is guaranteed to return a list of substrings.
I changed this back to a list and now it works perfectly.
Hmmm, could you please check again that the original patch had the above problem? All I changed in flags() was this: - return rf.read().strip().split(' ') + return rf.read().split() The second line is more general than the original line and according to the Python docs, it does what is intended here.
Note: your patch didn't apply against 1.0 setup.py
I guess that was the problem then. Anyway, if it works, it should also work with the above change. I'll apply it to the trunk for now and see if I can get it into 1.0.2 also. Stefan

On 23.06.06 13:36:32, Stefan Behnel wrote:
Dooh, my mistake. I had a too quick look at that change. As I said the patch didn't apply against 1.0 branch out of the box, so I applied it "by hand" and I mixed strip and split on that line. So I had return rf.read().strip() and not rf.read().split(). Sorry for the confustion. Your patch works perfect then also on 1.0 branch. Here's what I get when trying your patch against 1.0 branch: andreas@morpheus:~/compiling/python/xml/lxml-1.0>patch --dry-run -p0 <../lxml_official_rpath.patch patching file setup.py Hunk #2 FAILED at 23. Hunk #3 succeeded at 91 (offset -6 lines). Hunk #4 succeeded at 105 (offset -6 lines). Hunk #5 succeeded at 126 (offset -6 lines). 1 out of 5 hunks FAILED -- saving rejects to file setup.py.rej HTH. Andreas -- Your depth of comprehension may tend to make you lax in worldly ways.
participants (2)
-
Andreas Pakulat
-
Stefan Behnel