[lxml-dev] changes to lxml's setup.py

Hi there, I've just checked in a change to lxml that refactors lxml's setup.py to be more hackable. No functionality should've changed, though I'd like people to test the static build option on Windows. The way to set the static paths is somewhat easier now; see the static build section in doc/build.txt for more information. Basically I've split up a lot of stuff that'd grown into setup.py into separate modules and made everything a function instead of top-level code. This should make the code easier to follow and adjust in the future. Basically two new modules have been added: * versioninfo.py gets all kinds of version numbers and last change information * setupinfo.py prepares the building of Pyrex/C extensions. I've also made everything depend in a hard way on setuptools being available on the system - I think supporting old-style distutils is just going to be too much of a distraction to maintain properly. Next I'm investigating what to do with the --rpath option. I believe that currently --rpath is treated rather strangely by lxml's setup.py. --rpath normally expects a list of directory paths pointing to the libraries to link with. This is quite useful if you're linking against libraries in non-standard locations, such as in a buildout. It allows you to create a module that just works without having to set things like LD_LIBRARY_PATH before you try importing it. In case of lxml however, --rpath is ripped out of the list of options that reaches distutils/setuptools and action is taken by manipulating the other configuration parameters instead. This works so I haven't changed it (I hope), but it doesn't do enough. I *think* by adding an option --xslt-config pointing to a custom location for xslt-config we can make things work, though that would be hard to support in a straight buildout, as buildout obviously has no knowledge about passing such options. Food for more thought. Regards, Martijn

Hi Martijn, On 11/22/06, Martijn Faassen <faassen@infrae.com> wrote: <snip>
That dependency on setuptools broke the buildbot. Since it builds Python 2.5 and trunk from scratch before building lxml, should I add an extra step so it pulls in setuptools? -- Sidnei da Silva Enfold Systems http://enfoldsystems.com Fax +1 832 201 8856 Office +1 713 942 2377 Ext 214

Hi Sidnei, Martijn, Sidnei da Silva wrote:
Martijn, first of all: thanks for doing that - t'was a good idea.
That would be helpful, yes. I think the dependency on setuptools is acceptable - although lxml can actually be built with standard distutils, so it's somewhat artificial. One more thing: since you'll have setuptools available anyway - could you consider letting the buildbot build an egg from the lxml sources and copy them somewhere where they are publicly available? That would give us the opportunity to provide recent egg builds for the Windows platform. Given the download numbers on cheeseshop, those are the ones that are most wanted. It would be best to have those compiled against the Py2.5 branch. Thanks, Stefan

Hi Stefan, On 11/22/06, Stefan Behnel <behnel_ml@gkec.informatik.tu-darmstadt.de> wrote:
Yeah, well. The thing is that the buildbot uses a debug build. I can setup a standard build, very likely, but that could take some time. I will keep the idea in mind for the next time I have some free hours. -- Sidnei da Silva Enfold Systems http://enfoldsystems.com Fax +1 832 201 8856 Office +1 713 942 2377 Ext 214

Martijn Faassen wrote:
I think it would then be a good idea to add the ez_setup.py script to SVN. This would allow users that do not have setuptools installed but have an online connection to have setup.py download it for them automatically (and use it from the current directory). Stefan

Hey, To followup, I've done some more refactoring and hope I didn't break anything. I've now made it more explicit that we're setting library dirs and install dirs, instead of just passing everything as cflags and libs. I've also done a lot of thinking about the rpath issue. One first step is of course to make --rpath reach distutils again, and I've fixed that by making the current --rpath option be named --auto-rpath. Now I'm trying to figure out why buildout still doesn't work correctly, and I think I've identified the issue as at least partially in buildout. Passing --include-dirs and --library-dirs to setup.py explicitly certainly has the intented effect after all.. I've sent a message to the buildout people (mainly Jim Fulton) to see whether he has anything to say about this. I think setupinfo.py can be refactored somewhere further beyond this; I'm still not 100% happy with the way static options now override, and I still need to explore what happens when you override with commandline options. It would still be nice to indicate a custom xslt-config as well, but that would require a bit more magic in option parsing than I feel up to right now. Regards, Martijn

Hi Martijn, On 11/22/06, Martijn Faassen <faassen@infrae.com> wrote: <snip>
That dependency on setuptools broke the buildbot. Since it builds Python 2.5 and trunk from scratch before building lxml, should I add an extra step so it pulls in setuptools? -- Sidnei da Silva Enfold Systems http://enfoldsystems.com Fax +1 832 201 8856 Office +1 713 942 2377 Ext 214

Hi Sidnei, Martijn, Sidnei da Silva wrote:
Martijn, first of all: thanks for doing that - t'was a good idea.
That would be helpful, yes. I think the dependency on setuptools is acceptable - although lxml can actually be built with standard distutils, so it's somewhat artificial. One more thing: since you'll have setuptools available anyway - could you consider letting the buildbot build an egg from the lxml sources and copy them somewhere where they are publicly available? That would give us the opportunity to provide recent egg builds for the Windows platform. Given the download numbers on cheeseshop, those are the ones that are most wanted. It would be best to have those compiled against the Py2.5 branch. Thanks, Stefan

Hi Stefan, On 11/22/06, Stefan Behnel <behnel_ml@gkec.informatik.tu-darmstadt.de> wrote:
Yeah, well. The thing is that the buildbot uses a debug build. I can setup a standard build, very likely, but that could take some time. I will keep the idea in mind for the next time I have some free hours. -- Sidnei da Silva Enfold Systems http://enfoldsystems.com Fax +1 832 201 8856 Office +1 713 942 2377 Ext 214

Martijn Faassen wrote:
I think it would then be a good idea to add the ez_setup.py script to SVN. This would allow users that do not have setuptools installed but have an online connection to have setup.py download it for them automatically (and use it from the current directory). Stefan

Hey, To followup, I've done some more refactoring and hope I didn't break anything. I've now made it more explicit that we're setting library dirs and install dirs, instead of just passing everything as cflags and libs. I've also done a lot of thinking about the rpath issue. One first step is of course to make --rpath reach distutils again, and I've fixed that by making the current --rpath option be named --auto-rpath. Now I'm trying to figure out why buildout still doesn't work correctly, and I think I've identified the issue as at least partially in buildout. Passing --include-dirs and --library-dirs to setup.py explicitly certainly has the intented effect after all.. I've sent a message to the buildout people (mainly Jim Fulton) to see whether he has anything to say about this. I think setupinfo.py can be refactored somewhere further beyond this; I'm still not 100% happy with the way static options now override, and I still need to explore what happens when you override with commandline options. It would still be nice to indicate a custom xslt-config as well, but that would require a bit more magic in option parsing than I feel up to right now. Regards, Martijn
participants (3)
-
Martijn Faassen
-
Sidnei da Silva
-
Stefan Behnel