[lxml-dev] a buildout for lxml

Hi there, I've just created what's called a "buildout" for lxml. Buildout is a new system for constructing Python applications and its constituent parts. More information can be found here: http://cheeseshop.python.org/pypi/zc.buildout/ Now before you read all that (it's fairly technical), let me explain what problem the lxml buildout tries to solve. Yesterday I tried using lxml's XMLSchema validator against an XML file and a schema. It gave a validation error, but to the best of my knowledge, the XML file should've been valid. Thinking it might perhaps be a bug that was fixed in more recent versions of libxml2, I set out to build and install libxml2 and libxslt locally, compiled lxml against it, and confirmed it: the document does validate against the schema with the most recent release of libxml2. Now downloading, building and installing libxml2 and libxslt and linking lxml against it is quite a bit of work. Buildout to the rescue! Now it's just this: $ svn co https://infrae.com/svn/buildout/lxml-recent/trunk lxml-recent $ cd lxml-recent $ python bootstrap/bootstrap.py $ bin/buildout and then wait. It'll download, configure and compile libxml2, libxslt, will downoad the lxml source and compile it against the libraries. In the end you end up with a script that invokes the Python interpreter with the right paths, so you can import the new lxml: $ bin/lxmlpython Though you can also manipulate the PYTHONPATH yourself in your scripts to put the lxml development egg on it, and then you can use the regular Python you used to run bootstrap.py with. To see how this works, you can take a look at buildout.cfg. This specifies how things are put together. Note that 'zc.recipe.cmmi' is a 'buildout recipe'. Recipes are components responsible for deployment of particular bits of an application. If zc.recipe.cmmi were released as an egg onto the cheeseshop, buildout would go out and download and install the recipe from there. Since it's not on the cheeseshop yet, I've installed it as a development-egg and included it as an svn:external, but this can be fixed by wrapping that into an egg. Many buildout recipes exist: this one does 'configure; make; make install', hence 'cmmi'. There are also recipes that build and install Zope 3, recipes to build tricky Python modules that don't work as an egg yet, and so on. New recipes can be developed. At Infrae's we've created recipes that can deploy OpenOffice in server mode, so that it can be used through UNO for document conversion purposes. This expanding pool of recipes will allow us to put together extremely complicated applications in a reproducable fashion. The idea of buildout is to have control - to know exactly what is installed and not to have to install everything into site-packages. buildout uses setuptools and easy_install internally, so the whole egg story just works; it expands on that. I think this way of building lxml has more benefits than just my single case. It'll allow one to deploy lxml against a known-good version of libxml2 and libxslt, at least on a linux system. It should also help testing and developing lxml - it becomes a lot more easy to test and develop against different versions of libxml2. I've placed this buildout in infrae.com SVN for now (checkout command above), but if people think this is interesting, I can integrate this into the lxml trunk. This can be done without harming the current way of working, but it'll allow someone to test against any version of libxml2 and libxslt a lot more easily than one can do now. Comments? Regards, Martijn

Hey, I've just expanded on this and added some more explanation on my weblog, here: http://faassen.n--tree.net/blog/view/weblog/2006/10/03/0 Regards, Martijn

On Wed, Oct 04, 2006 at 09:14:38AM +0200, Stefan Behnel wrote: | Martijn Faassen wrote: | > I've just expanded on this and added some more explanation on my weblog, | > here: | > | > http://faassen.n--tree.net/blog/view/weblog/2006/10/03/0 Hey Martijn, Would you like to add the buildout to the buildbot so it gets tested too? -- Sidnei da Silva Enfold Systems http://enfoldsystems.com Fax +1 832 201 8856 Office +1 713 942 2377 Ext 214

Sidnei da Silva wrote:
I've no idea how to do that and don't really have time to learn buildbot, but if someone can do that then it might be a nice way to test a number of versions of lxml against a number of versions of libxml2/libxslt automatically. It'd essentially just take a bunch of different buildout.cfg files with some download URLs and version numbers changed. Regards, Martijn

On Wed, Oct 04, 2006 at 05:33:03PM +0200, Martijn Faassen wrote: | I've no idea how to do that and don't really have time to learn | buildbot, but if someone can do that then it might be a nice way to test | a number of versions of lxml against a number of versions of | libxml2/libxslt automatically. I'm not asking you to do that. I'm asking if you want to see it done. :) | It'd essentially just take a bunch of | different buildout.cfg files with some download URLs and version numbers | changed. Sounds good. I will take a look at that. -- Sidnei da Silva Enfold Systems http://enfoldsystems.com Fax +1 832 201 8856 Office +1 713 942 2377 Ext 214

Hey, I've just expanded on this and added some more explanation on my weblog, here: http://faassen.n--tree.net/blog/view/weblog/2006/10/03/0 Regards, Martijn

On Wed, Oct 04, 2006 at 09:14:38AM +0200, Stefan Behnel wrote: | Martijn Faassen wrote: | > I've just expanded on this and added some more explanation on my weblog, | > here: | > | > http://faassen.n--tree.net/blog/view/weblog/2006/10/03/0 Hey Martijn, Would you like to add the buildout to the buildbot so it gets tested too? -- Sidnei da Silva Enfold Systems http://enfoldsystems.com Fax +1 832 201 8856 Office +1 713 942 2377 Ext 214

Sidnei da Silva wrote:
I've no idea how to do that and don't really have time to learn buildbot, but if someone can do that then it might be a nice way to test a number of versions of lxml against a number of versions of libxml2/libxslt automatically. It'd essentially just take a bunch of different buildout.cfg files with some download URLs and version numbers changed. Regards, Martijn

On Wed, Oct 04, 2006 at 05:33:03PM +0200, Martijn Faassen wrote: | I've no idea how to do that and don't really have time to learn | buildbot, but if someone can do that then it might be a nice way to test | a number of versions of lxml against a number of versions of | libxml2/libxslt automatically. I'm not asking you to do that. I'm asking if you want to see it done. :) | It'd essentially just take a bunch of | different buildout.cfg files with some download URLs and version numbers | changed. Sounds good. I will take a look at that. -- Sidnei da Silva Enfold Systems http://enfoldsystems.com Fax +1 832 201 8856 Office +1 713 942 2377 Ext 214
participants (3)
-
Martijn Faassen
-
Sidnei da Silva
-
Stefan Behnel