Mailman 3 May 2006 - lxml - The Python XML Toolkit

[lxml-dev] Lxml 1.0 will be released on Thursday, June 1st
by Stefan Behnel 18 May '06

18 May '06

Hallo everyone, just to keep us from pushing back release dates, Martijn and I have fixed the release date of lxml 1.0 to June 1st. The (long) list of changes for 1.0 is here: http://codespeak.net/svn/lxml/trunk/CHANGES.txt This means for you: If you want a bug-free release ready to install, then come and help us by testing the trunk and reporting any bugs you can find. http://codespeak.net/svn/lxml/trunk Build instructions are here: http://codespeak.net/lxml/build.html It also really helps us if you read the docs and report any difficulties you encounter. The in-development documentation is here: http://codespeak.net/svn/lxml/trunk/doc/ especially http://codespeak.net/svn/lxml/trunk/doc/api.txt http://codespeak.net/svn/lxml/trunk/doc/resolvers.txt and a little bit of http://codespeak.net/svn/lxml/trunk/doc/extensions.txt Any help is appreciated to make lxml 1.0 the best XML tool for Python - ever. :) Have fun, Stefan

4 14

Re: [lxml-dev] What are the equivalents of nowrite, nomkdir options
by Stefan Behnel 18 May '06

18 May '06

Hi Noah, Noah Slater wrote: >> Noah, I'd be glad if you could test it and report back if it works as >> expected. I do not know if the directory creation business works now >> (i.e. if >> directories /are/ created). > > Hi, sorry I can't test until I have a .deb package to install - so I > am guessing I will have to wait until your changes arrive in Debian > unstable. That won't be before 1.0 then, I assume. It's actually not that hard to build Debian packages once they are part of Debian. Get the source .deb via apt and unpack it. Then you can replace the included lxml sources with the current SVN trunk (you may have to make a tgz from it), make sure the Debian package description has the version number you want and then rebuild it (IIRC, the command for that is "dpkg-rebuild" or something in that line). I don't have a Debian system to test, so I can't tell you what exactly you have to do. But there's an even simpler way. You can check out lxml from SVN and build it in-place in the source directory (see doc/build.txt on how to do that). To use it in your program, you can call Python like this: PYTHONPATH=/path/to/lxml-dir/src python myprogram.py To make sure it's the right version, use PYTHONPATH=/path/to/lxml-dir/src python -c \ 'import lxml.etree ; print lxml.etree.__version__' (the '/' makes it one line when you copy and paste it) Stefan

3 3

[lxml-dev] What are the equivalents of nowrite, nomkdir options
by Noah Slater 17 May '06

17 May '06

Hello all, Firstly thanks for such a great library. For a python module I am worryingly excited reading through the documentation. I have had a play around with etree.* and have almost replace 50% of the code in my project thus far to use this - but one concern remains: I cannot figure out from either the docs, snippets of source files I have read or google how I am meant to prevent XSLT from writing to the file system. Using xsltproc from the command line I can use the "--nowrite" and "--nomkdir" switches and would ideally like to be able to replicate this functionality. Thanks so much, Noah -- "Creativity can be a social contribution, but only in so far as society is free to use the results." - R. Stallman

4 7

[lxml-dev] new "doc/build.txt" on building lxml from sources
by Stefan Behnel 17 May '06

17 May '06

Hi all, I added a "doc/build.txt" apart from the simplified "INSTALL.txt" to help interested users build lxml from sources (and to prevent normal distrib users from caring about it). It also contains David's procedure on building lxml statically on Windows. http://codespeak.net/svn/lxml/trunk/doc/build.txt It's both in the trunk and the 0.9 branch, so, Martijn, please update the web site from the branch. Regards, Stefan

2 1

[lxml-dev] New CSS style for the lxml website @ codespeak
by Olivier Grisel 16 May '06

16 May '06

Hi list, As I found the current style a bit "too big" I refactored the css to get this: http://champiland.homelinux.net/lxml-newstyle/ If you like it I can check that in the svn. I enclose a patch to list the changes. To Stefan: your work on mail.txt has been done only in branch/lxml-0.9.x and not in trunk. Is it intentional? + it would be nice if your changes could went online. Who is in charge of the publishing process? Regards, -- Olivier Index: style.css =================================================================== --- style.css (revision 27205) +++ style.css (working copy) @@ -1,33 +1,62 @@ body { + /* CSS Hack for IE that does not respect the "margin: auto" rule at the + * document level */ + text-align: center; + padding: 1em; +} + +div.document { + width: 45em; + font: 13px Arial, Verdana, Helvetica, sans-serif; + margin: 1em auto 1em auto; + background-color: white; + color: #222; + text-align: left; +} + +h1.title { background: url(http://codespeak.net/img/codespeak1b.png) no-repeat; - font: 120% Arial, Verdana, Helvetica, sans-serif; - border: 0; - margin: 0.5em 0em 0.5em 0.5em; - padding: 0 0 0 145px; + padding: 20px 0 0 180px; + height: 60px; + font-size: 200%; } -a { - text-decoration: underline; - background-color: transparent; +h1, h2, h3 { + color: #333; + font-weight: bold; } -p { - /*margin: 0.5em 0em 1em 0em;*/ - text-align: left; - line-height: 1.5em; - margin: 0.5em 0em 0em 0em; +h1 { + font-size: 120%; } -p a { - text-decoration: underline; +h2 { + font-size: 110%; } +h3 { + font-size: 105%; +} -p a:active { - color: Red; +a, a:visited { background-color: transparent; + font-weight: bold; + color: Black; + text-decoration: none; } +a:active { + color: Red; + text-decoration: underline; +} + +p { + /*margin: 0.5em 0em 1em 0em;*/ + text-align: justify; + line-height: 1.5em; + margin: 0.5em 0em 0em 0em; +} + hr { clear: both; height: 1px; @@ -35,10 +64,8 @@ background-color: transparent; } - -ul { +ul { line-height: 1.5em; - /*list-style-image: url("bullet.gif"); */ margin-left: 1em; } @@ -47,28 +74,21 @@ margin-left: 0em; } -ul a, ol a { - text-decoration: underline; -} - blockquote { font-family: Times, "Times New Roman", serif; font-style: italic; - font-size: 120%; } code { - font-size: 120%; color: Black; - /*background-color: #dee7ec;*/ background-color: #cccccc; + font-family: Courier, monospace; } pre { - font-size: 120%; - padding: 1em; + padding: 0.5em; border: 1px solid #8cacbb; color: Black; - background-color: #dee7ec; background-color: #cccccc; + font-family: Courier, monospace; } Index: publish.py =================================================================== --- publish.py (revision 27205) +++ publish.py (working copy) @@ -1,9 +1,12 @@ -import os, sys +import os, shutil, sys def publish(dirname, lxml_path, release): if not os.path.exists(dirname): os.mkdir(dirname) - stylesheet_url = 'http://codespeak.net/lxml/style.css' + stylesheet_url = 'style.css' + + shutil.copy(stylesheet_url, dirname) + for name in ['main.txt', 'intro.txt', 'api.txt', 'compatibility.txt', 'extensions.txt', 'namespace_extensions.txt', 'sax.txt']: path = os.path.join(lxml_path, 'doc', name) @@ -22,10 +25,10 @@ os.path.join(dirname, 'index.html')) def rest2html(source_path, dest_path, stylesheet_url): - - command = ('rest2html --stylesheet=%s %s > %s' % + + command = ('rest2html --stylesheet=%s --link-stylesheet %s > %s' % (stylesheet_url, source_path, dest_path)) os.system(command) - + if __name__ == '__main__': publish(sys.argv[1], sys.argv[2], sys.argv[3])

3 9

[lxml-dev] Building lxml on windows: a guide.
by David Sankel 12 May '06

12 May '06

Note: This is my first post on this list. There was an absence of windows binaries for lxml on the net (all the sites that had older versions were down). Unfortunately, there wasn't enough information to make the process of building on windows easy. I wrote this guide to solve both these problems. I hope that the information here can at worst help out others trying to build on windows and at best get included in the documentation and maintenance. I also have a windows installer created for lxml 0.9.1 that is staticly built with iconv-1.9.1, libxml2 2.6.23, libxslt 1.1.15, and zlib 1.2.3. If there is some ftp site or something I can upload it to, in order to give wide access to it, please let me know what to do. David Sankel Building lxml on windows =================== First you'll need to download the latest version of all the required files. Download them all to the same directory. * libxml: Availible from http://codespeak.net/lxml/ * iconv, libxml2, libxslt, and zlib are all availible from xmlsoft.org. The place to go on the ftp site is ftp://xmlsoft.org/libxml2/win32. Your directory should now have something like the following files in it: iconv-1.9.1.win32.zip libxml2-2.6.23.win32.zip libxslt-1.1.15.win32.zip lxml-0.9.1.tgz zlib-1.2.3.win32.zip Now extract each of those files in the _same_ directory. Now you should have something like this: iconv-1.9.1.win32/ iconv-1.9.1.win32.zip libxml2-2.6.23.win32/ libxml2-2.6.23.win32.zip libxslt-1.1.15.win32/ libxslt-1.1.15.win32.zip lxml-0.9.1/ lxml-0.9.1.tgz zlib-1.2.3.win32/ zlib-1.2.3.win32.zip Go to the lxml-0.9.1 directory and edit the Makefile. There should be a section that looks like this:: ext_modules = [ Extension( "lxml.etree", sources = sources, extra_compile_args = ['-w'] + flags('xslt-config --cflags'), extra_link_args = flags('xslt-config --libs') )], Change it to this (Warning: make sure you are using version numbers that correspond to your downloads) (Note: the _a portion of the libraries means that we are statically linking. If you want to use dlls(why?), link to the dll version of the libraries):: ext_modules = [ Extension( "lxml.etree", sources = sources, extra_compile_args = [ "-w", "-I..\\libxml2-2.6.23.win32\\include", "-I..\\libxslt-1.1.15.win32\\include", "-I..\\zlib-1.2.3.win32\\include", "-I..\\iconv-1.9.1.win32\\include" ], extra_link_args = [ "..\\libxml2-2.6.23.win32\\lib\\libxml2_a.lib", "..\\libxslt-1.1.15.win32\\lib\\libxslt_a.lib", "..\\zlib-1.2.3.win32\\lib\\zlib.lib", "..\\iconv-1.9.1.win32\\lib\\iconv_a.lib" ] )], Now you should be able to use setup.py and everything should work well. "python setup.py bdist_wininst" will create a windows installer in the pkg directory.

4 12

[lxml-dev] Uche's OT-benchmark on lxml and (c)ElementTree
by Stefan Behnel 12 May '06

12 May '06

Hi all, I just ran a slight variant (doesn't print, builds list) of Uche's OT benchmark on * ElementTree 1.2.6 * cElementTree 1.0.5 * lxml (trunk/SVN 27065, timings for 0.9.2 are similar) The code I used is attached, it runs against the 3.3M ot.xml file from http://www.ibiblio.org/bosak/xml/eg/religion.2.00.xml.zip Here's the (somewhat bogus) output on my machine (bench, result, time): bench_ET 120 1.59 bench_cET 120 0.31 bench_lxml_findall 120 0.32 bench_lxml_xpath 120 0.33 bench_lxml_xpath_all 120 0.26 I should note that lxml's findall() uses ElementTree's ElementPath implementation. Here are the real numbers from timeit: # python -m timeit -s "from otbench import *" "bench_ET()" 10 loops, best of 3: 1.36 sec per loop # python -m timeit -s "from otbench import *" "bench_cET()" 10 loops, best of 3: 260 msec per loop # python -m timeit -s "from otbench import *" "bench_lxml_findall()" 10 loops, best of 3: 295 msec per loop # python -m timeit -s "from otbench import *" "bench_lxml_xpath()" 10 loops, best of 3: 261 msec per loop # python -m timeit -s "from otbench import *" "bench_lxml_xpath_all()" 10 loops, best of 3: 214 msec per loop I think it's pretty interesting how close the timings of cET.findall() and lxml.xpath() are. Regards, Stefan

3 12

[lxml-dev] Python unicode string support in lxml
by Stefan Behnel 12 May '06

12 May '06

Hi all, I had a discussion with Fredrik lately and it convinced me (though not Fredrik) that it would be a good idea to improve the support for Python unicode strings in lxml.etree. I think that unicode strings are the most comfortable way of doing XML I/O from/to strings in Python, so I added support for simply calling unicode() on XML nodes and ElementTrees. It behaves just like tostring(), but always returns Python unicode strings: >>> from lxml import etree >>> uxml = u'<test> \uf8d1 + \uf8d2 </test>' >>> root = etree.XML(uxml) >>> unicode(root) u'<test> \uf8d1 + \uf8d2 </test>' >>> el = etree.Element("test") >>> unicode(el) u'<test/>' >>> subel = etree.SubElement(el, "subtest") >>> unicode(el) u'<test><subtest/></test>' >>> unicode( etree.ElementTree(el) ) u'<test><subtest/></test>' Note that ElementTree does not support this at all. It will raise a parser exception in the XML() call in the second line and return the same generic strings for unicode() as it does for str(). There is a longer doctest in http://codespeak.net/svn/lxml/trunk/doc/api.txt that explains this in more detail. As usual: any comments appreciated. Stefan

6 59

[lxml-dev] tounicode(), again
by Stefan Behnel 11 May '06

11 May '06

Hi all, we had a lengthy discussion yesterday and I guess we found a few use cases where tounicode() makes sense and a few counter-arguments why it might not be a good idea to expose that API at a similarly visible place as tostring(). I'm still convinced that it's a good idea to have that API, but as one of the arguments was that "people who don't understand unicode" (PeWDUUs) would be more likely to write broken code, I added this paragraph to api.txt, in the section that describes the unicode support of lxml. """ Note that the unicode strings returned by ``tounicode()`` never have an XML declaration and therefore do not specify an encoding. This makes it possible to pass them back into the lxml parsers. However, you may have to add a declaration yourself if you want to serialize such a unicode string to a byte stream later. In contrast, the ``tostring()`` function automatically adds a declaration as needed that reflects the encoding of the returned byte string. """ I hope that makes it clear enough for PeWDUUs what the advantage of using tostring() over tounicode() is and that you have to take care what you do with unicode strings. So, I propose leaving the API (and implementation) just as it is now. Regards, Stefan

2 1

[lxml-dev] lxml 0.9.2 released
by Stefan Behnel 10 May '06

10 May '06

Hello everyone, I just uploaded another bug fix release of the 0.9 branch to the Cheese shop. http://cheeseshop.python.org/pypi/lxml http://cheeseshop.python.org/packages/source/l/lxml/lxml-0.9.2.tar.gz This will (hopefully) be the last 0.9.x release before 1.0. It fixes a number of more or less annoying bugs and adds a number of smaller features. All the major, new, fancy features will go into the 1.0 release. Regards, Stefan Features added * Speedup for Element.makeelement(): the new element now reuses the original libxml2 document instead of creating a new empty one * Speedup for reversed() iteration over element children (Py2.4+ only) * ElementTree compatible QName class * RelaxNG and XMLSchema now accept any Element, not only ElementTrees Bugs fixed * str(xslt_result) was broken for XSLT output other than UTF-8 * Memory leak if write_c14n fails to write the file after conversion * Crash in XMLSchema and RelaxNG when passing non-schema documents * Memory leak in RelaxNG() when RelaxNGParseError is raised

1 0