Mailman 3 May 2006 - lxml - The Python XML Toolkit

[lxml-dev] findall()/xpath() differences ?
by Steve Howe 26 May '06

26 May '06

Hello all, How compatible are the findall() and xpath() methods ? findall() don't seem to handle more complicated XPath expressions. Why there is a difference between what they can handle ? I would expect findall() to be the same as xpath(), but searching from the context node and always returning a list of items, making it compatible with ET's. An example: Python 2.4.3 (#2, Apr 17 2006, 14:29:19) [GCC 3.4.4 [FreeBSD] 20050518] on freebsd6 Type "help", "copyright", "credits" or "license" for more information. >>> from lxml import etree >>> root = etree.XML('''<obj class="list"> ... <obj class="str" value="xxx"/> ... </obj>''') >>> print root.xpath('obj[not(@name)]') [<Element obj at 81daa2c>] >>> print root.findall('obj[not(@name)]') Traceback (most recent call last): File "<stdin>", line 1, in ? File "etree.pyx", line 937, in etree._Element.findall File "/usr/local/lib/python2.4/site-packages/lxml-1.0.beta-py2.4-freebsd-6.1-RELEASE-i386.egg/lxml/_elementpath.py", line 193, in findall return _compile(path).findall(element) File "/usr/local/lib/python2.4/site-packages/lxml-1.0.beta-py2.4-freebsd-6.1-RELEASE-i386.egg/lxml/_elementpath.py", line 171, in _compile p = Path(path) File "/usr/local/lib/python2.4/site-packages/lxml-1.0.beta-py2.4-freebsd-6.1-RELEASE-i386.egg/lxml/_elementpath.py", line 87, in __init__ raise SyntaxError( SyntaxError: expected path separator ([) -- Best regards, Steve mailto:howe@carcass.dhs.org

6 13

[lxml-dev] XMLFormatter revisited
by Stefan Behnel 22 May '06

22 May '06

Hi all, coming back to what I proposed a while ago, we currently have this: class XMLParser: def __init__(self, **options): self.options = options class HTMLParser: ... doc = ET.parse(source, parser=XMLParser(configuration)) At the time, I brought up an equivalent for output formatting: class XMLFormatter: ... That would give us a nice, symmetric API for input and output options. And it allows you to use sublasses to provide different formats: class XMLPrettyPrinter(XMLFormatter): def __init__(self): self._pretty_print = True class XHTMLFormatter(XMLFormatter): def __init__(self): self._xhtml = True xml = ET.tostring(element, formatter=XMLPrettyPrinter()) ET.parse("myfile.xhtml").write("out.xhtml", formatter=XHTMLFormatter()) After Noah's latest approach in that direction, I think we should adopt this API for 1.0. A version 1.0 is supposed to provide a stable and somewhat future-proof API, and adding keyword arguments to various API functions all the time is not quite what I call future-proof. One reason is that you can't test for the availability of keyword arguments, so adding features at that level is difficult to handle for code that wants to support them as an option. So, I propose to replace the current pretty_print keyword (which only appeared in the beta version anyway) with a new XMLPrettyPrinter class and to provide new features preferably at a subclass level of XMLFormatter (e.g. XHTMLFormatter, etc.). An alternative name would be XMLSerializer, maybe that's more general. We could also leave the pretty_print keyword in as a shortcut, but that would obviously make things a bit more complicated internally. Maybe pretty printing will just become a general keyword of the XMLFormatter class, I guess that would make sense. I will have to check how to make these classes nicely usable internally, but that's a minor problem. We already use a lot of xmlBuffer code, so that should give us a common ground for this API. If there are no objections, I'll start getting my hands on this next week, so if anyone has an opinion on this, please speak up soon. Stefan

1 1

Re: [lxml-dev] Can't change text in comments
by Stefan Behnel 22 May '06

22 May '06

Hi Noah, Noah Slater wrote: > I have noticed that setting the 'text' of a Comment does not seem to > change anything when the Comment is serialised. Oh well, /that/ was an old bug you found there. The code in that area dates from SVN revision 8082 - we're at 27509 now. Changing the text of a comment was never implemented, although ET supports it. The reason (I believe) why Martijn did not implement it at the time (unless he just forgot) is that changing comment texts in libxml2 is a bit tricky. There are no libxml2 functions for doing that and it requires checking the document dictionary to see if the original comment string isn't still in use anywhere else. Great place to add some more segfaults. I implemented this and actually found a couple of other bugs that I fixed (including the test cases that relied on these bugs). What bothers me is that lxml is consistent with ElementTree in that it adds whitespace around comment texts. I have no idea why ElementTree does that in the first place. AFAICT, this happily breaks things like SSI. Maybe Fredrik can enlighten us here? Stefan

4 4

[lxml-dev] lxml crash
by Steve Howe 22 May '06

22 May '06

Hello all, I know that the code is wring, but this crash is happening on my FreeBSD 6.1 and Win XP systems, with lxml 0.9.2: [carcass@/home/howe] python Python 2.4.3 (#2, Apr 17 2006, 14:29:19) [GCC 3.4.4 [FreeBSD] 20050518] on freebsd6 Type "help", "copyright", "credits" or "license" for more information. >>> from lxml import etree >>> a = etree.Element('a') >>> a.append(None) Exception exceptions.TypeError: 'Argument must not be None.' in 'etree._raiseIfNone' ignored zsh: segmentation fault python It should be more friendly by just displaying the message and not crashing, right ? By the way, is there any way to get the lxml version from code ? Some module '__version__' attribute ? Shouldn't it be there ? Something like: lxml.__version__ -- Best regards, Steve mailto:howe@carcass.dhs.org

5 10

Re: [lxml-dev] XMl Processing Instructions
by Stefan Behnel 20 May '06

20 May '06

Hi Noah, Noah Slater wrote: > If I understand your email correctly the behaviour I describe is not > intentional and will be fixed shortly? It has been fixed in the developer version, but there is not yet a release that has the fix. > Just to clarify - I can parse any file, but when I serialise I loose > any processing instructions. This includes the <?xml ... ?> > declaration. I understood that. > This also happens with the ResultTree (?) when I transform using XSLT. > > As an example, I use the DocBook XSLT stylesheets to transform DocBook > XML. This can often set various things up with the processing > instructions - character encoding being the most important. > > When I perform these transformations using ElementTree I loose this > information. > > As a work around at the moment I am using lxml.etree to do the > transformations using UTF-8 as the encoding. I am then using libxml2 > and libxslt to transform the serialized document bytestream a second > time with the only operation being converting from UTF-8 to another > (variable) character encoding. > > This feels quite hackish Well, it /is/ hackish. :) > - and to be honest the whole point of me > moving to lxml was because I find the libxml2 and libxslt bindings > hateful. That's one of the main reasons that drive us in writing lxml - right after users saying "thank you, it's great and helps us do stuff!" :) > To summarise, in an ideal world I would like to be able to transform a > document using XSLT specifying an encoding at transformation time and > have the ResultTree serialise with all processing instructions intact. > Additionally I would like to be able to access these programmatically > - which I don't think is possible at the moment. That's also a feature of the developer version that will eventually become lxml 1.0. See the 'docinfo' feature described here: http://codespeak.net/svn/lxml/trunk/doc/api.txt See here for a complete list of changes since 0.9.2: http://codespeak.net/svn/lxml/trunk/CHANGES.txt > I hope all this makes sense. It does. Thanks for the report. Please use the current SVN trunk version for now, lxml 1.0 is expected to be released this month. See http://codespeak.net/lxml/installation.html and the bottom of the download section in http://codespeak.net/lxml/ on how to check it out of the subversion repository and compile it. Stefan

2 2

[lxml-dev] XMl Processing Instructions
by Noah Slater 20 May '06

20 May '06

Hello, Why I use the write method of the ElementTree class why does it strip out the XML processing insturctions? I would like my documents to start with the processing instruction so I can specify encodings other than UTF-8. Thanks, Noah -- "Creativity can be a social contribution, but only in so far as society is free to use the results." - R. Stallman

2 5

[lxml-dev] Encoding Issues
by Noah Slater 20 May '06

20 May '06

Hello, I notice that I can pass an encoding parameter to an elementtree's write method. Two things: 1) When lxml supports xml processing instructions, will these be updated accordingly? I notice that the DocBook XSL will update meta-equiv HTML elements according to the value I pass in here - which is good, except for the following... 2) Why does this let me make up encodings such as "Noahs-Cool-Encoding" without raising an exception. If I was to take a guess I would imagine this falls back on UTF-8 but these is a bug IMO. In case you are wondering I am in the process of writing an XMl based HTTP publishing framework and lxml will be sitting at the very core of how I handle document conversion/manipulation/transformation. The problem with encoding lies within my content negotiation module which will (in a pythonic manner IMO) try to transform the document with each encoding specified in the Accept-Charset header of the client request. If the transform raises an exception we move on to the next one. My previous way of working would raise an exception if I tried an encoding it didn't recognise. While I am aware of how to look up encoding names using the python standard library - I am not sure if this correlates 100% with lxml and additionally I don't feel this extra step should be necessary. Thanks so much. Noah -- "Creativity can be a social contribution, but only in so far as society is free to use the results." - R. Stallman

2 2

[lxml-dev] lxml/ET/cET performance comparison in doc/performance.txt
by Stefan Behnel 19 May '06

19 May '06

Hi all, I've re-run the benchmark suite (which works nicely now) and written up some documentation about the performance of lxml in comparison to ET/cET. It's in http://codespeak.net/svn/lxml/trunk/doc/performance.txt If anyone is interested in extending the benchmark suite over other parts of the API or in adding to the comparison, I'd be glad to see some input. :) Have fun, Stefan

1 0

[lxml-dev] Bugs in 1.0.beta
by Olivier Grisel 18 May '06

18 May '06

Hi list, On my box (Ubuntu Linux Dapper Drake on i686 / python2.4) all tests pass but running the bench scripts yields some problems: >>> lxml.etree.LIBXML_COMPILED_VERSION (2, 6, 24) >>> lxml.etree.LIBXSLT_COMPILED_VERSION (1, 1, 15) xpath_extensions_old and xslt_extensions_old trigger the following exceptions: lxe: xpath_extensions_old (-- T1 ) Traceback (most recent call last): File "bench.py", line 652, in ? result = run_bench(bench, *benchmark_setup) File "bench.py", line 606, in run_bench method_call(*args) File "bench.py", line 453, in bench_xpath_extensions_old xpath = self.etree.XPath("child(.)", extensions=extensions) File "xpath.pxi", line 166, in etree.XPath.__init__ File "xpath.pxi", line 53, in etree.XPathEvaluatorBase.__init__ File "xpath.pxi", line 17, in etree._XPathContext.__init__ File "extensions.pxi", line 45, in etree._BaseContext.__init__ TypeError: unindexable object lxe: xslt_extensions_old (-- T1 ) Traceback (most recent call last): File "bench.py", line 652, in ? result = run_bench(bench, *benchmark_setup) File "bench.py", line 606, in run_bench method_call(*args) File "bench.py", line 479, in bench_xslt_extensions_old transform = self.etree.XSLT(tree, extensions) File "xslt.pxi", line 265, in etree.XSLT.__init__ File "xslt.pxi", line 189, in etree._XSLTContext.__init__ File "extensions.pxi", line 45, in etree._BaseContext.__init__ ValueError: unpack sequence of wrong size And append_element and replace_children triggers segmentation faults. The first one crashed after child <Element {bcd}zzzzz at -48285674>. I don't have time to dig further right now. I'll have more time tonight. Best, -- Olivier

2 2

[lxml-dev] lxml 1.0.beta is on cheeseshop
by Stefan Behnel 18 May '06

18 May '06

Hi all, lxml 1.0.beta is available from cheeseshop. This is expected to be the only beta release before the glorious 1.0 will arrive on June 1st (this year). http://cheeseshop.python.org/pypi/lxml The changelog is long and the time-to-market for 1.0 is short, so, as usual, we are happy to receive your bug reports. :) Have fun, Stefan

1 0