Mailman 3 November 2013 - lxml - The Python XML Toolkit

When is an xml declaration required?
by Frank Millman 17 Nov '13

17 Nov '13

Hi all I am using lxml to serialise some fairly complex structures, store them offline in a database, and read them in at runtime when required, using etree.fromstring(). Up to now I have been hand-crafting the xml, and I assumed from what I had read that it had to start with an xml declaration, such as - <?xml version="1.0" encoding="UTF-8"?> I am now turning to generating the xml programatically, and I see that you have to specify a declaration if you want one. I tried it without creating one and it works just the same. Therefore I realise that I do not understand when a declaration is required, and when it is not. Please can someone explain it to me. Thanks Frank Millman

2 2

Possible memory leak in xpath()
by Brian Bird 08 Nov '13

08 Nov '13

I think I've found a memory leak in lxml and I'm wondering if anyone can confirm if this is a problem? I've created a simple script to demonstrate below. Calling the xpath() method on a document seems to leave something in memory if (and only if) the method is called within a thread. Even when the thread completes the memory doesn't appear to be freed (it's not much, but it accumulates over time). You can see this using "ps" and looking at the memory usage for the python script. I'm using Ubuntu 12.04 and this happens with the default Python 2.7.3, Lxml 2.3.2, LibXml 2.7.8 and LibXslt 1.1.26 I tried upgrading to Lxml 3.2.3 but the same thing seems to happen. I'm about to attempt upgrading the LibXml package too but I'd appreciate any suggestions/help - even if someone could confirm if the same problem happens on their system would be good. Thanks! import lxml.etree import thread import time def test(): doc = lxml.etree.fromstring("<Root></Root>") doc.xpath("/Root") # This line seems to cause memory usage to go up only when used in a thread # doc.getchildren() # If this line is used instead the memory usage stays constant for i in range(100000): thread.start_new_thread(test,()) #test() # Using this line instead of starting a thread causes the memory usage to stay constant time.sleep(1) # Give plenty of time for the thread to complete raw_input("finished")

4 12

Get text content of an element with iterparse
by Bogdan Cristea 08 Nov '13

08 Nov '13

Hi I am trying to emulate the API of QXmlStreamReader and I am using iterparse(). My problem is that I cannot find a reliable way of getting the chars between the elements. I can have two situations: chars that are element text or element tail. When receiving "start" event I can obtain both text and tail, but these values are not reliable, not always the text is parsed at this point. Is there a way to get around this issue ? thanks Bogdan

2 3

lxml 3.2.4 released
by Stefan Behnel 07 Nov '13

07 Nov '13

Hi everyone, I released lxml 3.2.4 as a bug-fix release for the stable 3.2 release series. This release fixes a memory leak in XPath when using threads, so upgrading is recommended for threaded environments. You can get it from PyPI: http://pypi.python.org/pypi/lxml/3.2.4 The documentation is here: http://lxml.de/ Download: http://lxml.de/files/lxml-3.2.4.tgz Signature: http://lxml.de/files/lxml-3.2.4.tgz.asc Changelog: http://lxml.de/3.2/changes-3.2.4.html GitHub: https://github.com/lxml/lxml/commit/f07ca33d53618d96c5752b2e35e3464c5bdc2a58 This release was built using Cython 0.19.2. Note that the build no longer uses Cython, even if it is installed. Recompilation of the sources has to be requested explicitly with the setup.py option "--with-cython". If you are interested in commercial support or customisations for the lxml package, please contact me directly. Have fun, Stefan 3.2.4 (2013-11-07) ================== Bugs fixed ---------- * Memory leak when creating an XPath evaluator in a thread. * LP#1228881: ``repr(XSLTAccessControl)`` failed in Python 3. * Raise ``ValueError`` when trying to append an Element to itself or to one of its own descendants. * LP#1206077: htmldiff discarded whitespace from the output. * Compressed plain-text serialisation to file-like objects was broken.

1 0

Xpath, span control and iterate inside xpath result.
by Alexandre Delanoë 06 Nov '13

06 Nov '13

Hello and many thanks for lxml module! I am discovering its capabilities but I can hardly end up my first parser, because of 2 issues. First, I have a valid file.html to parse, here is an example: <html> <body> <tbody> <tr> <td> <table id="tbHeaderImages"> <span class="Title"> BLO BLO <span style="color:red; font-weight:bold">BLU BLU</span> BLA BLA </span> <br> <br> Some texttext <br> ... </table> </tbody> </body> </html> I have many documents each inside table: for document in TREE.xpath('/html/body/table'): Then, I want to extract each title: title = document.xpath("./tr/td/span[@class = 'Title']/text()") Problem 1: I get "BLO BLO" and "BLA BLA" whereas I would like "BLO BLO" and "BLU BLU" and "BLA BLA". I have heard that I should get the value of the text from the first node. Get the second node (not the text value) and use the .text attribute to replace the text. But I do not really understand how to manage it. Please may you give me an example. Problem 2: I want also to extract "Some texttext" just after the span class. Then, I have to trigger an event in an SAX like method. If you agree with with how I could do such trick if I am already working in the TREE.xpath('/html/body/table') ? Many thanks for your help, -- Alexandre Delanoë

4 6