Mailman 3 December 2007 - lxml - The Python XML Toolkit

[lxml-dev] lxml has its page on launchpad
by Stefan Behnel 11 Apr '23

11 Apr '23

Hi all, I added the lxml project to launchpad, the Ubuntu Bug-Tracker. It also has a FAQ engine and a couple of other goodies. https://launchpad.net/lxml It's easy to sign up for launchpad, BTW, no 90%-footnotes-contract. Have fun, Stefan

9 9

[lxml-dev] Reparenting a node
by Lawrence Oluyede 30 Jan '23

30 Jan '23

I have a doc A and a doc B, I'd like to put a node extracted from A in the document B but I always get a ValueError: ValueError: Element is not a child of this node. I didn't find any "setparent" in the API. How can I do this? -- Lawrence, oluyede.org - neropercaso.it "It is difficult to get a man to understand something when his salary depends on not understanding it" - Upton Sinclair

3 2

[lxml-dev] Building LXML Trunk
by Sidnei da Silva 31 Aug '22

31 Aug '22

Hi, I've tried to build lxml from trunk today, on Win32. Got the following error: src\lxml\etree.c(880) : error C2059: syntax error : ')' src\lxml\etree.c(881) : error C2059: syntax error : ')' src\lxml\etree.c(882) : error C2059: syntax error : ')' src\lxml\etree.c(883) : error C2059: syntax error : ')' Any clue? Smells like a Pyrex issue? -- Sidnei da Silva Enfold Systems http://enfoldsystems.com Fax +1 832 201 8856 Office +1 713 942 2377 Ext 214

4 4

[lxml-dev] Pickling objectified trees
by Christian Zagrodnick 09 Jun '08

09 Jun '08

Hi, the other day I had to pickle objectified trees. I just thought to share my findings. Pickling is about serialization. IMHO the natural serialization of an objectified tree is its XML representation. So the following basically does that: -------------------------- import copy_reg import lxml.etree import lxml.objectify def treeFactory(state): """Un-Pickle factory.""" return lxml.objectify.fromstring(state) copy_reg.constructor(treeFactory) def reduceObjectifiedElement(object): """Reduce function for lxml.objectify trees. See http://docs.python.org/lib/pickle-protocol.html for details. """ return (treeFactory, (lxml.etree.tostring(object), )) copy_reg.pickle(lxml.objectify.ObjectifiedElement, reduceObjectifiedElement, treeFactory) ----------------------------------------- You might consider just registering the reduce function in lxml itself. Shouldn't hurt, should it. -- Christian Zagrodnick gocept gmbh & co. kg · forsterstrasse 29 · 06112 halle/saale www.gocept.com · fon. +49 345 12298894 · fax. +49 345 12298891

2 4

[lxml-dev] lxml namespaces
by Maxim Sloyko 20 Feb '08

20 Feb '08

Hi All! I have a little problem with XML namespaces. In my application I have two XML processors, that process the same document, one after the other. The first one looks for nodes in 'ns1' namespace, and substitutes them, according to some algorithm. After this processor is finished, it is guaranteed that there are no more 'ns1' nodes left in the tree. 'ns1' namespace dclaration is still there, in the root node (well, I put it there manually). Now, when this namespace is no longer needed, I want to get rid of it, because it confuses some other processors (namely, my browser) So, the question is, how do I do that? del tree.getroot().nsmap['ns1'] does not seem to do the trick :( Thanks in advance and Happy Holidays! -- Maxim Sloyko

3 5

[lxml-dev] lxml \ libxslt \ libxml2 leads to apache 2 crash on freebsd/amd64
by Dmitri Fedoruk 09 Jan '08

09 Jan '08

Greetings! First of all, happy holidays. Looks like it's not the perfect time to report a problem, but I can't find any solution by myself. More than that, I'm not sure this is a lxml problem. But I really hope for somebody to take a look at this long message :) My application is a web service written in Python. It is running inside of mod_python handled by Apache2 (precise versions follow). Apache runs in prefork mode. I use xml to collect data from sources. I use xslt for their processing and, eventually, forming the html output. I use lxml library for this. Xslt transformations are compiled once for every instance of apache fork and then they run in their own thread independently. Up to one moment everything went smoothly and painlessly. Then I decided to upgrade all the software and came to the following situation. When I try to serialize an etree object ... transform = etree.XSLT(xslt_doc) result_tree = transform(data, **variables ) return etree.tostring(result_tree, 'utf-8') ( or return str(result_tree) ) I have a core dump of apache. The point is that it happens during serialization. I tried to omit 'utf-8', I tried to use str() - did not help. What is strange - this error occurs with only several xsl templates, and not all the time. What is even more strange: this happens only on amd64 machines _within_ apache. On my i386 desktop I can not reproduce this bug. Also I can not reproduce this from the console. Now the configuration details. I tried almost all possible combinations of this software. FreeBSD 6.2-20070330-SNAP Apache/2.0.61, Apache/2.2.6 mod_python-3.3.1 lxml 1.3.5, 1.3.6 libxml2-2.6.27, libxml2-2.6.30 libxslt-1.1.20, libxslt-1.1.22 I have to mention that I do also use some modules written in C that have pythonic binding. We suggested that they could be the reason of memory corruption. We run the application without their usage and still got the same core dump, so they are not the reason. The error messages are httpd in free(): error: chunk is already free httpd in free(): error: modified (chunk-) pointer httpd in free(): error: pointer to wrong page And the backtrace: (gdb) bt #0 0x00000008013824bc in kill () from /lib/libc.so.6 #1 0x000000080138134d in abort () from /lib/libc.so.6 #2 0x000000080131a265 in _UTF8_init () from /lib/libc.so.6 #3 0x000000080131a29c in _UTF8_init () from /lib/libc.so.6 #4 0x000000080131b23d in _UTF8_init () from /lib/libc.so.6 #5 0x00000008055dc3f9 in xmlFreeNodeList () from /usr/local/lib/libxml2.so.5 #6 0x00000008055dc29d in xmlFreeProp () from /usr/local/lib/libxml2.so.5 #7 0x00000008055dc2dc in xmlFreePropList () from /usr/local/lib/libxml2.so.5 #8 0x00000008055dc4bb in xmlFreeNodeList () from /usr/local/lib/libxml2.so.5 #9 0x00000008055dc385 in xmlFreeNodeList () from /usr/local/lib/libxml2.so.5 #10 0x00000008055dc385 in xmlFreeNodeList () from /usr/local/lib/libxml2.so.5 #11 0x00000008055dc385 in xmlFreeNodeList () from /usr/local/lib/libxml2.so.5 #12 0x00000008055dc385 in xmlFreeNodeList () from /usr/local/lib/libxml2.so.5 #13 0x00000008055dc385 in xmlFreeNodeList () from /usr/local/lib/libxml2.so.5 #14 0x00000008055dcad5 in xmlFreeDoc () from /usr/local/lib/libxml2.so.5 #15 0x00000008051a8399 in __pyx_tp_dealloc_5etree__Document () from /usr/local/lib/python2.5/site-packages/lxml-1.3.6-py2.5-freebsd-6.2-20070912-SNAP-amd64.egg/lxml/etree.so #16 0x00000008051be15b in __pyx_tp_dealloc_5etree__Element () from /usr/local/lib/python2.5/site-packages/lxml-1.3.6-py2.5-freebsd-6.2-20070912-SNAP-amd64.egg/lxml/etree.so #17 0x00000008051a9b8f in __pyx_tp_dealloc_5etree__ElementTree () from /usr/local/lib/python2.5/site-packages/lxml-1.3.6-py2.5-freebsd-6.2-20070912-SNAP-amd64.egg/lxml/etree.so #18 0x00000008036cbb4b in _PyFloat_Unpack8 () from /usr/local/lib/libpython2.5.so #19 0x0000000803723c03 in PyEval_EvalCodeEx () from /usr/local/lib/libpython2.5.so #20 0x00000008037229eb in PyEval_EvalFrameEx () from /usr/local/lib/libpython2.5.so #21 0x0000000803723c24 in PyEval_EvalCodeEx () from /usr/local/lib/libpython2.5.so #22 0x00000008037229eb in PyEval_EvalFrameEx () from /usr/local/lib/libpython2.5.so #23 0x0000000803723326 in PyEval_EvalFrameEx () from /usr/local/lib/libpython2.5.so #24 0x0000000803723326 in PyEval_EvalFrameEx () from /usr/local/lib/libpython2.5.so #25 0x0000000803723326 in PyEval_EvalFrameEx () from /usr/local/lib/libpython2.5.so #26 0x0000000803723326 in PyEval_EvalFrameEx () from /usr/local/lib/libpython2.5.so #27 0x0000000803723326 in PyEval_EvalFrameEx () from /usr/local/lib/libpython2.5.so #28 0x0000000803723326 in PyEval_EvalFrameEx () from /usr/local/lib/libpython2.5.so #29 0x0000000803723c24 in PyEval_EvalCodeEx () from /usr/local/lib/libpython2.5.so #30 0x00000008036cd8ae in PyFunction_SetClosure () from /usr/local/lib/libpython2.5.so #31 0x00000008036b3c73 in PyObject_Call () from /usr/local/lib/libpython2.5.so #32 0x0000000803721262 in PyEval_EvalFrameEx () from /usr/local/lib/libpython2.5.so #33 0x0000000803723c24 in PyEval_EvalCodeEx () from /usr/local/lib/libpython2.5.so #34 0x00000008037229eb in PyEval_EvalFrameEx () from /usr/local/lib/libpython2.5.so #35 0x0000000803723326 in PyEval_EvalFrameEx () from /usr/local/lib/libpython2.5.so #36 0x0000000803723326 in PyEval_EvalFrameEx () from /usr/local/lib/libpython2.5.so #37 0x0000000803723326 in PyEval_EvalFrameEx () from /usr/local/lib/libpython2.5.so #38 0x0000000803723c24 in PyEval_EvalCodeEx () from /usr/local/lib/libpython2.5.so #39 0x00000008037229eb in PyEval_EvalFrameEx () from /usr/local/lib/libpython2.5.so #40 0x0000000803723c24 in PyEval_EvalCodeEx () from /usr/local/lib/libpython2.5.so #41 0x00000008036cd8ae in PyFunction_SetClosure () from /usr/local/lib/libpython2.5.so #42 0x00000008036b3c73 in PyObject_Call () from /usr/local/lib/libpython2.5.so #43 0x00000008036bbd64 in PyMethod_New () from /usr/local/lib/libpython2.5.so #44 0x00000008036b3c73 in PyObject_Call () from /usr/local/lib/libpython2.5.so #45 0x00000008036b3d09 in PyObject_Call () from /usr/local/lib/libpython2.5.so #46 0x00000008036b4024 in PyObject_CallMethod () from /usr/local/lib/libpython2.5.so #47 0x000000080356acf2 in python_handler () from /usr/local/libexec/apache2/mod_python.so #48 0x0000000000425e4a in ap_run_handler (r=0x190b090) at config.c:152 #49 0x0000000000426755 in ap_invoke_handler (r=0x190b090) at config.c:364 #50 0x0000000000422550 in ap_process_request (r=0x190b090) at http_request.c:249 #51 0x000000000041bbd7 in ap_process_http_connection (c=0x77f1b0) at http_core.c:253 #52 0x0000000000433c1a in ap_run_process_connection (c=0x77f1b0) at connection.c:43 #53 0x0000000000434075 in ap_process_connection (c=0x77f1b0, csd=0x77f090) at connection.c:176 #54 0x000000000042437f in child_main (child_num_arg=3) at prefork.c:610 #55 0x000000000042450b in make_child (s=0x5a94f8, slot=3) at prefork.c:704 #56 0x00000000004247b0 in perform_idle_server_maintenance (p=0x578028) at prefork.c:839 #57 0x0000000000424c37 in ap_mpm_run (_pconf=0x578028, plog=0x5a4028, s=0x5a94f8) at prefork.c:1040 #58 0x000000000042d27e in main (argc=1, argv=0x7fffffffea08) at main.c:656 The irony of it all is that I still can not downgrade all this packages to the version that did not have core dump... I've spent a week already trying to figure out the gist of the problem, combining different lxml, libxml2\xslt and apache versions. I start with this mailing list as the fatal call is made with the lxml function. I really hope there is an explanation and solution for this. I do not want to switch to another xslt processor or rebuild the architecture of the whole service in the worst case. I'd be grateful for any ideas and suggestions. Cheers, Dmitri

3 12

[lxml-dev] lxml homepage broken
by Peter Rust 28 Dec '07

28 Dec '07

FYI: The "default document" of the lxml site appears to be broken. I can get to the page directly (http://codespeak.net/lxml/index.html), but the codespeak homepage link (and most external links) point to http://codespeak.net/lxml, which doesn't work. Peter Rust Developer, Cornerstone Systems PS: I started using lxml.etree yesterday as a drop-in replacement for elementtree and have been very pleased with the way it handles comments and doctypes, thank you!

3 3

[lxml-dev] Python crash with lxml use with multi-thread
by Pedro Cardoso 20 Dec '07

20 Dec '07

Hi I am using lxml (just updated to version 1.3.6) on python 2.5, on a windows XP machine. I have a wrapper class for an XML file, and worked fine until I used it on several threads. At that moment it will crash. The piece of code I have it : projectNode = self.dom.xpath('//Project[@Code="%s"]' %projectCode)[0] logger.debug("here") previewsTask = self.dom.xpath('//Task[@Code="%s"]' % taskData.xml_Code)[0] logger.debug("here") projectNode.replace(previewsTask, taskData.dom) #projectNode.remove(previewsTask) #projectNode.append(taskData.dom) logger.debug("here") self.save() logger.debug("here") return None The loggin is to know the last line to be called . The last log msg is written, but the one right after the call to this function is not, that makes me think the crash is on the return. There should be no problem of thread safety, since I am using locks. The crash seems to be on the return, and The calling sequence is : thread 1 : append a node to xml and save thread 2 : replaces node and save (the piece of code above) At this moment it crash python ! Any idea of the problem ? Thanks in advance Pedro Cardoso

2 2

[lxml-dev] lxml 2.0alpha6 released
by Stefan Behnel 19 Dec '07

19 Dec '07

Hi, another 'last' alpha version of the 2.0 series was released to PyPI. The reason this is still not a beta version is that there were a couple of signature changes in methods for iteration, XPath and XSLT to remove some inconsistencies. I hope these do not cause too much hassle for maintainers of existing code. Backwards compatibility is provided through the use of keyword arguments (which are now enforced in some places where they make sense). I'm open to discussion about these changes, but would otherwise declare this release ready for the API freeze. From my point of view, the next release will only focus on bug fixing (if required :) before the final release (hopefully still this year). This release already features some major bug fixes including a memory leak that was introduced in 2.0alpha5 (by newer Cython versions). lxml now requires the not-yet-released Cython 0.9.6.10 to build, which will hopefully contain the bug fix. One thing to improve for 2.0 is the documentation (as usual), especially the generated API docs: http://codespeak.net/lxml/dev/api/index.html Any help is appreciated, as are bug reports and criticism. Have fun, Stefan 2.0alpha6 (2007-12-19) ====================== Features added -------------- * New properties ``position`` and ``code`` on ParseError exception (as in ET 1.3) Bugs fixed ---------- * Memory leak in the ``parse()`` function. * Minor bugs in XSLT error message formatting. * Result document memory leak in target parser. Other changes ------------- * Various places in the XPath, XSLT and iteration APIs now require keyword-only arguments. * The argument order in ``element.itersiblings()`` was changed to match the order used in all other iteration methods. The second argument ('preceding') is now a keyword-only argument. * The ``getiterator()`` method on Elements and ElementTrees was reverted to return an iterator as it did in lxml 1.x. The ET API specification allows it to return either a sequence or an iterator, and it traditionally returned a sequence in ET and an iterator in lxml. However, it is now deprecated in favour of the ``iter()`` method, which should be used in new code wherever possible. * The 'pretty printed' serialisation of ElementTree objects now inserts newlines at the root level between processing instructions, comments and the root tag. * A 'pretty printed' serialisation is now terminated with a newline. * Second argument to ``lxml.etree.Extension()`` helper is no longer required, third argument is now a keyword-only argument ``ns``. * ``lxml.html.tostring`` takes an ``encoding`` argument.

1 0

[lxml-dev] Huge memory leak in latest 2.0
by Artur Siekielski 19 Dec '07

19 Dec '07

Hi. I'm using latest 2.0 version from trunk, rev. 49494 (because it supports 'encoding' keyword in HTMLParser). I'm parsing many HTML documents in loop, 100-200kB each. I have noticed that memory used by my program increases about 1MB after each document processed, so after a few hundreds of passes system is about to hang. Running the same code with lxml 1.3.6 doesn't cause such memory usage increase. I'm using the following library calls: tree = etree.parse( <opened file>, HTMLParser(encoding=...)) etree.tostring(tree) el.xpath(...) getting children and attributes of elements I'm using libxml2 version 2.6.28. If anyone knows about solution/workaround, please write. Regards, Artur

2 3