Mailman 3 April 2015 - lxml - The Python XML Toolkit

[lxml-dev] lxml has its page on launchpad
by Stefan Behnel 11 Apr '23

11 Apr '23

Hi all, I added the lxml project to launchpad, the Ubuntu Bug-Tracker. It also has a FAQ engine and a couple of other goodies. https://launchpad.net/lxml It's easy to sign up for launchpad, BTW, no 90%-footnotes-contract. Have fun, Stefan

9 9

[lxml-dev] Checking whether a node is a comment/element
by Geoffrey Sneddon 10 Apr '23

10 Apr '23

Hi, What's the best way to check whether a given node is a comment or an element? For the former, I'm currently using isinstance(node, etree._Comment), which is rather obviously sub-optimal. -- Geoffrey Sneddon <http://gsnedders.com/>

6 6

[lxml-dev] Reparenting a node
by Lawrence Oluyede 30 Jan '23

30 Jan '23

I have a doc A and a doc B, I'd like to put a node extracted from A in the document B but I always get a ValueError: ValueError: Element is not a child of this node. I didn't find any "setparent" in the API. How can I do this? -- Lawrence, oluyede.org - neropercaso.it "It is difficult to get a man to understand something when his salary depends on not understanding it" - Upton Sinclair

3 2

[lxml-dev] lxml 2.0.5 released
by Stefan Behnel 11 Jan '23

11 Jan '23

Hi all, lxml 2.0.5 is on PyPI. This is a bug-fix-only release of the stable 2.0 series. Have fun, Stefan 2.0.5 (2008-05-01) Bugs fixed * Resolving to a filename in custom resolvers didn't work. * lxml did not honour libxslt's second error state "STOPPED", which let some XSLT errors pass silently. * Memory leak in Schematron with libxml2 >= 2.6.31.

3 4

[lxml-dev] Building LXML Trunk
by Sidnei da Silva 31 Aug '22

31 Aug '22

Hi, I've tried to build lxml from trunk today, on Win32. Got the following error: src\lxml\etree.c(880) : error C2059: syntax error : ')' src\lxml\etree.c(881) : error C2059: syntax error : ')' src\lxml\etree.c(882) : error C2059: syntax error : ')' src\lxml\etree.c(883) : error C2059: syntax error : ')' Any clue? Smells like a Pyrex issue? -- Sidnei da Silva Enfold Systems http://enfoldsystems.com Fax +1 832 201 8856 Office +1 713 942 2377 Ext 214

4 4

XSLT extension: process_children
by Jens Tröger 10 May '15

10 May '15

Hi, I am trying to understand the process_children() function. If in my extension is have def execute(self, context, self_node, input_node, output_parent): self.process_children(context, output_parent) then processing works just fine (although I haven't done anything with the output here). However, using def execute(self, context, self_node, input_node, output_parent): x = self.process_children(context, output_parent=None) output_parent.append(x) crashes: x = self.process_children(context, output_parent=None) File "xsltext.pxi", line 111, in lxml.etree.XSLTExtension.process_children (src/lxml/lxml.etree.c:164356) File "readonlytree.pxi", line 550, in lxml.etree._nonRoNodeOf (src/lxml/lxml.etree.c:78766) TypeError: invalid argument type <class 'NoneType'> although I had assumed the two to be synonymous. I ask because I want to inject my own node and dangle children off of my own node like this: p = lxml.etree.Element("p") p.attrib["style"] = "..." output_parent.append(p) self.process_children(context, p) but that, too, fails. What am I doing wrong here? Thanks! Jens -- Jens Tröger http://savage.light-speed.de/

2 5

Performance of _Element.find()
by Jens Tröger 26 Apr '15

26 Apr '15

Hi, Suppose I have a large and shallow XML tree; in my case a <book> with several thousand <par> elements. I also have a large number (thousands) of paths like ... chapter[2]/par[19] ... chapter[2]/par[538]/em ... chapter[2]/par[1937] ... I started with a loop that iterates over these paths, calls find() and then manipulates the attributes of the found _Element instance. That could take several tens of seconds. It turns out that the culprit in my loop was the call to find(), accounting for 99% of the time. So I tried to see what would happen if I used an index map, and that's being generated in two passes: 1. Iterate over all paths, split them into their components and use those components as dict keys. Nest the dictionaries according to their path component. The value is a tuple (elem, dict()) where elem will be filled in by the second pass, and the dictionary is for nesting. For the above example: { 'chapter[2]': (None, { 'par[19]': (None, {}), 'par[538]': (None, { 'em': (None, {}) }), 'par[1937]: (None, {}), }), } 2. Iterate over all nodes of the XML tree (xpath('//*')) and get their path. Then fill the above dictionary with the elem references for those which are in that dictionary, i.e. replace the None with _Element instance references. Building that index map is negligible. Using this index map to find the elements in the XML tree is orders of magnitude faster than using find() -- iterating over all path expressions to manipulate attributes of elements went from several tens of seconds to a fraction of a second. I am astonished that find() is so slow? Why is that? Is walking down the tree based on the path string that expensive? Cheers, Jens -- Jens Tröger http://savage.light-speed.de/

2 3

lxml 3.4.3 released
by Stefan Behnel 15 Apr '15

15 Apr '15

Hi all, I just released a conservative bug fix version of lxml, 3.4.3. Updating is recommended as it fixes the ElementPath expression cache and thus a previous performance regression in the find*() methods. Note that the 3.4 release series no longer supports Python versions before Py2.6 and Py3.2. Support for very old versions of libxml2 and libxslt (<=2008) was also removed. The documentation is here: http://lxml.de/ Download: http://lxml.de/files/lxml-3.4.3.tgz Signature: http://lxml.de/files/lxml-3.4.3.tgz.asc Changelog: http://lxml.de/3.4/changes-3.4.3.html Github: https://github.com/lxml/lxml/commit/fe7dfaa133ec963a5173169993d464c324640f87 This release was built using Cython 0.21.2. If you are interested in commercial support or customisations for the lxml package, please contact me directly. Have fun, Stefan 3.4.3 (2015-04-15) ================== Bugs fixed ---------- * Expression cache in ElementPath was ignored. Fix by Changaco. * LP#1426868: Passing a default namespace and a prefixed namespace mapping as nsmap into ``xmlfile.element()`` raised a ``TypeError``. * LP#1421927: DOCTYPE system URLs were incorrectly quoted when containing double quotes. Patch by Olli Pottonen. * LP#1419354: meta-redirect URLs were incorrectly processed by ``iterlinks()`` if preceded by whitespace.

1 0

Very slow XMLSchema parsing
by Andreas Jung 11 Apr '15

11 Apr '15

Hi there, I am using this code https://github.com/xml-director/xmldirector.demo/blob/master/xmldirector/de… for benchmarking the XSD parsing speed against some variants of the MODS schema. mods-3-1.xsd and mods-3-2.xsd take about 15 seconds for parsing while the other variants parser in less than 0.3 seconds. How can one explain this huge difference? Andreas ---- mods-3-1.xsd 0.00209999084473 15.2129580975 -------------------------------------------------------------------------------- mods-3-2.xsd 0.00260806083679 15.2835290432 -------------------------------------------------------------------------------- mods-3-3.xsd 0.00289702415466 0.300955057144 -------------------------------------------------------------------------------- mods-3-4.xsd 0.00385713577271 0.313620090485 -------------------------------------------------------------------------------- mods-3-5.xsd 0.00278782844543 0.278451919556

3 4

HTML alignment of Offsets and XPath
by John R. Frank 04 Apr '15

04 Apr '15

Hi lxml experts, I have a question about character offsets and Xpath that would not be necessary if everything in the world were XML: tl;dr: what is the best way to translate back and forth between a character (or byte) offset in the string from of the HTML and the Xpath-plus-relative-offset in the rendered DOM? Context: We're working on an HTML highlighting tool that is intended to allow users to modify/create text selections generated by automatic named entity recognition algorithms. Many parts of this are working, and a key element uses your wonderful lxml.html.clean.Cleaner [1] After cleansing a page to make `clean_html`, we also generate a tag-stripped form that has exactly the same byte offsets by replacing tags with whitespace of the same byte length. We call this `clean_visible`. This allows named entity recognizers, such as LingPipe, Basis Rosette, Stanford CoreNLP, Clear Forest, etc to recognize names of things (people, companies, locations, etc) in the natural language. The resulting offsets then are correct for both the HTML string and the tag-stripped form. Some NER tools can parse HTML to do an even better job of their task, and many do not. The user facing components are in a not-yet-released FOSS javascript component called "HTML highlighter" that operates *in the browser* on the live DOM rendered from the clean_html. Using ideas similar to those in Rangy [2], it figures out xpath+offset for start and end of the user's selection. This form of offset is really all that JavaScript can handle. To make this whole thing work perfectly, we need to construct a python service that translates between absolute offsets in the HTML string and xpath+offsets in the corresponding DOM. Can lxml help with this? Thanks for any guidance. (and please no flames about how the whole world should be in XML, because... that's not this world :-) John [1] https://github.com/trec-kba/streamcorpus-pipeline/blob/master/streamcorpus_… [2] https://github.com/timdown/rangy -- ______________________________ John R. Frank <jrf(a)diffeo.com>

1 0