Mailman 3 - lxml - The Python XML Toolkit

license for newly added _difflib.py
by Jakub Kulik May 18, 2026

May 18, 2026

Hi, The latest version of lxml introduced _difflib.py, which is taken (as the comment on the top suggests) from Python 3.14 sources. The file mentions no licensing information, thus appearing BSD Licensed like the rest of lxml sources, but since it's from Python, I presume it should be PSF licensed? Should the PSF License be mentioned in e.g. LICENSES.txt? Thanks! Jakub

2 1

Re: what you can and cannot do with lxml etree.strip_tags
by jholg＠gmx.de April 14, 2026

April 14, 2026

Hi, On Montag, 13. April 2026 22:14:09 CEST Martin Mueller wrote: > Many thanks for your helpful response. I did indeed leave out the critical > detail of the number of hi children. > > This was the code that used strip_tags > > > def process_tree(tree,filename, item): > for text in tree.iter(tei + 'text'): > for hi in text.iter(tei + 'hi'): > if len(hi) == 1 : > etree.strip_tags(text, tei+ 'hi') > > This ignore the "len(hi) == 1" condition and drops all hi elements but > leaves the children in place. It would be nice if there were a way of > telling strip_tags only to remove hi elements that meet some condition. > That said, it's easy enough to use the addnext() method and loop through > the hi element in reverse order. > > The URL you sent me got wrapped in some security garbage and was missing > something at the end. I couldn't open it, but it looked like a useful > source. Could you send it again? That was actually a mailing list link to a previous conversation of yours on the lxml mailing list from March 7th 2026: https://mail.python.org/archives/list/lxml@python.org/message/ 2F7ZICTGEBIQEHRUOSGUE7DZQ7OAA4AZ/ Be aware that that my mailer may put in an unwanted line break so you might need to make sure you copy the full link to the browser. It's a nice tip by Jens Tröger, quoting him here for convenience: " Are you familiar with XPath? I think for w in tree.xpath("//hi[count(w) < 4]/w"): w.attrib["rend"] = "hi" # Add the @rend attribute to the <w> children. hi = w.getparent() # Get the <hi> parent. hi.tag = "hi-remove-me" # Change the <hi> element’s tag so we can find it. etree.strip_tags(tree, "hi-remove-me") # Not sure about the ns here. should get you started… 🤔 Jens " Best, Holger

1 0

what you can and cannot do with lxml etree.strip_tags
by Martin Mueller April 13, 2026

April 13, 2026

Lxml is an excellent program, but its documentation is very terse, and often assumes a reader who already "sort of" knows what the actual reader does not. Steven Pinker in his Sense of Style has a chapter on "The Curse of Knowledge" with the wonderful topic sentence: "The main cause of incomprehensible prose is the difficulty of imaging what it's like for someone else not to know something that you know". I had trouble with etree.strip_tags but discovered that you can't add secondary constraints to a strip_tag command. Thus you can strip all "hi" children of a div, but you cannot do this for <hi> tags with less than child elements. Is that the case? ) If you want to operate on <hi> elements with further specification, you select them and then use the "addnext" method, looping through the <hi> element in reverse order. Is that correct? Martin Mueller Professor emeritus of English and Classics Northwestern University

2 1

baffled by namespace problem in xpath calls
by Martin Mueller March 11, 2026

March 11, 2026

I am probably missing something trivial, but I am completely baffled by the use of namespaces when using xpath with lxml. I have a template that creates a tree from an XML document and passes it to a function for processing at the w level. If I change the root element of a TEI document into ABC (turning it into a generic XML document),the following code works: for w in tree.xpath('//w'): print(etree.tostring(w)) This shouldn't work in a TEI document, because the xpath method requires a qualified name or namespace. I assumed that the full tag name {http://www.tei-c.org/ns/1.0} might work: for w in tree.xpath('http://www.tei-c.org/ns/1.0}w'): print(etree.tostring(w)) But it doesn't work. Neither does a special xmlns:tei declaration. So I am totally baffled. For ordinary lxml processing the ordinary mapping of the URI to tei would work. But it fails for xpath. The rather terse lxml manual was no help. I'll be grateful for any help.

3 3

specifying conditions for removing the tags of a particular elemenet
by Martin Mueller March 9, 2026

March 9, 2026

I'm having trouble with the etree.strip_tags function. I have a large TEI corpus with <w> elements for every word token. There are a lot of short <hi> element, and I would like to get rid of them and move the <hi> markers into @rend attributes, which simplifies processing at the level beyond <w> elements. I would like to 1. mark all the <w> children of <hi> elements with less than four children with appropriate @rend attributes 2. use the strip_tags function to get rid of <hi> elements with less than four children I know how to get rid of all <hi> elements with something like def process_tree(tree,filename, item): for div in tree.iter(tei +'div'): for el in div.iter('*'): etree.strip_tags(el, tei + 'hi') newname = os.path.join(allerlei, item) tree.write(newname, encoding='utf-8') But something like for el in div.iter('*'): if len(el) < 4: etree.strip_tags(el, tei + 'hi') does not work. Is it not possible to specify constraints on the strip_tags function or do I not know what I am doing? The latter very likely, and I will be grateful for any advice Martin Mueller Professor emeritus of English and Classics Northwestern University

3 2

Questions about XSLT (XSL Transformations)
by Schimon Jehudah March 5, 2026

March 5, 2026

Greetings. I am interested to know more of the XSLT component of the module LXML. Is this XSLT component an own XSLT implementation, or is it a so called "binding" with another software (e.g. libxslt)? I ask because, the XSLT component of Python appears to fail, whereas Saxon JS suceeds. So, I am interested to improve the XSLT component of LXML. Regards, Schimon

2 2

running xpath query on result of an xpath query gives results which were excluded by the first xpath query
by lord of edges Feb. 18, 2026

Feb. 18, 2026

Will start by saying I am sure this is documented somehow and I probably just don't know the right word to find that documentation. I'm trying to query a table from a document, after which I will query details from that table. The issue I have is that the second xpath query is pulling results from the entire document. As an example, I have: ``` from lxml import html exampledocument=""" <head> example </head> <body> <table><tbody> <tr><th>exampleCellNotToFind</th> </tbody></table> <table id="exampleTableToFind"><tbody> <tr><th>exampleCellToFind</th> </tbody></table> </body> """ example=html.fromstring(exampledocument) xpath1=example.xpath('//table[@id="exampleTableToFind"]', smart_strings=False) xpath2=xpath1[0].xpath('//table', smart_strings=False) xpath3=html.fromstring(html.tostring(xpath1[0])).xpath('//table', smart_strings=False) ``` In this case, xpath2 includes both tables (xpath1 only includes 1 table), and I don't understand why. From reading "XPath return values" section of https://lxml.de/xpathxslt.html#xpath I thought smart_strings=True would the reason, but setting it to false didn't seem to change the output as far as I can tell. I do have a workable solution in this html.fromstring(html.tostring()), but thought I should ask here to try to understand how these element type variables are working. Thanks, for any help.

4 4

Re: running xpath query on result of an xpath query gives results which were excluded by the first xpath query
by Holger.Joukl＠LBBW.de Feb. 18, 2026

Feb. 18, 2026

Hi, > Thanks, that makes a little bit more sense. I had understood that // > was relative and / was absolute, [...] You either made a typo or you got that wrong. In XPath "." (leading dot) is relative, while "/" starts at the root node. Doubling the slash basically means "search all nodes below" i.e. ".//" will search over all children relative to the current node while "//" will search over all children, starting from the root node (=every node). It's all in here: https://www.w3.org/TR/xpath-10/ ;-) > Incase anyone else is dumb like me and manages to find this, I'll also > mention that > xpath1[0].xpath('.//table') > doesn't return anything, but Because there's no <table> element below (relative to) <table id="exampleTableToFind">: >>> xpath1[0].xpath('.//*') [<Element tbody at 0x7efc77fa39d0>, <Element tr at 0x7efc77fa3a70>, <Element th at 0x7efc77fa3ac0>] > xpath1[0].xpath('.//tr') > returns the one row that I want to see, while Because there's a <tr> child node, see above. > xpath1[0].xpath('//tr') > returns the rows from both tables And that's because below the root node there are two table nodes: >> xpath1[0].xpath('//*') [<Element html at 0x7efc77f3b7a0>, <Element head at 0x7efc77fa3890>, <Element body at 0x7efc77fa3b10>, <Element p at 0x7efc77fa3b60>, <Element table at 0x7efc77fa3bb0>, <Element tbody at 0x7efc77fa3c00>, <Element tr at 0x7efc77fa3c50>, <Element th at 0x7efc77fa3ca0>, <Element table at 0x7efc77fa3a20>, <Element tbody at 0x7efc77fa39d0>, <Element tr at 0x7efc77fa3a70>, <Element th at 0x7efc77fa3ac0>] Remember that an XPath starting with "/" will always search from the root node, even if the current context node >> xpath1[0].xpath('.') [<Element table at 0x7efc77fa3a20>] you start the search from (in this case <table id="exampleTableToFind">) is in the tree, below the root node. Cheers, Holger Landesbank Baden-Wuerttemberg Anstalt des oeffentlichen Rechts Hauptsitze: Stuttgart, Karlsruhe, Mannheim, Mainz HRA 12704 Amtsgericht Stuttgart HRA 4356, HRA 104 440 Amtsgericht Mannheim HRA 40687 Amtsgericht Mainz Die LBBW verarbeitet gemaess Erfordernissen der DSGVO Ihre personenbezogenen Daten. Informationen finden Sie unter https://www.lbbw.de/datenschutz.

1 0

Surprising behavior of lxml.html.tostring()
by Jens Tröger Feb. 9, 2026

Feb. 9, 2026

Hello, I’m a little puzzled by the behavior of the lxml.html.tostring() function, and would appreciate if somebody could shed some light on this. The test code is as follows: first we parse a small HTML document (derived from an actual real-world document!) s = """<?xml version="1.0" encoding="UTF-8"?> <!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.1//EN" "http://www.w3.org/TR/xhtml11/DTD/xhtml11.dtd"> <html xml:lang="en" xmlns="http://www.w3.org/1999/xhtml"> <head> </head> <body> </body> </html> """ This reads ok as XML: lxml.etree.XML(s.encode()) # <Element {http://www.w3.org/1999/xhtml}html at 0x10e837d00> lxml.etree.fromstring(s.encode()) # <Element {http://www.w3.org/1999/xhtml}html at 0x10e848980> and HTML: elm = lxml.html.fromstring(s.encode()) # <Element html at 0x10e7d00f0> root = elm.getroottree() root.docinfo.doctype # '<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.1//EN" "http://www.w3.org/TR/xhtml11/DTD/xhtml11.dtd”>' Serializing this back to HTML creates an unexpected string, though: lxml.html.tostring(elm.getroottree(), method="xml", encoding="unicode") Produces for lxml v5.3.0 <!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.1//EN" "http://www.w3.org/TR/xhtml11/DTD/xhtml11.dtd"> <?xml version="1.0" encoding="UTF-8"??><html xml:lang="en" xmlns="http://www.w3.org/1999/xhtml"> <head> </head> <body> </body> </html> and for lxml v6.0.2 <!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.1//EN" "http://www.w3.org/TR/xhtml11/DTD/xhtml11.dtd"> <html xml:lang="en" xmlns="http://www.w3.org/1999/xhtml"> <head> </head> <body> </body> </html> The latter parses ok with both lxml.etree.XML() and lxml.html.fromstring() whereas the former fails to parse as an XML file using lxml.etree.XML(). So it seem that *some* behavior was changed/fixed but I was unable to find that mentioned in the changelog. Both serialized documents, though, are different than the original in that the <!DOCTYPE> and <?XML?> elements are swapped, and removed/commented out entirely. Why? Also, is there a way to generate both elements in the original order? Much thanks! Jens

2 1

lxml.html.fromstring() doesn’t seem to get the doctype right?
by Jens Tröger Feb. 9, 2026

Feb. 9, 2026

Hello, Following from my previous post ( https://mail.python.org/archives/list/lxml@python.org/thread/NT7GNLORN676BM… ) I also noticed that reading an x/html file without doctype produces an incorrect/unexpected doctype. For example: b = b"""<?xml version="1.0" encoding="UTF-8”?> <html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en"></html> “"" parses ok into an element and element tree: elm = lxml.html.fromstring(b) # <Element html at 0x10fbea530> but the doctype for that document is — I believe — incorrect: root = elm.getroottree() root.docinfo.doctype # '<!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.0 Transitional//EN" "http://www.w3.org/TR/REC-html40/loose.dtd”>' Considering the xml declaration and the html element’s namespace, I would have expected the derived doctype to be <!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN” "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd”> for an xhtml file. Also, the DocInfo ( https://lxml.de/apidoc/lxml.etree.html#lxml.etree.DocInfo ) doesn’t actually denote whether the original document contained an xml declaration; wouldn’t a flag be useful? I ask because ideally round-tripping a document should produce that same document, but that is currently not the case: b = b"""<?xml version="1.0" encoding="UTF-8"?><html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en"></html>”"" elm = lxml.html.fromstring(b) # <Element html at 0x10fbea670> lxml.html.tostring(elm) # b'<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en"></html>' lxml.html.tostring(elm.getroottree()) # b'<!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.0 Transitional//EN" "http://www.w3.org/TR/REC-html40/loose.dtd">\n<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en"></html>’ lxml.html.tostring(elm.getroottree(), method="xml”) # b'<!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.0 Transitional//EN" "http://www.w3.org/TR/REC-html40/loose.dtd">\n<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en"/>' Cheers, Jens

1 0