Mailman 3 February 2026 - lxml - The Python XML Toolkit

running xpath query on result of an xpath query gives results which were excluded by the first xpath query
by lord of edges Feb. 18, 2026

Feb. 18, 2026

Will start by saying I am sure this is documented somehow and I probably just don't know the right word to find that documentation. I'm trying to query a table from a document, after which I will query details from that table. The issue I have is that the second xpath query is pulling results from the entire document. As an example, I have: ``` from lxml import html exampledocument=""" <head> example </head> <body> <table><tbody> <tr><th>exampleCellNotToFind</th> </tbody></table> <table id="exampleTableToFind"><tbody> <tr><th>exampleCellToFind</th> </tbody></table> </body> """ example=html.fromstring(exampledocument) xpath1=example.xpath('//table[@id="exampleTableToFind"]', smart_strings=False) xpath2=xpath1[0].xpath('//table', smart_strings=False) xpath3=html.fromstring(html.tostring(xpath1[0])).xpath('//table', smart_strings=False) ``` In this case, xpath2 includes both tables (xpath1 only includes 1 table), and I don't understand why. From reading "XPath return values" section of https://lxml.de/xpathxslt.html#xpath I thought smart_strings=True would the reason, but setting it to false didn't seem to change the output as far as I can tell. I do have a workable solution in this html.fromstring(html.tostring()), but thought I should ask here to try to understand how these element type variables are working. Thanks, for any help.

4 4

Re: running xpath query on result of an xpath query gives results which were excluded by the first xpath query
by Holger.Joukl＠LBBW.de Feb. 18, 2026

Feb. 18, 2026

Hi, > Thanks, that makes a little bit more sense. I had understood that // > was relative and / was absolute, [...] You either made a typo or you got that wrong. In XPath "." (leading dot) is relative, while "/" starts at the root node. Doubling the slash basically means "search all nodes below" i.e. ".//" will search over all children relative to the current node while "//" will search over all children, starting from the root node (=every node). It's all in here: https://www.w3.org/TR/xpath-10/ ;-) > Incase anyone else is dumb like me and manages to find this, I'll also > mention that > xpath1[0].xpath('.//table') > doesn't return anything, but Because there's no <table> element below (relative to) <table id="exampleTableToFind">: >>> xpath1[0].xpath('.//*') [<Element tbody at 0x7efc77fa39d0>, <Element tr at 0x7efc77fa3a70>, <Element th at 0x7efc77fa3ac0>] > xpath1[0].xpath('.//tr') > returns the one row that I want to see, while Because there's a <tr> child node, see above. > xpath1[0].xpath('//tr') > returns the rows from both tables And that's because below the root node there are two table nodes: >> xpath1[0].xpath('//*') [<Element html at 0x7efc77f3b7a0>, <Element head at 0x7efc77fa3890>, <Element body at 0x7efc77fa3b10>, <Element p at 0x7efc77fa3b60>, <Element table at 0x7efc77fa3bb0>, <Element tbody at 0x7efc77fa3c00>, <Element tr at 0x7efc77fa3c50>, <Element th at 0x7efc77fa3ca0>, <Element table at 0x7efc77fa3a20>, <Element tbody at 0x7efc77fa39d0>, <Element tr at 0x7efc77fa3a70>, <Element th at 0x7efc77fa3ac0>] Remember that an XPath starting with "/" will always search from the root node, even if the current context node >> xpath1[0].xpath('.') [<Element table at 0x7efc77fa3a20>] you start the search from (in this case <table id="exampleTableToFind">) is in the tree, below the root node. Cheers, Holger Landesbank Baden-Wuerttemberg Anstalt des oeffentlichen Rechts Hauptsitze: Stuttgart, Karlsruhe, Mannheim, Mainz HRA 12704 Amtsgericht Stuttgart HRA 4356, HRA 104 440 Amtsgericht Mannheim HRA 40687 Amtsgericht Mainz Die LBBW verarbeitet gemaess Erfordernissen der DSGVO Ihre personenbezogenen Daten. Informationen finden Sie unter https://www.lbbw.de/datenschutz.

1 0

Surprising behavior of lxml.html.tostring()
by Jens Tröger Feb. 10, 2026

Feb. 10, 2026

Hello, I’m a little puzzled by the behavior of the lxml.html.tostring() function, and would appreciate if somebody could shed some light on this. The test code is as follows: first we parse a small HTML document (derived from an actual real-world document!) s = """<?xml version="1.0" encoding="UTF-8"?> <!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.1//EN" "http://www.w3.org/TR/xhtml11/DTD/xhtml11.dtd"> <html xml:lang="en" xmlns="http://www.w3.org/1999/xhtml"> <head> </head> <body> </body> </html> """ This reads ok as XML: lxml.etree.XML(s.encode()) # <Element {http://www.w3.org/1999/xhtml}html at 0x10e837d00> lxml.etree.fromstring(s.encode()) # <Element {http://www.w3.org/1999/xhtml}html at 0x10e848980> and HTML: elm = lxml.html.fromstring(s.encode()) # <Element html at 0x10e7d00f0> root = elm.getroottree() root.docinfo.doctype # '<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.1//EN" "http://www.w3.org/TR/xhtml11/DTD/xhtml11.dtd”>' Serializing this back to HTML creates an unexpected string, though: lxml.html.tostring(elm.getroottree(), method="xml", encoding="unicode") Produces for lxml v5.3.0 <!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.1//EN" "http://www.w3.org/TR/xhtml11/DTD/xhtml11.dtd"> <?xml version="1.0" encoding="UTF-8"??><html xml:lang="en" xmlns="http://www.w3.org/1999/xhtml"> <head> </head> <body> </body> </html> and for lxml v6.0.2 <!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.1//EN" "http://www.w3.org/TR/xhtml11/DTD/xhtml11.dtd"> <html xml:lang="en" xmlns="http://www.w3.org/1999/xhtml"> <head> </head> <body> </body> </html> The latter parses ok with both lxml.etree.XML() and lxml.html.fromstring() whereas the former fails to parse as an XML file using lxml.etree.XML(). So it seem that *some* behavior was changed/fixed but I was unable to find that mentioned in the changelog. Both serialized documents, though, are different than the original in that the <!DOCTYPE> and <?XML?> elements are swapped, and removed/commented out entirely. Why? Also, is there a way to generate both elements in the original order? Much thanks! Jens

2 1

lxml.html.fromstring() doesn’t seem to get the doctype right?
by Jens Tröger Feb. 9, 2026

Feb. 9, 2026

Hello, Following from my previous post ( https://mail.python.org/archives/list/lxml@python.org/thread/NT7GNLORN676BM… ) I also noticed that reading an x/html file without doctype produces an incorrect/unexpected doctype. For example: b = b"""<?xml version="1.0" encoding="UTF-8”?> <html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en"></html> “"" parses ok into an element and element tree: elm = lxml.html.fromstring(b) # <Element html at 0x10fbea530> but the doctype for that document is — I believe — incorrect: root = elm.getroottree() root.docinfo.doctype # '<!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.0 Transitional//EN" "http://www.w3.org/TR/REC-html40/loose.dtd”>' Considering the xml declaration and the html element’s namespace, I would have expected the derived doctype to be <!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN” "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd”> for an xhtml file. Also, the DocInfo ( https://lxml.de/apidoc/lxml.etree.html#lxml.etree.DocInfo ) doesn’t actually denote whether the original document contained an xml declaration; wouldn’t a flag be useful? I ask because ideally round-tripping a document should produce that same document, but that is currently not the case: b = b"""<?xml version="1.0" encoding="UTF-8"?><html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en"></html>”"" elm = lxml.html.fromstring(b) # <Element html at 0x10fbea670> lxml.html.tostring(elm) # b'<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en"></html>' lxml.html.tostring(elm.getroottree()) # b'<!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.0 Transitional//EN" "http://www.w3.org/TR/REC-html40/loose.dtd">\n<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en"></html>’ lxml.html.tostring(elm.getroottree(), method="xml”) # b'<!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.0 Transitional//EN" "http://www.w3.org/TR/REC-html40/loose.dtd">\n<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en"/>' Cheers, Jens

1 0