XML namespaces are not propagated over from the ancestor elements when using find* methods
I have a XML document that has namespaces in it. I want to use the find* methods to select elements but it's looks like it is not possible without specifying namespaces explicitly to every call. Is this true? Is this by design? This seems very burdensome to do so when namespaces are included in the XML document. It would be really nice if the namespaces in the XML document could be considered. xml = b"""<?xml version="1.0" encoding="UTF-8"?><root xmlns="urn:defaultnamespace"> <Timestamp>2025-01-09T17:46:08.766Z</Timestamp> <namespaced xmlns:ns="urn:subns"> <rootdefault>text</rootdefault> <ns:element>element</ns:element> </namespaced></root>"""from lxml import etreexml_doc = etree.fromstring(xml)print(xml_doc)print("namespaces: " + str(xml_doc.nsmap))print(f"find root without namespace {xml_doc.find(".")}")print(f"find root with namespace {xml_doc.find(".", namespaces = {**xml_doc.nsmap})}")print(f"find Timestamp without namespace {xml_doc.find("./Timestamp")}")print(f"find Timestamp with namespace {xml_doc.find("./Timestamp", namespaces = {**xml_doc.nsmap})}")print(f"find namespaced without namespace {xml_doc.find("./namespaced")}")print(f"find namespaced with namespace {xml_doc.find("./namespaced", namespaces = {**xml_doc.nsmap})}")print(f"find element with namespace {xml_doc.find("./namespaced/rootdefault")}")print(f"find rootdefault with namespace {xml_doc.find("./namespaced/rootdefault", namespaces = {**xml_doc.nsmap})}")print(f"find namespaced no namespace {xml_doc.find("./namespaced/element")}")try: print(f"find namespaced default namespace only {xml_doc.find("./namespaced/ns:element", namespaces = {None: "urn:defaultnamespace"})}")except SyntaxError as e: print(f"find namespaced default namespace only error {e}")print(f"find namespaced ns namespace only {xml_doc.find("./namespaced/ns:element", namespaces = {"ns": "urn:subns"})}")print(f"find namespaced with full namespace {xml_doc.find("./namespaced/ns:element", namespaces = {None: "urn:defaultnamespace", "ns": "urn:subns"})}")
Hi,
I have a XML document that has namespaces in it. I want to use the find* methods to select elements but it's looks like it is not possible without specifying namespaces explicitly to every call. Is this true? Is this by design? This seems very burdensome to do so when namespaces are included in the XML document. It would be really nice if the namespaces in the XML document could be considered.
IMHO the way to go is to just predefine a prefix namespace mapping that fits the prefixes you use in your find/xpath expressions, once. Then simply use this as your xpath/find argument. E.g.
from lxml import etree xml = b"""<?xml version="1.0" encoding="UTF-8"?> ... <root xmlns="urn:defaultnamespace"> ... <Timestamp>2025-01-09T17:46:08.766Z</Timestamp> ... <namespaced xmlns:ns="urn:subns"> ... <rootdefault>text</rootdefault> ... <ns:element>element</ns:element> ... </namespaced> ... </root> ... """
root = etree.fromstring(xml) # subsequently use this ns mapping: namespaces = {"default": "urn:defaultnamespace", "sub": "urn:subns"} root.find("./default:namespaced/sub:element", namespaces=namespaces) <Element {urn:subns}element at 0x7e14867f4dc0>
or
root.xpath("./default:namespaced/sub:element", namespaces=namespaces) [<Element {urn:subns}element at 0x7e14867f4dc0>]
Not burdensome in my book. ;-) Note that I deliberately deviate from the prefixes used in the original docs here, just for illustration. So you don't really need to know about the prefixes used in the document you want to process beforehand - but of course you need to know the qualified names for your find/xpath expressions (i.e. "{namespace-uri}element-name" in Clark notation). For XPath, you can't use an empty prefix; see also https://lxml.de/ xpathxslt.html#namespaces-and-prefixes. You might even want to "precompile" xpath expressions using etree.XPath, like
namespaces = {"default": "urn:defaultnamespace", "sub": "urn:subns"} find_element = etree.XPath("./default:namespaced/sub:element", namespaces=namespaces) find_element(root) [<Element {urn:subns}element at 0x7e14868127c0>]
If you really wanted to you could do some functools.partial currying to create your own namespace map-aware find functions:
import functools find = functools.partial(root.__class__.find, namespaces={None: "urn:defaultnamespace", "ns": "urn:subns"}) find(root, "./namespaced/ns:element") <Element {urn:subns}element at 0x7e148686cb80>
If you wanted to use unqualified names you could do s.th. like
root.xpath("./*[local-name()='namespaced']/*[local-name()='element']") [<Element {urn:subns}element at 0x7e14867f4dc0>]
But I wouldn't advise it: it's clunky in XPath 1.0 anyhow and has performance implications. I'd just go with the simplest option i.e. define and reuse a namespaces dict. Best regards, Holger
Hi, Matthew Ouyang schrieb am 10.01.25 um 20:08:
It would be really nice if the namespaces in the XML document could be considered.
I read this request quite often. It's in the FAQs: https://lxml.de/FAQ.html#how-can-i-find-out-which-namespace-prefixes-are-use... Namespace prefixes in the document can freely be chosen by the document creator/provider. They are really a pure serialisation detail and should not be considered fixed or useful for any kind of processing besides plain parsing. Stefan
Thank you Holger and Stefan for your feedback. I needed to pull several elements from an XML document that had namespace and create a new XML document based on the pulled elements. This meant multiple find* calls and lost time in debugging when I forgot to include the namespace parameter. Because I used etree.fromstring, it provided the namespaces included in the original XML and it appeared to be a nice idea to use those prefixes. However that was merely a convenience and it does not apply universally. The namespace is what matters and I can define whatever prefixes that I want. I hope I processed your feedback correctly. Please let me know if I have not.
participants (3)
-
jholg＠gmx.de -
Matthew Ouyang -
Stefan Behnel