lxml - The Python XML Toolkit March 2022

lxml@python.org

7 participants
15 discussions

[lxml-dev] Checking whether a node is a comment/element
by Geoffrey Sneddon 11 Jan '23

11 Jan '23

Hi, What's the best way to check whether a given node is a comment or an element? For the former, I'm currently using isinstance(node, etree._Comment), which is rather obviously sub-optimal. -- Geoffrey Sneddon <http://gsnedders.com/>

4 4

[lxml-dev] lxml 2.0.5 released
by Stefan Behnel 11 Jan '23

11 Jan '23

Hi all, lxml 2.0.5 is on PyPI. This is a bug-fix-only release of the stable 2.0 series. Have fun, Stefan 2.0.5 (2008-05-01) Bugs fixed * Resolving to a filename in custom resolvers didn't work. * lxml did not honour libxslt's second error state "STOPPED", which let some XSLT errors pass silently. * Memory leak in Schematron with libxml2 >= 2.6.31.

3 4

[lxml-dev] lxml has its page on launchpad
by Stefan Behnel 21 Sep '22

21 Sep '22

Hi all, I added the lxml project to launchpad, the Ubuntu Bug-Tracker. It also has a FAQ engine and a couple of other goodies. https://launchpad.net/lxml It's easy to sign up for launchpad, BTW, no 90%-footnotes-contract. Have fun, Stefan

3 2

[lxml-dev] Building LXML Trunk
by Sidnei da Silva 31 Aug '22

31 Aug '22

Hi, I've tried to build lxml from trunk today, on Win32. Got the following error: src\lxml\etree.c(880) : error C2059: syntax error : ')' src\lxml\etree.c(881) : error C2059: syntax error : ')' src\lxml\etree.c(882) : error C2059: syntax error : ')' src\lxml\etree.c(883) : error C2059: syntax error : ')' Any clue? Smells like a Pyrex issue? -- Sidnei da Silva Enfold Systems http://enfoldsystems.com Fax +1 832 201 8856 Office +1 713 942 2377 Ext 214

4 4

Tested value modified in validation error message
by Bob Kline 16 Jul '22

16 Jul '22

Before I file a ticket, I wanted to check here to find out if this is a known problem (I searched the bug tracker, but didn't see any matches). Our customer is using a commercial XML editor and the editor's implementation of the DOM API sometimes garbles whitespace inside text nodes when serializing XML documents for the document's xml property. This causes schema validation errors when we send that serialization off to be validated against the document type's schema using lxml. The problem we have run into with lxml is that the message property of the error object which comes back in the schema object's error_log has altered the value being validated. So, for example, if the text content of an element in the serialized DOM is "Processing\ncomplete" and the value it was supposed to match in the enumerated valid values is "Processing complete" then the message attribute for the corresponding object in the error_log has Element 'ProcessingStatusValue': [facet 'enumeration'] The value 'Processing complete' is not an element of the set {'Ready for English peer review', 'Ready for English scientific review', 'Ready for English OCCM review', 'Ready for Spanish peer review', 'Ready for Spanish OCCM review', 'Ready for publishing', 'Ready for translation', 'Processing complete'}. In effect saying "the value 'Processing complete' does not match the value 'Processing complete'." Took a very long time for us to figure out that the error message was wrong, and that it should have said ... The value 'Processing\ncomplete' is not an element of the set .... -- Bob Kline https://www.rksystems.com mailto:bkline@rksystems.com

4 10

reconstruct a dtd content model
by Robin Becker 20 Mar '22

20 Mar '22

I wonder if anybody has code to reconstruct the content definition of a DTD element. It's needed for some automatic documentation process. I inherited a recursive code, but it's clear that it's doing the wrong thing; it's fairly obvious that the parentheses are being done wrongly. I think that parens ought to go before the occur processing at "if node.occur == 'plus':", but it's also obvious that they are not always required. > content reconstruction test > '<!ELEMENT a (b?)>' content=( b ?) > '<!ELEMENT a (a|b)+>' content=( a | b +) > '<!ELEMENT a (a|b|c)+>' content=( a | b | c +) > '<!ELEMENT a (a|b?|c)+>' content=( a | b | c *) > '<!ELEMENT a (z)>' content=( z ) > '<!ELEMENT a (#PCDATA)>' content=(#PCDATA) > '<!ELEMENT a (#PCDATA|b)*>' content=(#PCDATA | b *) > '<!ELEMENT a (a,b,c)*>' content=( a , b , c *) > '<!ELEMENT a ANY>' content=ANY > '<!ELEMENT a EMPTY>' content=EMPTY The code I have is as follows def elementContent(node): return node.name def _contentRecur(node, elFmt=elementContent): """ node.type: ("element" | "seq") node.occur: ("once" | "opt") none.name: (str | None) """ s = "" if node is None: return s t = node.type left = node.left right = node.right if t == 'or': if left and right: s = f"{_contentRecur(left,elFmt=elFmt)} | {_contentRecur(right,elFmt=elFmt)}" elif left is not None: s = _contentRecur(left,elFmt=elFmt) elif right is not None: s = _contentRecur(right,elFmt=elFmt) elif t == 'seq': if left and right: s = f"{_contentRecur(left,elFmt=elFmt)} , {_contentRecur(right,elFmt=elFmt)}" elif left is not None: s = _contentRecur(left,elFmt=elFmt) elif right is not None: s = _contentRecur(right,elFmt=elFmt) elif t == "element": s = f" {elFmt(node)} " elif t=='pcdata': s = '#PCDATA' if node.occur == 'plus': s += "+" elif node.occur == 'opt': s += '?' elif node.occur=='mult': s += '*' return s -- Robin Becker

1 1

Re: lxml.objectify2: More fun, namespaces, pythonic
by Holger.Joukl＠LBBW.de 16 Mar '22

16 Mar '22

Hi, I've deliberately waited a good week before replying, but part of this just still sits wrong with me. See below. On 08.03.2022 at 12:26 Volker wrote: > I learned that some members of the list were annoyed by my posts. My aim was never to disrespect > any person nor their work for lxml. I you feel disrespected I am truely sorry for that. Appreciated! Thanks for clarifying. > But I am still sure that lxml.objectify is not the perfect solution. lxml.objectify has its advantages, for instance > in simplicity and dealing with non-namespace usecases. And I think we can agree that there is no perfection > in all usecase to be gained. > lxml.o has its usecases and lxml.o2 will have its usecases. In any of my posts I have pointed out that lxml.o > should not be replaced by lxml.o2, and that I opt for a coexistence. I also pointed out that I respected your > code - I not even touched it. > If something is not perfect it is limited in a way. To motivate my work I find it quite legitimate to point out the limitations > of lxml.objectify. I addressed the limitiations of lxml.o2 already in my last post and I am sure there will be more surfacing > before I have finalized it. It's absolutely fine and helpful to point out limitations and potential enhancements. Still, there's the question on how you present that. In other words, it's not what you say, but how you say it. > And a few last words on the reception of the ML from my perspective. I had right from the start the feeling that my > ideas were not really taken seriously. I can only speak for myself, but why would I reply in the 1st place if I did not take an idea seriously? Re seriously: several propositions have been made (like potential enhanced/extended __dict__ behavior, using iterchildren() in the IDE or teaching the IDE to provide more helpful output) but remained unanswered from your side, IMHO. > Many negative arguments like "it was so since 2006" or "it cannot work" were > brought up. I'm not quite sure what a "negative argument" is supposed to mean. But as far as I'm concerned it's valuable information that software has a certain legacy (read: there's history + existing design decisions, with trade-offs, plus compatibility concerns to think about). And I don’t remember anyone saying that "it cannot work". What I sure did say is that I don't find the proposed changed ways convincing. Which is still the case, btw. > Even the "but I am strongly biased" club aka "I am one of the developer/maintainer" was waved. I don't get that at all. Revealing that I'm biased is simply a disclosure that I've quite solid convictions about the way things work, and you'd better have very strong arguments to convince me otherwise. I can't wave any maintainer flag either (because I'm not ;-)). And disclosing that I was involved in lxml.objectify's "birth" is just this very same full disclosure that I'm not a neutral party. > Now I would like to come back onto the rational plane and do constructive work together. I take it that's probably not what you meant but to bring some perspective: To me, the wording just sounds an awful lot like we (as in I, or Stefan) have not acted rational, or constructive, in the meantime, in contrast to you. And that's simply not the case. Holger Landesbank Baden-Wuerttemberg Anstalt des oeffentlichen Rechts Hauptsitze: Stuttgart, Karlsruhe, Mannheim, Mainz HRA 12704 Amtsgericht Stuttgart HRA 4356, HRA 104 440 Amtsgericht Mannheim HRA 40687 Amtsgericht Mainz Die LBBW verarbeitet gemaess Erfordernissen der DSGVO Ihre personenbezogenen Daten. Informationen finden Sie unter https://www.lbbw.de/datenschutz.

1 0

Compatibility issues between `lxml.etree.set_element_class_lookup`` and `lxml.html`
by Xavier Morel 08 Mar '22

08 Mar '22

Hello, Sorry for the bother, but I've been looking at `lxml.etree.set_element_class_lookup`[0] as a way to add validation and features to lxml usage without having to ban "standard" lxml constructs (and to control usage by dependencies as well). However while it seems to work fine for etree itself it interacts quite badly with both lxml.html and objectify: >>> from lxml import etree, html, objectify >>> class Custom(etree.ElementBase): ... ... ... >>> etree.set_element_class_lookup(etree.ElementDefaultClassLookup(element=Custom)) >>> >>> print(type(html.fromstring('<a>'))) <class '__main__.Custom'> >>> print(type(objectify.fromstring('<a/>'))) <class '__main__.Custom'> As can be seen here, both the html parser and the objectify parser seem to complete lose their "magic". Is there a "proper" way to make these things collaborate? I looked at lxml.html and it looked like it might have to be rebuilt from the HTMLMixin (which already seems icky) but `objectify` is a cython module so there doesn't seem to be a good way to interact with it. [0] https://lxml.de/3.1/api/public/lxml.etree-module.html#set_element_class_loo…

3 4

lxml.objectify2: More fun, namespaces, pythonic
by Dr. Volker Jaenisch 08 Mar '22

08 Mar '22

Dear XLML Users! I am developing lxml.objectify2(lxml.o2). Lxml.o2 has tree objectives: * making lxml more pythonic * introducing robust namespaced properties * making lxml more fun *Following the old ways** * Imagine the following xml file. xml_str = '''\ <obj:root xmlns:obj="objectified" xmlns:other="otherNS"> <obj:c1 a1="A1" a2="A2" other:a3="A3"> <obj:c2>0</obj:c2> <obj:c2>1</obj:c2> <obj:c2>2</obj:c2> </obj:c1> <obj:c1> <other:c2>3</other:c2> <other:c2>5</other:c2> <obj:c2>2</obj:c2> </obj:c1> <obj:c1> <other:c2>42</other:c2> </obj:c1> </obj:root>''' Please notice that the tags obj:c1 and obj/other:c2 are multiple childs of the same {ns}name. Here a glance at the data processed by xlml.o (standard lxml.objectfy) from the PyCharm IDE perspective. https://backend.datenadler.de/kram/bildschirmfoto-vom-2022-03-07-23-02-33.p… You may notice that there is no multiplicity at all. lxml.o is quite limited and not really pythonic. Therefore any Python-IDE will struggles with a representation of lxml processed data. *Following the new ways* Let's use lxml.objectify2 instead. from lxml.objectify2 import ObjectifiedElement2 obj2_lookup = ObjectifyElementClassLookup(tree_class=ObjectifiedElement2) parser = etree.XMLParser() parser.set_element_class_lookup(obj2_lookup) node = etree.XML(xml_str, parser=parser) A look from the PyCharm debugger into the data structure processed by lxml.o2: https://backend.datenadler.de/kram/bildschirmfoto-vom-2022-03-07-22-34-10.p… As you can see lxml.o2 handles multiple children with same qtag by assigning an "[index]" to them. *<rant>Yeah, that is nice screenwork, but this will never work in code?**</rant>* >>> node.obj_c1[2].obj_c2 [3] here the call to node.obj_c1 returns a list. Then python takes over get the desired second element. *<rant>Ok, but this will not work with getattr**</rant>* >>> getattr(node, 'obj_c1[0]').obj_c2 [0, 1, 2] Here lxml.o2 does the selection of the element [0] really fast in c-space. ** *<rant>OK, and where is the catch**</rant>* To implement this functionality we need to ensure that two rules are followed by the user. 1) If there are elements without a namespace, a default namespace has to be defined. 2) Any access to a "tag" has to be done qualified, with the exception of the default namespace. node.<namespace>_<name> mit default namespace node.<name> If these rules a too much for you, go back to lxml.objectify and be happy. *<rant>Ah, go away. Where do you find such nice XML**</rant>* Mh. I have never seen so simple XML documents like in the lxml.objectify tests in the real world. But I am aware that lxml.o2 will have to be tested thoroughly. * * *<rant>You will never convince all the users of lxml to change to lxml.o2**</rant>* That is true. But I do not even try. lxml.o2 is an alternative to lxml.o for certain usecases. You are welcome to rant at me :-) You are also welcome to help with the development of lxml.o2. This is a spare time job for me. If you do not have the time to help, you may express your liking of lxml.o2, here. lxml.o2 lives at https://github.com/Inqbus/lxml <https://github.com/Inqbus/lxml> in the branch https://github.com/Inqbus/lxml/tree/objectify_prefix Cheers, Volker -- ========================================================= inqbus Scientific Computing Dr. Volker Jaenisch Hungerbichlweg 3 +49 (8860) 9222 7 92 86977 Burggenhttps://inqbus.de =========================================================

1 1

Is there an ElementTree class lookup hook?
by Xavier Morel 08 Mar '22

08 Mar '22

lxml provides support for custom Element classes (as well as element-ish e.g. Comment or PI) via the `ElementDefaultClassLookup` registry, and the ability to hook it into a parser. But that registry does not seem to have a slot for the root tree of the elements. Is there a hook somewhere to set *that*? I tried looking around the API docs but nothing really jumped out. PS: the documentation for `set_default_parser` explains that it sets the default parser *for the current thread* and that "You can create a separate parser for each thread explicitly or use a parser pool.", does it mean that in a "don't call any API which gets an implicit parser and manage your parsers by hand" sense or something else?

2 1

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

2005

2004

lxml - The Python XML Toolkit March 2022