lxml - The Python XML Toolkit September 2022

lxml@python.org

5 participants
9 discussions

[lxml-dev] Checking whether a node is a comment/element
by Geoffrey Sneddon 11 Jan '23

11 Jan '23

Hi, What's the best way to check whether a given node is a comment or an element? For the former, I'm currently using isinstance(node, etree._Comment), which is rather obviously sub-optimal. -- Geoffrey Sneddon <http://gsnedders.com/>

4 4

[lxml-dev] lxml 2.0.5 released
by Stefan Behnel 11 Jan '23

11 Jan '23

Hi all, lxml 2.0.5 is on PyPI. This is a bug-fix-only release of the stable 2.0 series. Have fun, Stefan 2.0.5 (2008-05-01) Bugs fixed * Resolving to a filename in custom resolvers didn't work. * lxml did not honour libxslt's second error state "STOPPED", which let some XSLT errors pass silently. * Memory leak in Schematron with libxml2 >= 2.6.31.

3 4

Question about inheritance in cssselect.py
by Dani Litovsky Alcala 23 Dec '22

23 Dec '22

lxml v.4.9 cssselect.py:CSSSelector.__init__ calls on `etree.XPath.__init__(self, path, namespaces=namespaces)` to initialize the parent class. Is there a reason why `super()` or even `super(CSSSelector, self)__init__...` is not used? I bring this up as if I attempt to monkey patch (private project) `etree.XPath.__init__` the current code causes an error ``` TypeError: super(type, obj): obj must be an instance or subtype of type ``` while replacing it with the suggested use of `super()` fixes my error.

3 3

can I tell lxml to ignore xmlids?
by Martin Mueller 31 Oct '22

31 Oct '22

Duplicate xmlids have way of creeping into my 60,000 documents. The ids keep the document from parsing, which is helpful in drawing attention to errors, but it makes it harder to correct the errors. I work with documents in the TEI namespace, and I have a very kludgy workaround: I comment out the reference to the schema and change ‘xml:id ‘ to ‘xmlom’. Then I can loop through the document and fix errors with a script. There must be a more elegant way to do this. Is there away of telling lxml: “never mind the duplicate IDs. Just carry on”. Then I can toggle between a script that cares or doesn’t care about duplicate IDS. With thanks in advance for any help Martin Mueller Professor emeritus of English and Classics Northwestern University

4 4

Turn three-line block into single?
by Gilles 31 Oct '22

31 Oct '22

Hello, Before I resort to a regex, I figured I should ask here. To find and remove possible duplicates, I need to turn each block into a single line: FROM <wpt lat="46.98520" lon="6.8831"> <name>blah</name> </wpt> TO <wpt lat="46.98520" lon="6.8831"><name>blah</name></wpt> Do you know of a way to do this in lxml? Thank you.

6 20

Incremental XML generation questions
by matias.wargelin＠gmail.com 22 Sep '22

22 Sep '22

Hi! I'm interested in incremental XML generation. I have two questions: 1. Is it somehow possible to first create an element, then open it with context manager? The reasoning for this is that I have pretty complicated elements and I'm using a helper library to create them, so it would be useful to just get the element from the helper library and start writing to it. I'm thinking something like this, but this does not work: f = BytesIO() with etree.xmlfile(f) as xf: element = helper_library.create_root_element(...) with element: xf.write('text') 2. Is it possible to clean up unused namespaces from an element when generating XML incrementally? The problem is that the lxml.etree.cleanup_namespaces function wants the element as an argument, but I have not come up with a way to access the element with incremental XML generation: f = BytesIO() with etree.xmlfile(f) as xf: # How to access the element created below? # with xf.element('abc') as element does not seem to work with xf.element('abc'): xf.write(...) # Now I have nothing to give to this function: lxml.etree.cleanup_namespaces(tree_or_element=":(")

1 1

[lxml-dev] lxml has its page on launchpad
by Stefan Behnel 21 Sep '22

21 Sep '22

Hi all, I added the lxml project to launchpad, the Ubuntu Bug-Tracker. It also has a FAQ engine and a couple of other goodies. https://launchpad.net/lxml It's easy to sign up for launchpad, BTW, no 90%-footnotes-contract. Have fun, Stefan

3 2

html diff changes the structure of the html (and a mystery about debugging)
by Steve 12 Sep '22

12 Sep '22

Hi, I'm using lxml's html diff functionality in a project and it has been working well so far. Sometime back though I noticed that the generated diff changes the structure of the html in a manner that's less than ideal for our use case. An example: >>> from lxml.html import diff >>> a = "<div id='first'>some old text</div><div id='last'>more old text</div>" >>> b = "<div id='first'>some old text</div><div id='middle'>and new text</div><div id='last'>more old text</div>" >>> diff.htmldiff(a, b) ('<div id="middle"> <div id="first"><ins>some old text</ins></div><ins>and new</ins> <del>some old</del> text</div><div id="last">more old ' 'text</div>') >>> As you can see, the div with id=middle has been inserted at the beginning of the document and it encloses the div with id=first. I believe this happens because lxml unconditionally inserts 'unbalanced tags' at the beginning of set of 'chunks' when it surrounds the chunks with the <ins> tags: https://github.com/lxml/lxml/blob/master/src/lxml/html/diff.py#L241 Could we potentially be a bit smarter about this and insert the unbalanced tags, as we encounter them instead ? For instance, closing out the opened `<ins>` tag, inserting the unbalanced tag and opening a new `<ins>` tag. Something like: Secondly, it would be great if someone could help me understand (or give me pointers to) why I'm seeing this differences of behaviour between what's being returned from the compiled version of the code and executing the exact same set of functions from the REPL !?! (assuming the same context in the REPL as above) >>> tokens_a = diff.tokenize(a) >>> tokens_b = diff.tokenize(b) >>> diff.htmldiff_tokens(tokens_a, tokens_b) ['<div id="middle"> ', '<ins>', '<div id="first">', 'some ', 'old ', 'text', '</div>', 'and ', 'new', '</ins> ', '<del>', 'some ', 'old', '</del> ', 'text', '</div>', '<div id="last">', 'more ', 'old ', 'text', '</div>'] >>> s = diff.InsensitiveSequenceMatcher(tokens_a, tokens_b) >>> commands = s.get_opcodes() >>> list(commands) [('delete', 0, 9, 0, 0)] The htmldiff_tokens() is obviously getting 2 opcodes, either an insert or replace, followed by a delete, but if I instantiate InsensitiveSequenceMatcher() from the REPL, it generates only the delete ! This is driving me nuts ! Am I doing something wrong ? cheers, Steve

1 1

lxml build system for pypi downloads of it
by Kacvinsky, Tom 05 Sep '22

05 Sep '22

Hi, What build system is being used to make the wheels for lxml, ones that can be installed by pip? I ask because I want to see if some performance issues I am seeing that seem to be tied to the version of libxml2 and lxml. I want to see if I am building from source the same way the wheels available via pip are built, so I am thinking I can look at a build configuration - compiler, glibc, Linux distro, compiler/linker optimizations, version of cython used - if lxml wheels are built using cython, etc... Thanks and regards, Tom

2 1

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

2005

2004

lxml - The Python XML Toolkit September 2022