lxml - The Python XML Toolkit March 2019

lxml@python.org

5 participants
9 discussions

[lxml-dev] Checking whether a node is a comment/element
by Geoffrey Sneddon 11 Jan '23

11 Jan '23

Hi, What's the best way to check whether a given node is a comment or an element? For the former, I'm currently using isinstance(node, etree._Comment), which is rather obviously sub-optimal. -- Geoffrey Sneddon <http://gsnedders.com/>

4 4

[lxml-dev] lxml 2.0.5 released
by Stefan Behnel 11 Jan '23

11 Jan '23

Hi all, lxml 2.0.5 is on PyPI. This is a bug-fix-only release of the stable 2.0 series. Have fun, Stefan 2.0.5 (2008-05-01) Bugs fixed * Resolving to a filename in custom resolvers didn't work. * lxml did not honour libxslt's second error state "STOPPED", which let some XSLT errors pass silently. * Memory leak in Schematron with libxml2 >= 2.6.31.

3 4

[lxml-dev] lxml has its page on launchpad
by Stefan Behnel 21 Sep '22

21 Sep '22

Hi all, I added the lxml project to launchpad, the Ubuntu Bug-Tracker. It also has a FAQ engine and a couple of other goodies. https://launchpad.net/lxml It's easy to sign up for launchpad, BTW, no 90%-footnotes-contract. Have fun, Stefan

3 2

[lxml-dev] Building LXML Trunk
by Sidnei da Silva 31 Aug '22

31 Aug '22

Hi, I've tried to build lxml from trunk today, on Win32. Got the following error: src\lxml\etree.c(880) : error C2059: syntax error : ')' src\lxml\etree.c(881) : error C2059: syntax error : ')' src\lxml\etree.c(882) : error C2059: syntax error : ')' src\lxml\etree.c(883) : error C2059: syntax error : ')' Any clue? Smells like a Pyrex issue? -- Sidnei da Silva Enfold Systems http://enfoldsystems.com Fax +1 832 201 8856 Office +1 713 942 2377 Ext 214

4 4

fomrating lxml output
by Martin Mueller 31 Mar '19

31 Mar '19

Could the documentation of lxml include some explanation and examples of how to format output? There is very little of it in the current documentation, and I haven’t found anything about it in Shipman’s otherwise excellent side. There is something about it in Frederick Lund’s ElementTree site, but I can’t quite make sense of it, and since lxml sems to have established itself as the go-to Python place for munging XML, it should have self-contained documentation about every aspect of that munging. lxml has a “pretty-Print=True” keyword. It would be better if the ‘magic’ of it were explained and it were modifiable. It works for most of my purposes, but has some quirks. My files are all text files where every token is a <w> element. Sometimes I split an existing token and create a new token using ‘addnext’ to insert it. Whenever I do that, pretty_print doesn’t put the new element in a new line. This puzzles me because, as far as I know, pretty_print operates on a tree when it is written out. How would it know or care whether a <w> element has been in that tree forever or has just been added? Many formatting routines seem to rely on the number of nestings, adding a space or tab character for every additional level. This does not make a whole lot of sense for linguistically annotated texts where every token is a leaf node wrapped in a <w> or <pc> element and that leaf node may be eight or more levels below the top level. nor does it make sense in that environment to distinguish between inline and block elements for the purpose of formatting. The simplest, most economic, and most readable formatting would take the form of saying: 1. Start all new elements at the left margin, except for <pc> and <w> 2. Indent <pc> and <w> by two space Or alternately: 1. Put all <w> and <pc> elements at the beginning of a line 2. Indent all other elements by two spaces I’d be very grateful for any advice on how to do this. And I think it would be a good thing if the lxml documentation included a section on formatting choices, preferably with an explanation and examples.

3 2

_XSLTResultTree.write_output is documented to write to file but it writes to URI
by Jan Pazdziora 28 Mar '19

28 Mar '19

Hello, the documentation of _XSLTResultTree.write_output states write_output(self, file, *, compression=0) Serialise the XSLT output to a file or file-like object. As opposed to the generic .write() method, .write_output() serialises the result as defined by the <xsl:output> tag. The problem is that it's actually not file which is being written here, even if plain string is passed. The code in write_output is if _isString(file): file_path = _encodeFilename(file) c_filename = _cstr(file_path) with nogil: r = xslt.xsltSaveResultToFilename( c_filename, doc._c_doc, self._xslt._c_style, c_compression) and xsltSaveResultToFilename is documented at http://xmlsoft.org/XSLT/html/libxslt-xsltutils.html#xsltSaveResultToFilename as int xsltSaveResultToFilename(const char * URL, xmlDocPtr result, xsltStylesheetPtr style, int compression) Save the result @result obtained by applying the @style stylesheet to a file or @URL URL: a filename or URL The net effect is that the filename passed to _XSLTResultTree.write_output is URI-unescaped, potentiall resulting in different file being written. Consider the following small change to the test suite: diff --git a/src/lxml/tests/test_xslt.py b/src/lxml/tests/test_xslt.py index 96eb83ee..3e65bde5 100644 --- a/src/lxml/tests/test_xslt.py +++ b/src/lxml/tests/test_xslt.py @@ -170,12 +170,12 @@ class ETreeXSLTTestCase(HelperTestCase): else: self.assertTrue(False, "exception not raised") - def test_xslt_write_output_file(self): + def test_xslt_write_output_file_escape(self): with self._xslt_setup() as res: - f = NamedTemporaryFile(delete=False) + f = NamedTemporaryFile(prefix="tmp%2ejezek", delete=False) try: try: - res[0].write_output(f) + res[0].write_output(f.name) finally: f.close() with io.open(f.name, encoding='UTF-16') as f: Instead of passing in the file-like object, we pass in the file name, which would be something like /tmp/tmp%2fjezek8chac7f7. However, the tests then start to fail with Doctest: xpathxslt.txt ====================================================================== FAIL: test_xslt_write_output_file_escape (lxml.tests.test_xslt.ETreeXSLTTestCase) ---------------------------------------------------------------------- Traceback (most recent call last): File "/usr/lib64/python3.7/unittest/case.py", line 60, in testPartExecutor yield File "/usr/lib64/python3.7/unittest/case.py", line 632, in run testMethod() File "/home/adelton/project/lxml/src/lxml/tests/test_xslt.py", line 184, in test_xslt_write_output_file_escape os.unlink(f.name) File "/usr/lib64/python3.7/contextlib.py", line 119, in __exit__ next(self.gen) File "/home/adelton/project/lxml/src/lxml/tests/test_xslt.py", line 132, in _xslt_setup self.assertEqual(expected, data[0].replace('\n', '')) File "/usr/lib64/python3.7/unittest/case.py", line 856, in assertEqual assertion_func(first, second, msg=msg) File "/usr/lib64/python3.7/unittest/case.py", line 1237, in assertMultiLineEqual self.fail(self._formatMessage(msg, standardMsg)) File "/usr/lib64/python3.7/unittest/case.py", line 697, in fail raise self.failureException(msg) AssertionError: '<?xml version="1.0" encoding="UTF-16"?><foo>\uf8d2</foo>' != '' - <?xml version="1.0" encoding="UTF-16"?><foo></foo> + ---------------------------------------------------------------------- Ran 1854 tests in 13.498s FAILED (failures=1) make: *** [Makefile:74: test_inplace3] Error 1 because file /tmp/tmp.jezek8chac7f7 was created instead. The string is treated as URI-encoded, gets decoded by libxslt, and different filename than the one that was passed to write_output is used. This can also cause the file to be written to different directory, if filename /the/path/%2e/file is used -- the file will end up written as /the/path/file instead of being in the /the/path/%2e/ subdirectory. With diff --git a/src/lxml/tests/test_xslt.py b/src/lxml/tests/test_xslt.py index 96eb83ee..bb6762f8 100644 --- a/src/lxml/tests/test_xslt.py +++ b/src/lxml/tests/test_xslt.py @@ -170,12 +170,12 @@ class ETreeXSLTTestCase(HelperTestCase): else: self.assertTrue(False, "exception not raised") - def test_xslt_write_output_file(self): + def test_xslt_write_output_file_escape(self): with self._xslt_setup() as res: - f = NamedTemporaryFile(delete=False) + f = NamedTemporaryFile(prefix="krtek/%2e/jezek", delete=False) try: try: - res[0].write_output(f) + res[0].write_output(f.name) finally: f.close() with io.open(f.name, encoding='UTF-16') as f: and precreating the /tmp/krtek/%2e/ directory before running make test3, the test fails as well, and /tmp/krtek/jezekute41cmk is left behind because the os.unlink(f.name) at the end of the test removed /tmp/krtek/%2e/jezekute41cmk but the actual output file was different. One way to address the issue would be to properly escape the filename before passing it to xsltSaveResultToFilename. Another possibility would be to use xsltSaveResultToFile but that would mean that compression would need to be done by lxml. If any of those don't seem viable, at least the documentation of write_output should state that the serialization happens to "URI or file-like object", not to "file or file-like object", with some discussion about the escaping behaviour. -- Jan Pazdziora Senior Principal Software Engineer, Security Engineering, Red Hat

2 2

Fwd: validation error message consistency across Python versions
by Tom Kralidis 15 Mar '19

15 Mar '19

Hi: I'm finding different behaviour when trying validate XML documents against lxml 4.3.0 in Python 2.7 and 3.x environments. Example validation error using 4.3.0 and Python 2.7.15, libxml2 2.9.3+dfsg1-1ubuntu0.6 (xenial) Error: Element '{http://www.opengis.net/foo}Filter': This element is not expected. Expected is one of ( {http://www.opengis.net/ogc}Filter, {http://www.opengis.net/cat/csw/2.0.2}CqlText ). (line 0) Example validation error using 4.3.0 and Python 3.4.8, libxml2 2.9.3+dfsg1-1ubuntu0.6 (xenial) Error: Element '{http://www.opengis.net/foo}Filter': This element is not expected. Expected is one of ( {http://www.opengis.net/ogc}Filter, {http://www.opengis.net/cat/csw/2.0.2}CqlText ). (<string>, line 0) As a result this causes our (strict) functional testing to fail. Any idea what can be done to keep error messages consistent? Thanks ..Tom

3 3

Can't process from string
by Ovnicraft 15 Mar '19

15 Mar '19

Hi I am trying to process this file https://github-production-repository-file-5c1aeb.s3.amazonaws.com/142435147… But I have problems with tags, it doesn't in correct form, 3th party system generate <> with lt/gt; in tags. How can i resolve this to load with fromstring() ? -- Sent from Gmail Mobile

2 1

Crash in lxml 4.3.1
by Paul Adenot 01 Mar '19

01 Mar '19

Hi, I ran into a crash on lxml 4.3.1 (released not too long ago it seems), pulled in via pip, on travis. We had not pinned my dependency version, so we went from 4.3.0 to 4.3.1, and all our builds started failing. This can be reproduced deterministically (on travis and also on my local linux machine), by doing: git clone git@github.com:WebAudio/web-audio-api.git git clone https://github.com/tabatkins/bikeshed.git pip install --editable $PWD/bikeshed bikeshed update cd web-audio-api bikeshed --print=plain -f spec Let me know if I can help debugging this, I've posted some background info on https://github.com/WebAudio/web-audio-api/pull/1830#issuecomment-465229675 Thanks, Paul.

2 4

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

2005

2004

lxml - The Python XML Toolkit March 2019