lxml - The Python XML Toolkit July 2020

lxml@python.org

4 participants
8 discussions

[lxml-dev] Checking whether a node is a comment/element
by Geoffrey Sneddon 11 Jan '23

11 Jan '23

Hi, What's the best way to check whether a given node is a comment or an element? For the former, I'm currently using isinstance(node, etree._Comment), which is rather obviously sub-optimal. -- Geoffrey Sneddon <http://gsnedders.com/>

4 4

[lxml-dev] lxml 2.0.5 released
by Stefan Behnel 11 Jan '23

11 Jan '23

Hi all, lxml 2.0.5 is on PyPI. This is a bug-fix-only release of the stable 2.0 series. Have fun, Stefan 2.0.5 (2008-05-01) Bugs fixed * Resolving to a filename in custom resolvers didn't work. * lxml did not honour libxslt's second error state "STOPPED", which let some XSLT errors pass silently. * Memory leak in Schematron with libxml2 >= 2.6.31.

3 4

[lxml-dev] lxml has its page on launchpad
by Stefan Behnel 21 Sep '22

21 Sep '22

Hi all, I added the lxml project to launchpad, the Ubuntu Bug-Tracker. It also has a FAQ engine and a couple of other goodies. https://launchpad.net/lxml It's easy to sign up for launchpad, BTW, no 90%-footnotes-contract. Have fun, Stefan

3 2

[lxml-dev] Building LXML Trunk
by Sidnei da Silva 31 Aug '22

31 Aug '22

Hi, I've tried to build lxml from trunk today, on Win32. Got the following error: src\lxml\etree.c(880) : error C2059: syntax error : ')' src\lxml\etree.c(881) : error C2059: syntax error : ')' src\lxml\etree.c(882) : error C2059: syntax error : ')' src\lxml\etree.c(883) : error C2059: syntax error : ')' Any clue? Smells like a Pyrex issue? -- Sidnei da Silva Enfold Systems http://enfoldsystems.com Fax +1 832 201 8856 Office +1 713 942 2377 Ext 214

4 4

Accessing variables from within an XSL file
by Adrian Bool 05 Apr '21

05 Apr '21

Hi, I'm trying to extract from data from XSL files on an IBM DataPower system. I can't change the XSLT files - and therefore need to work around their current structure to get the info I need. I'm doing this from within Python 3.7 using etree.__version__ 4.3.3. Here are my imports, a dummy XML object and a cutdown version of the XSL file from the IBM system: from lxml import etree from textwrap import dedent xml = etree.XML("""<root></root>""") xsl = etree.XML(dedent( """<?xml version="1.0"?> <xsl:stylesheet version="1.0" xmlns:xsl="http://www.w3.org/1999/XSL/Transform" xmlns:dp="http://www.datapower.com/extensions" extension-element-prefixes="dp"> <xsl:template match="node()|@*"> <xsl:copy> <xsl:apply-templates select="node()|@*"/> </xsl:copy> <xsl:variable name="destination_url"> <xsl:value-of select="'https://my.domain.example.com/'" /> </xsl:variable>  <dp:set-variable name="'var://context/mpgw/Destination'" value="$destination_url" /> </xsl:template> </xsl:stylesheet> """)) My goal is to use lxml to process this file and extract the contents of the DataPower variable 'var://context/mpgw/Destination' - i.e. the contents of the xsl:variable $destination_url - in the above case ' https://my.domain.example.com/'. I've created an XSLT extension for the "dp:set-variable" element to try to capture this information: class MyExtElement(etree.XSLTExtension): def execute(self, context, self_node, input_node, output_parent): if self_node.tag == '{ http://www.datapower.com/extensions}set-variable': print(f"Setting variable of name {self_node.get('name')} to value {self_node.get('value')}") extensions = { ('http://www.datapower.com/extensions', 'set-variable') : MyExtElement() } result = etree.XSLT(xsl, extensions=extensions)(xml) print(f"Result: {result}") When this is being called; instead of the value of destination_url I'm getting the static "$destination_url" string instead - with the output of the above script being: Setting variable of name 'var://context/mpgw/Destination' to value $destination_url Result: <?xml version="1.0"?> <root/> I am rather suprised that the $destination_url xsl:variable was not being evaluated - does anyone know why this is the case? Given the above, I've been trying to find a way to access the $destination_url xsl:variable from within my XSLTExtension - but unfortunately with no luck. Some things I've tried: * Calling self.process_children(context) within the extension - this just results in an empty list. * I've used dir() to look through the context, self_node and input_node args to try to find somewhere the variables may be accessible but with no luck. * I've used deepcopy to clone self_node to get an xpath function on this argument and made some attempts at using xpath to access the xsl variable - all unsuccessful: node = deepcopy(self_node) node.xpath("""<xsl:value-of select="$destination_url">""") *** lxml.etree.XPathEvalError: Invalid expression node.xpath("""$destination_url""") *** lxml.etree.XPathEvalError: Undefined variable node.xpath("""xsl:$destination_url""") *** lxml.etree.XPathEvalError: Invalid expression node.xpath("""xsl:$destination_url""", namespaces={'xsl':'http://www.w3.org/1999/XSL/Transform'}) *** lxml.etree.XPathEvalError: Invalid expressio node.xpath("""$destination_url""", namespaces={'xsl':'http://www.w3.org/1999/XSL/Transform'}) *** lxml.etree.XPathEvalError: Undefined variable node.xpath("""/xsl:$destination_url""", namespaces={'xsl':'http://www.w3.org/1999/XSL/Transform'}) *** lxml.etree.XPathEvalError: Invalid expression node.xpath("""<xsl:value-of select="$destination_url">""", namespaces={'xsl':'http://www.w3.org/1999/XSL/Transform'}) *** lxml.etree.XPathEvalError: Invalid expression I'm lost now! Any suggestions greatly appreciated! Cheers, aid

4 5

lxml.etree.iterparse schema validation does not validate
by Kai Hillmann 28 Jul '20

28 Jul '20

Hi, I'm not sure whether I use the iterparse interface correctly but I did a look inside the tests for it (src/lxml/tests/test_xmlschema.py) and found out that the validation results of iterparse provided with a schema and schema.validate or schema.assertValid using a parsed tree (via etree.parse) differ. The probably necessary information to reproduce: (run on an xubuntu 18.04 with or without manually updated lxml via pip install -U lxml) Python 2.7.17 (default, Jul 20 2020, 15:37:01) [GCC 7.5.0] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>> import sys >>> from lxml import etree >>> >>> print("%-20s: %s" % ('Python', sys.version_info)) Python : sys.version_info(major=2, minor=7, micro=17, releaselevel='final', serial=0) >>> print("%-20s: %s" % ('lxml.etree', etree.LXML_VERSION)) lxml.etree : (4, 5, 2, 0) >>> print("%-20s: %s" % ('libxml used', etree.LIBXML_VERSION)) libxml used : (2, 9, 10) >>> print("%-20s: %s" % ('libxml compiled', etree.LIBXML_COMPILED_VERSION)) libxml compiled : (2, 9, 10) >>> print("%-20s: %s" % ('libxslt used', etree.LIBXSLT_VERSION)) libxslt used : (1, 1, 34) >>> print("%-20s: %s" % ('libxslt compiled', etree.LIBXSLT_COMPILED_VERSION)) libxslt compiled : (1, 1, 34) >>> The minimal script one could use to reproduce (based on your test test_xmlschema_iterparse_fail in src/lxml/tests/test_xmlschema.py): from lxml import etree from io import BytesIO import StringIO schema = etree.parse(StringIO.StringIO(''' <xsd:schema xmlns:xsd="http://www.w3.org/2001/XMLSchema"> <xsd:element name="a" type="AType"/> <xsd:complexType name="AType"> <xsd:sequence> <xsd:element name="b" type="xsd:string" /> </xsd:sequence> </xsd:complexType> </xsd:schema> ''')) schema = etree.XMLSchema(schema) raw_tree = BytesIO('<a><c></c></a>') etree.iterparse(raw_tree, schema=schema) tree = etree.parse(raw_tree) if not schema.validate(tree): print('Error: Different validation results:') schema.assertValid(tree) After some more research I found out that also the test_xmlschema_iterparse_fail seems to be broken. Using the one below which I corrected using with self.assertRaises instead to ensure not raised but expected exceptions also lead to a failing test you could reproduce the bug (if you agree that it is one) also within your test framework as there is no exception raised by iterparse invocation: def test_xmlschema_iterparse_fail(self): schema = self.parse(''' <xsd:schema xmlns:xsd="http://www.w3.org/2001/XMLSchema"> <xsd:element name="a" type="AType"/> <xsd:complexType name="AType"> <xsd:sequence> <xsd:element name="b" type="xsd:string" /> </xsd:sequence> </xsd:complexType> </xsd:schema> ''') schema = etree.XMLSchema(schema) with self.assertRaises(etree.XMLSyntaxError): etree.iterparse(BytesIO('<a><c></c></a>'), schema=schema) make test invocation result is then something like: FAIL: test_xmlschema_iterparse_fail (lxml.tests.test_xmlschema.ETreeXMLSchemaTestCase) ---------------------------------------------------------------------- Traceback (most recent call last): File "/usr/lib/python2.7/unittest/case.py", line 329, in run testMethod() File "/home/user/git/lxml/src/lxml/tests/test_xmlschema.py", line 289, in test_xmlschema_iterparse_fail etree.iterparse(BytesIO('<a><c></c></a>'), schema=schema) File "/usr/lib/python2.7/unittest/case.py", line 116, in __exit__ "{0} not raised".format(exc_name)) AssertionError: XMLSyntaxError not raised Could you advise me what to do next? Should I issue a bug for that? Or is my expectation wrong here that iterparse should raise an exception in case of a schema violation? Best regards, Kai

2 2

Interpretation of AssertionError
by Per 28 Jul '20

28 Jul '20

Hi! lxml seems to raise AssertionErrors quite often. How should they be interpreted? My understanding is that AssertionErrors in general should be interpreted as "this thing that shouldn't happen just happened", i.e. basically a bug rather than an error that's part of the interface. Since I get them as often as I do I'm wondering if lxml uses a different interpretation, that AssertionErrors are part of lxml's interface. For example here's a code snippet that raises an AssertionError (lxml 4.5.2, macOS 10.15.6): parser = XMLParser(target=TreeBuilder()) parser.feed('</a>') And this is the exception stack: Traceback (most recent call last): File "src/lxml/parsertarget.pxi", line 158, in lxml.etree._TargetParserContext._handleParseResult File "src/lxml/parser.pxi", line 654, in lxml.etree._raiseParseError File "<string>", line 1 lxml.etree.XMLSyntaxError: StartTag: invalid element name, line 1, column 2 During handling of the above exception, another exception occurred: Traceback (most recent call last): File "<stdin>", line 1, in <module> File "src/lxml/parser.pxi", line 1256, in lxml.etree._FeedParser.feed File "src/lxml/parser.pxi", line 1384, in lxml.etree._FeedParser.feed File "src/lxml/parsertarget.pxi", line 167, in lxml.etree._TargetParserContext._handleParseResult File "src/lxml/saxparser.pxi", line 811, in lxml.etree.TreeBuilder.close AssertionError: missing toplevel element Should I interpret this AssertionError as a bug in lxml? Thank you! Per

2 1

short line number in tree xmlNode raises exceptions
by Herm Fischer 01 Jul '20

01 Jul '20

XBRL applications built on lxml are raising an exception because line is a short in xmlNode in tree.pxd, whereas xmlGetLineNo is a long. (I remember we changed sourceline to long a few years ago.) Our usage now needs to set the line number, in some cases, as well as get it. Is there a reason why xmlNode line is still short instead of long? Can that be changed (can we locally recompile with long)? Herm Fischer

1 0

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

2005

2004

lxml - The Python XML Toolkit July 2020