Mailman 3 August 2019 - lxml - The Python XML Toolkit

[lxml-dev] lxml has its page on launchpad
by Stefan Behnel 10 Apr '23

10 Apr '23

Hi all, I added the lxml project to launchpad, the Ubuntu Bug-Tracker. It also has a FAQ engine and a couple of other goodies. https://launchpad.net/lxml It's easy to sign up for launchpad, BTW, no 90%-footnotes-contract. Have fun, Stefan

9 9

[lxml-dev] Checking whether a node is a comment/element
by Geoffrey Sneddon 09 Apr '23

09 Apr '23

Hi, What's the best way to check whether a given node is a comment or an element? For the former, I'm currently using isinstance(node, etree._Comment), which is rather obviously sub-optimal. -- Geoffrey Sneddon <http://gsnedders.com/>

6 6

[lxml-dev] Reparenting a node
by Lawrence Oluyede 30 Jan '23

30 Jan '23

I have a doc A and a doc B, I'd like to put a node extracted from A in the document B but I always get a ValueError: ValueError: Element is not a child of this node. I didn't find any "setparent" in the API. How can I do this? -- Lawrence, oluyede.org - neropercaso.it "It is difficult to get a man to understand something when his salary depends on not understanding it" - Upton Sinclair

3 2

[lxml-dev] lxml 2.0.5 released
by Stefan Behnel 11 Jan '23

11 Jan '23

Hi all, lxml 2.0.5 is on PyPI. This is a bug-fix-only release of the stable 2.0 series. Have fun, Stefan 2.0.5 (2008-05-01) Bugs fixed * Resolving to a filename in custom resolvers didn't work. * lxml did not honour libxslt's second error state "STOPPED", which let some XSLT errors pass silently. * Memory leak in Schematron with libxml2 >= 2.6.31.

3 4

[lxml-dev] Building LXML Trunk
by Sidnei da Silva 30 Aug '22

30 Aug '22

Hi, I've tried to build lxml from trunk today, on Win32. Got the following error: src\lxml\etree.c(880) : error C2059: syntax error : ')' src\lxml\etree.c(881) : error C2059: syntax error : ')' src\lxml\etree.c(882) : error C2059: syntax error : ')' src\lxml\etree.c(883) : error C2059: syntax error : ')' Any clue? Smells like a Pyrex issue? -- Sidnei da Silva Enfold Systems http://enfoldsystems.com Fax +1 832 201 8856 Office +1 713 942 2377 Ext 214

4 4

Accessing variables from within an XSL file
by Adrian Bool 05 Apr '21

05 Apr '21

Hi, I'm trying to extract from data from XSL files on an IBM DataPower system. I can't change the XSLT files - and therefore need to work around their current structure to get the info I need. I'm doing this from within Python 3.7 using etree.__version__ 4.3.3. Here are my imports, a dummy XML object and a cutdown version of the XSL file from the IBM system: from lxml import etree from textwrap import dedent xml = etree.XML("""<root></root>""") xsl = etree.XML(dedent( """<?xml version="1.0"?> <xsl:stylesheet version="1.0" xmlns:xsl="http://www.w3.org/1999/XSL/Transform" xmlns:dp="http://www.datapower.com/extensions" extension-element-prefixes="dp"> <xsl:template match="node()|@*"> <xsl:copy> <xsl:apply-templates select="node()|@*"/> </xsl:copy> <xsl:variable name="destination_url"> <xsl:value-of select="'https://my.domain.example.com/'" /> </xsl:variable>  <dp:set-variable name="'var://context/mpgw/Destination'" value="$destination_url" /> </xsl:template> </xsl:stylesheet> """)) My goal is to use lxml to process this file and extract the contents of the DataPower variable 'var://context/mpgw/Destination' - i.e. the contents of the xsl:variable $destination_url - in the above case ' https://my.domain.example.com/'. I've created an XSLT extension for the "dp:set-variable" element to try to capture this information: class MyExtElement(etree.XSLTExtension): def execute(self, context, self_node, input_node, output_parent): if self_node.tag == '{ http://www.datapower.com/extensions}set-variable': print(f"Setting variable of name {self_node.get('name')} to value {self_node.get('value')}") extensions = { ('http://www.datapower.com/extensions', 'set-variable') : MyExtElement() } result = etree.XSLT(xsl, extensions=extensions)(xml) print(f"Result: {result}") When this is being called; instead of the value of destination_url I'm getting the static "$destination_url" string instead - with the output of the above script being: Setting variable of name 'var://context/mpgw/Destination' to value $destination_url Result: <?xml version="1.0"?> <root/> I am rather suprised that the $destination_url xsl:variable was not being evaluated - does anyone know why this is the case? Given the above, I've been trying to find a way to access the $destination_url xsl:variable from within my XSLTExtension - but unfortunately with no luck. Some things I've tried: * Calling self.process_children(context) within the extension - this just results in an empty list. * I've used dir() to look through the context, self_node and input_node args to try to find somewhere the variables may be accessible but with no luck. * I've used deepcopy to clone self_node to get an xpath function on this argument and made some attempts at using xpath to access the xsl variable - all unsuccessful: node = deepcopy(self_node) node.xpath("""<xsl:value-of select="$destination_url">""") *** lxml.etree.XPathEvalError: Invalid expression node.xpath("""$destination_url""") *** lxml.etree.XPathEvalError: Undefined variable node.xpath("""xsl:$destination_url""") *** lxml.etree.XPathEvalError: Invalid expression node.xpath("""xsl:$destination_url""", namespaces={'xsl':'http://www.w3.org/1999/XSL/Transform'}) *** lxml.etree.XPathEvalError: Invalid expressio node.xpath("""$destination_url""", namespaces={'xsl':'http://www.w3.org/1999/XSL/Transform'}) *** lxml.etree.XPathEvalError: Undefined variable node.xpath("""/xsl:$destination_url""", namespaces={'xsl':'http://www.w3.org/1999/XSL/Transform'}) *** lxml.etree.XPathEvalError: Invalid expression node.xpath("""<xsl:value-of select="$destination_url">""", namespaces={'xsl':'http://www.w3.org/1999/XSL/Transform'}) *** lxml.etree.XPathEvalError: Invalid expression I'm lost now! Any suggestions greatly appreciated! Cheers, aid

4 5

Another question about formatting
by Martin Mueller 01 Sep '19

01 Sep '19

Is there a way of saying in lxml: “if the element is <w> or <pc>, indent it by two spaces from the left margin, else indent the element by six spaces” ? In the long texts I deal with, the typical hierarchical indentation doesn’t help much with readability. On the other hand, some <w> elements with their attributes can take up a whole line or more, so there is an advantage of keeping them as close to the beginning of a line. In my project, the readability of the printed XML is a non-trivial advantage: there is a lot of data checking and manual editing. Martin Mueller

2 1

Fwd: character encoding on windows
by Romain Goffe 27 Aug '19

27 Aug '19

Hi, Not sure this is a bug but I have some difficulties in parsing an xml file with the utf8 character superscript minus <https://www.fileformat.info/info/unicode/char/207b/index.htm> on windows (windows 10 64bits). I can reproduce this with the following script: ``` python import sys from lxml import etree print("%-20s: %s" % ('Python', sys.version_info)) print("%-20s: %s" % ('lxml.etree', etree.LXML_VERSION)) print("%-20s: %s" % ('libxml used', etree.LIBXML_VERSION)) print("%-20s: %s" % ('libxml compiled', etree.LIBXML_COMPILED_VERSION)) print("%-20s: %s" % ('libxslt used', etree.LIBXSLT_VERSION)) print("%-20s: %s" % ('libxslt compiled', etree.LIBXSLT_COMPILED_VERSION)) tree = etree.parse(sys.stdin) ``` Using the following xml file: ``` xml <?xml version="1.0" encoding="UTF-8"?> <root>⁻</root> ``` And I'm getting the following output: ``` python-traceback Python : sys.version_info(major=3, minor=6, micro=1, releaselevel='final', serial=0) lxml.etree : (4, 4, 1, 0) libxml used : (2, 9, 5) libxml compiled : (2, 9, 5) libxslt used : (1, 1, 30) libxslt compiled : (1, 1, 30) Traceback (most recent call last): File "test.py", line 14, in <module> tree = etree.parse(sys.stdin) File "src\lxml\etree.pyx", line 3467, in lxml.etree.parse File "src\lxml\parser.pxi", line 1860, in lxml.etree._parseDocument File "src\lxml\parser.pxi", line 1880, in lxml.etree._parseFilelikeDocument File "src\lxml\parser.pxi", line 1775, in lxml.etree._parseDocFromFilelike File "src\lxml\parser.pxi", line 1187, in lxml.etree._BaseParser._parseDocFromFilelike File "src\lxml\parser.pxi", line 601, in lxml.etree._ParserContext._handleParseResultDoc File "src\lxml\parser.pxi", line 707, in lxml.etree._handleParseResult File "src\lxml\etree.pyx", line 318, in lxml.etree._ExceptionContext._raise_if_stored File "src\lxml\parser.pxi", line 374, in lxml.etree._FileReaderContext.copyToBuffer UnicodeEncodeError: 'utf-8' codec can't encode character '\udc81' in position 46: surrogates not allowed ``` When called by the following command: ``` python script.py < input.xml ``` Note that this works flawlessly on an ubuntu system with both Python 2.7 and 3.7. Best regards, Romain

2 2

problem with getprevious in lxml
by Peng Yu 24 Aug '19

24 Aug '19

``` from lxml import html import sys doc = html.parse(sys.stdin, parser = html.HTMLParser(encoding='utf-8')) r = doc.xpath(sys.argv[1])[0] p = r.getprevious() if p is None: print p else: print html.tostring(r.getprevious()) ``` I have the above python code. The following runs show these. I'd expect the first run returns `None`, but it shows up as `<p>x</p>`. I don't think this makes sense. Is it a bug? ``` $ ./main.py //div <<< '<html>x<body><div>abc</div></html>' <p>x</p> $ ./main.py //div <<< '<html><body><h1>x<div>abc</div><h1></html>' None ``` -- Regards, Peng

2 2

replace nbsp with space
by Peng Yu 24 Aug '19

24 Aug '19

I use the following code to replace nbsp with space. Is it the best way to do so in lxml? Thanks. from lxml import html doc = html.parse(sys.stdin, parser = html.HTMLParser(encoding='utf-8')) for x in doc.iter(): if x.text is not None: x.text = x.text.replace(u'\xa0', ' ') if x.tail is not None: x.tail = x.tail.replace(u'\xa0', ' ') sys.stdout.write(html.tostring(doc).encode('utf-8')) -- Regards, Peng

2 3