Mailman 3 April 2020 - lxml - The Python XML Toolkit

[lxml-dev] lxml has its page on launchpad
by Stefan Behnel 11 Apr '23

11 Apr '23

Hi all, I added the lxml project to launchpad, the Ubuntu Bug-Tracker. It also has a FAQ engine and a couple of other goodies. https://launchpad.net/lxml It's easy to sign up for launchpad, BTW, no 90%-footnotes-contract. Have fun, Stefan

9 9

[lxml-dev] Checking whether a node is a comment/element
by Geoffrey Sneddon 10 Apr '23

10 Apr '23

Hi, What's the best way to check whether a given node is a comment or an element? For the former, I'm currently using isinstance(node, etree._Comment), which is rather obviously sub-optimal. -- Geoffrey Sneddon <http://gsnedders.com/>

6 6

[lxml-dev] Reparenting a node
by Lawrence Oluyede 30 Jan '23

30 Jan '23

I have a doc A and a doc B, I'd like to put a node extracted from A in the document B but I always get a ValueError: ValueError: Element is not a child of this node. I didn't find any "setparent" in the API. How can I do this? -- Lawrence, oluyede.org - neropercaso.it "It is difficult to get a man to understand something when his salary depends on not understanding it" - Upton Sinclair

3 2

[lxml-dev] lxml 2.0.5 released
by Stefan Behnel 11 Jan '23

11 Jan '23

Hi all, lxml 2.0.5 is on PyPI. This is a bug-fix-only release of the stable 2.0 series. Have fun, Stefan 2.0.5 (2008-05-01) Bugs fixed * Resolving to a filename in custom resolvers didn't work. * lxml did not honour libxslt's second error state "STOPPED", which let some XSLT errors pass silently. * Memory leak in Schematron with libxml2 >= 2.6.31.

3 4

[lxml-dev] Building LXML Trunk
by Sidnei da Silva 31 Aug '22

31 Aug '22

Hi, I've tried to build lxml from trunk today, on Win32. Got the following error: src\lxml\etree.c(880) : error C2059: syntax error : ')' src\lxml\etree.c(881) : error C2059: syntax error : ')' src\lxml\etree.c(882) : error C2059: syntax error : ')' src\lxml\etree.c(883) : error C2059: syntax error : ')' Any clue? Smells like a Pyrex issue? -- Sidnei da Silva Enfold Systems http://enfoldsystems.com Fax +1 832 201 8856 Office +1 713 942 2377 Ext 214

4 4

Accessing variables from within an XSL file
by Adrian Bool 05 Apr '21

05 Apr '21

Hi, I'm trying to extract from data from XSL files on an IBM DataPower system. I can't change the XSLT files - and therefore need to work around their current structure to get the info I need. I'm doing this from within Python 3.7 using etree.__version__ 4.3.3. Here are my imports, a dummy XML object and a cutdown version of the XSL file from the IBM system: from lxml import etree from textwrap import dedent xml = etree.XML("""<root></root>""") xsl = etree.XML(dedent( """<?xml version="1.0"?> <xsl:stylesheet version="1.0" xmlns:xsl="http://www.w3.org/1999/XSL/Transform" xmlns:dp="http://www.datapower.com/extensions" extension-element-prefixes="dp"> <xsl:template match="node()|@*"> <xsl:copy> <xsl:apply-templates select="node()|@*"/> </xsl:copy> <xsl:variable name="destination_url"> <xsl:value-of select="'https://my.domain.example.com/'" /> </xsl:variable>  <dp:set-variable name="'var://context/mpgw/Destination'" value="$destination_url" /> </xsl:template> </xsl:stylesheet> """)) My goal is to use lxml to process this file and extract the contents of the DataPower variable 'var://context/mpgw/Destination' - i.e. the contents of the xsl:variable $destination_url - in the above case ' https://my.domain.example.com/'. I've created an XSLT extension for the "dp:set-variable" element to try to capture this information: class MyExtElement(etree.XSLTExtension): def execute(self, context, self_node, input_node, output_parent): if self_node.tag == '{ http://www.datapower.com/extensions}set-variable': print(f"Setting variable of name {self_node.get('name')} to value {self_node.get('value')}") extensions = { ('http://www.datapower.com/extensions', 'set-variable') : MyExtElement() } result = etree.XSLT(xsl, extensions=extensions)(xml) print(f"Result: {result}") When this is being called; instead of the value of destination_url I'm getting the static "$destination_url" string instead - with the output of the above script being: Setting variable of name 'var://context/mpgw/Destination' to value $destination_url Result: <?xml version="1.0"?> <root/> I am rather suprised that the $destination_url xsl:variable was not being evaluated - does anyone know why this is the case? Given the above, I've been trying to find a way to access the $destination_url xsl:variable from within my XSLTExtension - but unfortunately with no luck. Some things I've tried: * Calling self.process_children(context) within the extension - this just results in an empty list. * I've used dir() to look through the context, self_node and input_node args to try to find somewhere the variables may be accessible but with no luck. * I've used deepcopy to clone self_node to get an xpath function on this argument and made some attempts at using xpath to access the xsl variable - all unsuccessful: node = deepcopy(self_node) node.xpath("""<xsl:value-of select="$destination_url">""") *** lxml.etree.XPathEvalError: Invalid expression node.xpath("""$destination_url""") *** lxml.etree.XPathEvalError: Undefined variable node.xpath("""xsl:$destination_url""") *** lxml.etree.XPathEvalError: Invalid expression node.xpath("""xsl:$destination_url""", namespaces={'xsl':'http://www.w3.org/1999/XSL/Transform'}) *** lxml.etree.XPathEvalError: Invalid expressio node.xpath("""$destination_url""", namespaces={'xsl':'http://www.w3.org/1999/XSL/Transform'}) *** lxml.etree.XPathEvalError: Undefined variable node.xpath("""/xsl:$destination_url""", namespaces={'xsl':'http://www.w3.org/1999/XSL/Transform'}) *** lxml.etree.XPathEvalError: Invalid expression node.xpath("""<xsl:value-of select="$destination_url">""", namespaces={'xsl':'http://www.w3.org/1999/XSL/Transform'}) *** lxml.etree.XPathEvalError: Invalid expression I'm lost now! Any suggestions greatly appreciated! Cheers, aid

4 5

Re: [lxml] WG: lxml can't load external DTD from HTTPS internetaddress
by Holger.Joukl＠LBBW.de 26 Apr '20

26 Apr '20

Hi, I have a problem with a DTD that is stored on a server and can be retrieved from the following path. https://smile.htw-berlin.de/TMP_dtd/XML_DTDValidation.dtd <-- Internetside 1 Unfortunately the website runs under the HTTPS protocol. Since this is provided by the university, I can't change it. I have already been informed that I can't get an area that only runs with the HTTP protocol. Error: warning: failed to load external entity "https://smile.htw-berlin.de/TMP_dtd/XML_DTDValidation.dtd At the following Internet address: http://mum.mxndmrtz.de/wp-content/uploads/2020/02/XML_DTDValidation.txt <-- Internetside 2 the same DTD but with a different file suffix, otherwise I could not upload this file on the server. Works fine. I'm unsure if libxml2 actually supports https, and thus lxml(?). For direct primary resource access that shouldn't pose any problem at all, as you can just use urllib2/3 or requests to retrieve it and hand a file-like object to etree.parse() instead of the URL, s.th. like etree.parse(io.BytesIO(requests.get("https://www.w3schools.com/xml/note.xml").content))) # or etree.fromstring without BytesIO However it might get more complicated for included/imported resources and DTDs, as these dependency URLs will probably just be fed to libxml2 http client, again. If that's the case and libxml2 does indeed not support https you probably need to - use an XML Catalog to "localize" those DTDs, .i.e. download them and make the catalog pick up the local versions or - use some kind of custom resolver and mirror/proxy the https resources with http "shims" (never done that myself) Probably that's the place to start looking: https://lxml.de/resolvers.html When I try to check an example XML with LXML using the DTD, it only works with the DTD that is on the web page 2. As an example the two references to the respective DTD are shown. <!DOCTYPE book PUBLIC "-//JR//HTW Berlin book//EN" "https://smile.htw-berlin.de/TMP_dtd/XML_DTDValidation.dtd"> <!DOCTYPE book PUBLIC "-//JR//HTW Berlin book//EN" "http://mum.mxndmrtz.de/wp-content/uploads/2020/02/XML_DTDValidation.txt"> One other thing that comes to mind when s.th. works over http but not https is certificate validation problems - but I suspect that's not the problem here. Holger [cid:image001_e3261df1-efb3-4c64-a4de-9b63c5e52cfb.png]<https://www.lbbw.de/> [cid:image002_15f56f09-b8fe-4ddc-a247-e3cf3d25c71f.png]<https://twitter.com/lbbw> [cid:image003_e01dca75-944e-4ea3-8463-c9c061e5b36b.png] <https://www.linkedin.com/company/lbbw> [cid:image004_5642e2b4-e190-4f0c-a274-7ceb59fe913c.png] <https://www.xing.com/company/lbbw> [cid:image005_53d53024-7202-4d60-8d95-dc0f361239ca.png] <https://www.facebook.com/LBBW.Stuttgart/> [cid:image006_9c0b6462-8935-4b9c-9fe3-0caa90d8623b.png] <https://www.youtube.com/user/LBBWDirekt> [cid:image007_9a7ebd18-8e4e-4c3e-931a-ecc4bf575ace.png] <https://www.instagram.com/lbbw_karriere/> Landesbank Baden-Wuerttemberg Anstalt des oeffentlichen Rechts Hauptsitze: Stuttgart, Karlsruhe, Mannheim, Mainz HRA 12704 Amtsgericht Stuttgart HRA 4356, HRA 104 440 Amtsgericht Mannheim HRA 40687 Amtsgericht Mainz Die LBBW verarbeitet gemaess Erfordernissen der DSGVO Ihre personenbezogenen Daten. Informationen finden Sie unter https://www.lbbw.de/datenschutz.

2 1

CDATA and .text
by Jens Tröger 25 Apr '20

25 Apr '20

Hello, As a follow-up to the code example here: https://lxml.de/api.html#cdata >>> parser = etree.XMLParser(strip_cdata=False) >>> root = etree.XML('<root><![CDATA[test]]></root>', parser) >>> root.text 'test' So here, the .text is a string and if I check the type of it >>> type(root.text) <class 'str'> then it’s an actual str instance. Yet it serializes with the knowledge that .text originated from a CDATA: >>> etree.tostring(root) b'<root><![CDATA[test]]></root>' Is there a way to find out whether .text came from a CDATA, that root’s text() node should be wrapped? It seems to be stored away somewhere, but I can’t find that bit… Thanks, Jens -- Jens Tröger http://savage.light-speed.de/

2 1

WG: lxml can't load external DTD from HTTPS internetaddress
by Jörg Rademann 24 Apr '20

24 Apr '20

Dear Sirs, I have a problem with a DTD that is stored on a server and can be retrieved from the following path. <https://smile.htw-berlin.de/TMP_dtd/XML_DTDValidation.dtd> https://smile.htw-berlin.de/TMP_dtd/XML_DTDValidation.dtd <-- Internetside 1 Unfortunately the website runs under the HTTPS protocol. Since this is provided by the university, I can't change it. I have already been informed that I can't get an area that only runs with the HTTP protocol. Error: warning: failed to load external entity "https://smile.htw-berlin.de/TMP_dtd/XML_DTDValidation.dtd At the following Internet address: <http://mum.mxndmrtz.de/wp-content/uploads/2020/02/XML_DTDValidation.txt> http://mum.mxndmrtz.de/wp-content/uploads/2020/02/XML_DTDValidation.txt <-- Internetside 2 the same DTD but with a different file suffix, otherwise I could not upload this file on the server. Works fine. When I try to check an example XML with LXML using the DTD, it only works with the DTD that is on the web page 2. As an example the two references to the respective DTD are shown. <!DOCTYPE book PUBLIC "-//JR//HTW Berlin book//EN" " <https://smile.htw-berlin.de/TMP_dtd/XML_DTDValidation.dtd> https://smile.htw-berlin.de/TMP_dtd/XML_DTDValidation.dtd"> <!DOCTYPE book PUBLIC "-//JR//HTW Berlin book//EN" " <http://mum.mxndmrtz.de/wp-content/uploads/2020/02/XML_DTDValidation.txt> http://mum.mxndmrtz.de/wp-content/uploads/2020/02/XML_DTDValidation.txt"> I send you an example XML, which points to DTD on Internet page 1 and cannot be retrieved by LXML. (I have changed ".xml" to ".txt" in the file) In a forum I also read that a DTD on a HTTPS page cannot be used. Is this true? Now to my question. Can I somehow access the DTD on web page 1 with the scope of LXML? Mit freundlichen Grüßen Jörg Rademann --------------------------------------------------------------------- Dipl.-Ing. Jörg Rademann Wissenschaftlicher Mitarbeiter Fachbereich 2 Fahrzeugtechnik Hochschule für Technik und Wirtschaft (HTW) Berlin Raum C596 Wilhelminenhofstraße 75a 12459 Berlin Tel. +49 30 5019-3854 | Fax +49 30 5019-48-3854 <mailto:Joerg.Rademann@HTW-Berlin.de> Joerg.Rademann(a)HTW-Berlin.de | <http://www.htw-berlin.de/> www.htw-berlin.de

1 0

test - please ignore
by Stefan Behnel 21 Apr '20

21 Apr '20

Hi, I received word that the mailing list was broken, so this is just a test to confirm that it works again. Happy XMLing, Stefan

2 1