Mailman 3 September 2017 - lxml - The Python XML Toolkit

[lxml-dev] lxml has its page on launchpad
by Stefan Behnel 11 Apr '23

11 Apr '23

Hi all, I added the lxml project to launchpad, the Ubuntu Bug-Tracker. It also has a FAQ engine and a couple of other goodies. https://launchpad.net/lxml It's easy to sign up for launchpad, BTW, no 90%-footnotes-contract. Have fun, Stefan

9 9

[lxml-dev] Checking whether a node is a comment/element
by Geoffrey Sneddon 10 Apr '23

10 Apr '23

Hi, What's the best way to check whether a given node is a comment or an element? For the former, I'm currently using isinstance(node, etree._Comment), which is rather obviously sub-optimal. -- Geoffrey Sneddon <http://gsnedders.com/>

6 6

[lxml-dev] Reparenting a node
by Lawrence Oluyede 30 Jan '23

30 Jan '23

I have a doc A and a doc B, I'd like to put a node extracted from A in the document B but I always get a ValueError: ValueError: Element is not a child of this node. I didn't find any "setparent" in the API. How can I do this? -- Lawrence, oluyede.org - neropercaso.it "It is difficult to get a man to understand something when his salary depends on not understanding it" - Upton Sinclair

3 2

[lxml-dev] lxml 2.0.5 released
by Stefan Behnel 11 Jan '23

11 Jan '23

Hi all, lxml 2.0.5 is on PyPI. This is a bug-fix-only release of the stable 2.0 series. Have fun, Stefan 2.0.5 (2008-05-01) Bugs fixed * Resolving to a filename in custom resolvers didn't work. * lxml did not honour libxslt's second error state "STOPPED", which let some XSLT errors pass silently. * Memory leak in Schematron with libxml2 >= 2.6.31.

3 4

[lxml-dev] Building LXML Trunk
by Sidnei da Silva 31 Aug '22

31 Aug '22

Hi, I've tried to build lxml from trunk today, on Win32. Got the following error: src\lxml\etree.c(880) : error C2059: syntax error : ')' src\lxml\etree.c(881) : error C2059: syntax error : ')' src\lxml\etree.c(882) : error C2059: syntax error : ')' src\lxml\etree.c(883) : error C2059: syntax error : ')' Any clue? Smells like a Pyrex issue? -- Sidnei da Silva Enfold Systems http://enfoldsystems.com Fax +1 832 201 8856 Office +1 713 942 2377 Ext 214

4 4

thread-specific data for xslt callbacks
by Bob Kline 13 Oct '17

13 Oct '17

I need to make some thread-specific data available to my xslt function callbacks. I think I've sort of reverse-engineered a way to do it, by creating my own parser class, stuffing the data I need into an instance of that class, using the parser instance to parse the XSL/T filter, and then in the extension function following the context argument's context_node to its root tree, which in turn gets me back to my own parser object (and my thread-specific data). Is there a more elegant way to do it that I just haven't yet stumbled across in reading through the API docs? Thanks! -- Bob Kline http://www.rksystems.com mailto:bkline@rksystems.com

2 8

Strange results of xpath element search using string-value
by Алексей Алексей 30 Sep '17

30 Sep '17

2 1

xs:include not finding included file after XSLT transformation
by Markus Schöpflin 29 Sep '17

29 Sep '17

Using python-lxml-2.2.3-1.1.el6.x86_64 on RHEL6. I have a schema file using relative include statements like this: <xs:include schemaLocation="some/relative/path/blah.xsd"/> From this file an element tree object is created: schematree = etree.parse(schemafile) Then a schema object is created using: etree.XMLSchema(schematree) This works fine, the include statement is resolved relative to the locate of the schema file. Now I need to modify the schema tree object via an XSLT transformation, before the schema object is created: transform = etree.XSLT(...) schematree = transform(schematree) Now the creation of the schema object fails with the error message: lxml.etree.XMLSchemaParseError: Element '{http://www.w3.org/2001/XMLSchema}include': Failed to load the document 'some/relative/path/blah.xsd' for inclusion., line 3 I suppose this is caused by the fact that applying the XML transformation to the original element tree object creates a new element tree object (or rather _XSLTResultTree which is derived from _ElementTree), and the new object does not have a valid docinfo property. (It is empty.) So the include file is resolved relative to the current working directory, and not relative to the location of the original schema XML document. Now, is there a way to get the included file resolved relative to the location of the original schema. I tried setting the docinfo property on the result of the transformation, but this is not allowed. Is there any other way? TIA, Markus

2 5

C14N namespace prefixes
by Nagy László Zsolt 27 Sep '17

27 Sep '17

Hello, I need to calculate the SHA1 digest of an XML node. In order to do this correctly, I need to do C14N on the node. Specifically, http://www.w3.org/2001/10/xml-exc-c14n Here is a minimal working example: import io import copy from lxml import etree ORIGINAL_XML = b'''<soap:Envelope xmlns:ns="http://docs.oasis-open.org/ws-sx/ws-trust/200512" xmlns:soap="http://www.w3.org/2003/05/soap-envelope"> <soap:Header> <wsse:Security xmlns:wsse="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-secext-1…" xmlns:wsu="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-utility-…"> <wsu:Timestamp wsu:Id="TS-1a8236fc-8e3a-9b71-495f-20f52709e893"> <wsu:Created>2017-09-19T07:38:45Z</wsu:Created><wsu:Expires>2017-09-19T08:38:45Z</wsu:Expires> </wsu:Timestamp> </wsse:Security> </soap:Header> <soap:Body xmlns:wsu="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-utility-…" wsu:Id="id-1ea12929-0b1a-54d4-08db-b049cee527b5"> <ns:RequestSecurityToken> <ns:RequestType>http://docs.oasis-open.org/ws-sx/ws-trust/200512/Issue</ns:RequestType> <ns:TokenType>http://docs.oasis-open.org/wss/oasis-wss-saml-token-profile-1.1#SAMLV2.0</ns:TokenType> </ns:RequestSecurityToken> </soap:Body> </soap:Envelope> '''.replace(b'\n', b'') PREFIXES = ["soap", "wsse", "wsu"] # This is known to be the "good" C14N version of the wsu:Timestamp node GOOD_C14N_XML = b'''<wsu:Timestamp xmlns:soap="http://www.w3.org/2003/05/soap-envelope" xmlns:wsse="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-secext-1…" xmlns:wsu="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-utility-…" wsu:Id="TS-1a8236fc-8e3a-9b71-495f-20f52709e893"><wsu:Created>2017-09-19T07:38:45Z</wsu:Created><wsu:Expires>2017-09-19T08:38:45Z</wsu:Expires></wsu:Timestamp>''' root_node = etree.fromstring(ORIGINAL_XML) test_node = root_node.find(".//{*}Timestamp") # This is the node to calculate digest for output = io.BytesIO() node = copy.copy(test_node) node.getroottree().write(output, method="c14n", inclusive_ns_prefixes=PREFIXES, exclusive=True, with_comments=False, pretty_print=False) output.seek(0) TEST_RESULT = output.read() assert TEST_RESULT == GOOD_C14N_XML The GOOD_C14N_XML contains the "good" canonical version of the Timestamp element. I say that it is "good" because for the ORIGINAL_XML, the C14N version created by a java program looks like this, and I must replicate this exact format. So by "good" I mean: this is the one that I need to replicate. In this example program, the assertion fails. The GOOD_C14N_XML contains all namespace declarations that were given in the PREFIXES, *in that specific order*. The TEST_RESULT only contains the wsu namespace. Questions: * Is it possible to achieve my goal with lxml? E.g. create a C14N format that matches the GOOD_C14N_XML in every bit. * I have noticed that the write method accepts pretty_print=True when write method is c14n. It has no effect. But shouldn't it throw an exception? (The same way it throws an exception when encoding is specified for c14n method.) Thanks, Laszlo

2 2

lxml 4.0.0 released
by Stefan Behnel 21 Sep '17

21 Sep '17

Hi all, I just released lxml 4.0.0 with several new features. Thanks to everyone who contributed. The documentation is here: http://lxml.de/ Download: https://pypi.python.org/packages/07/76/9f14811d3fb91ed7973a798ded15eda41607… Signature: https://pypi.python.org/packages/07/76/9f14811d3fb91ed7973a798ded15eda41607… Changelog: http://lxml.de/4.0/changes-4.0.0.html Github: https://github.com/lxml/lxml/releases/tag/lxml-4.0.0 This release was built using Cython 0.26.1. If you are interested in commercial support or customisations for the lxml package, please contact me directly. Have fun, Stefan 4.0.0 (2017-09-17) ================== Features added -------------- * The ElementPath implementation is now compiled using Cython, which speeds up the ``.find*()`` methods quite significantly. * The modules ``lxml.builder``, ``lxml.html.diff`` and ``lxml.html.clean`` are also compiled using Cython in order to speed them up. * ``xmlfile()`` supports async coroutines using ``async with`` and ``await``. * ``iterwalk()`` has a new method ``skip_subtree()`` that prevents walking into the descendants of the current element. * ``RelaxNG.from_rnc_string()`` accepts a ``base_url`` argument to allow relative resource lookups. * The XSLT result object has a new method ``.write_output(file)`` that serialises output data into a file according to the ``<xsl:output>`` configuration. Bugs fixed ---------- * GH#251: HTML comments were handled incorrectly by the soupparser. Patch by mozbugbox. * LP#1654544: The html5parser no longer passes the ``useChardet`` option if the input is a Unicode string, unless explicitly requested. When parsing files, the default is to enable it when a URL or file path is passed (because the file is then opened in binary mode), and to disable it when reading from a file(-like) object. Note: This is a backwards incompatible change of the default configuration. If your code parses byte strings/streams and depends on character detection, please pass the option ``guess_charset=True`` explicitly, which already worked in older lxml versions. * LP#1703810: ``etree.fromstring()`` failed to parse UTF-32 data with BOM. * LP#1526522: Some RelaxNG errors were not reported in the error log. * LP#1567526: Empty and plain text input raised a TypeError in soupparser. * LP#1710429: Uninitialised variable usage in HTML diff. * LP#1415643: The closing tags context manager in ``xmlfile()`` could continue to output end tags even after writing failed with an exception. * LP#1465357: ``xmlfile.write()`` now accepts and ignores None as input argument. * Compilation under Py3.7-pre failed due to a modified function signature. Other changes ------------- * The main module source files were renamed from ``lxml.*.pyx`` to plain ``*.pyx`` (e.g. ``etree.pyx``) to simplify their handling in the build process. Care was taken to keep the old header files as fallbacks for code that compiles against the public C-API of lxml, but it might still be worth validating that third-party code does not notice this change.

3 10