Mailman 3 August 2013 - lxml - The Python XML Toolkit

[lxml-dev] lxml has its page on launchpad
by Stefan Behnel 11 Apr '23

11 Apr '23

Hi all, I added the lxml project to launchpad, the Ubuntu Bug-Tracker. It also has a FAQ engine and a couple of other goodies. https://launchpad.net/lxml It's easy to sign up for launchpad, BTW, no 90%-footnotes-contract. Have fun, Stefan

9 9

[lxml-dev] Checking whether a node is a comment/element
by Geoffrey Sneddon 10 Apr '23

10 Apr '23

Hi, What's the best way to check whether a given node is a comment or an element? For the former, I'm currently using isinstance(node, etree._Comment), which is rather obviously sub-optimal. -- Geoffrey Sneddon <http://gsnedders.com/>

6 6

[lxml-dev] Reparenting a node
by Lawrence Oluyede 30 Jan '23

30 Jan '23

I have a doc A and a doc B, I'd like to put a node extracted from A in the document B but I always get a ValueError: ValueError: Element is not a child of this node. I didn't find any "setparent" in the API. How can I do this? -- Lawrence, oluyede.org - neropercaso.it "It is difficult to get a man to understand something when his salary depends on not understanding it" - Upton Sinclair

3 2

[lxml-dev] lxml 2.0.5 released
by Stefan Behnel 11 Jan '23

11 Jan '23

Hi all, lxml 2.0.5 is on PyPI. This is a bug-fix-only release of the stable 2.0 series. Have fun, Stefan 2.0.5 (2008-05-01) Bugs fixed * Resolving to a filename in custom resolvers didn't work. * lxml did not honour libxslt's second error state "STOPPED", which let some XSLT errors pass silently. * Memory leak in Schematron with libxml2 >= 2.6.31.

3 4

[lxml-dev] Building LXML Trunk
by Sidnei da Silva 31 Aug '22

31 Aug '22

Hi, I've tried to build lxml from trunk today, on Win32. Got the following error: src\lxml\etree.c(880) : error C2059: syntax error : ')' src\lxml\etree.c(881) : error C2059: syntax error : ')' src\lxml\etree.c(882) : error C2059: syntax error : ')' src\lxml\etree.c(883) : error C2059: syntax error : ')' Any clue? Smells like a Pyrex issue? -- Sidnei da Silva Enfold Systems http://enfoldsystems.com Fax +1 832 201 8856 Office +1 713 942 2377 Ext 214

4 4

lxml.objectify drops leading whitespace
by Wolfgang Schnerring 02 Oct '13

02 Oct '13

Hello, lxml.objectify drops leading whitespace before a tag: >>> lxml.etree.tostring(lxml.objectify.fromstring(' <a>foo</a>')) '<a>foo</a>' while lxml.etree handles the same situation just fine: >>> lxml.etree.tostring(lxml.etree.fromstring(' <a>foo</a>')) ' <a>foo</a>' What's the reason for the different behaviour? Is this intentional? Most importantly, how can I make lxml.objectify preserve whitespace correctly? Thanks for your help, Wolfgang

2 2

Base file problem with XInclude
by Radek Pazdera 31 Aug '13

31 Aug '13

Hello, I'm using lxml to resolve XIncludes in a document. This works very well, however, I need to know the file each element originates from for error reporting. If I understand that correclty, the location should be stored in the 'base' and 'sourceline' variables for each element. Unfortunately, I can't seem to get the right file name from there. What am I getting is the same base for all the included files files. The line numbers are correct though. Am I doing something wrong or could that be a bug? I use python-lxml-3.2.1 and here is what I'm working with: $ cat parent.xml <parent xmlns:xi="http://www.w3.org/2001/XInclude"> <xi:include href="level1.xml"/> <xi:include href="level2.xml"/> </parent> $ cat level1.xml <level1 xmlns:xi="http://www.w3.org/2001/XInclude"> <xi:include href="level2.xml"/> </level1> $ cat level2.xml <level2> <content/> </level2> -------- The Code -------- $ cat try.py #!/usr/bin/python import sys from lxml import etree from StringIO import StringIO def print_elem(e): print e.tag, e.base, e.sourceline for c in e.getchildren(): print_elem(c) doc = etree.parse(open("parent.xml", "r")) doc.xinclude() print etree.tostring(doc.getroot()) print "" print_elem(doc.getroot()) print "" # example from: # http://lxml.de/api.html#xinclude-and-elementinclude data = StringIO('''\ <doc xmlns:xi="http://www.w3.org/2001/XInclude"> <foo/> <xi:include href="level2.xml"/> </doc>''') tree = etree.parse(data) tree.xinclude() print_elem(tree.getroot()) print "" print("%-20s: %s" % ('Python', sys.version_info)) print("%-20s: %s" % ('lxml.etree', etree.LXML_VERSION)) print("%-20s: %s" % ('libxml used', etree.LIBXML_VERSION)) print("%-20s: %s" % ('libxml compiled', etree.LIBXML_COMPILED_VERSION)) print("%-20s: %s" % ('libxslt used', etree.LIBXSLT_VERSION)) print("%-20s: %s" % ('libxslt compiled', etree.LIBXSLT_COMPILED_VERSION)) ------ Output ------ $ ./try.py <parent xmlns:xi="http://www.w3.org/2001/XInclude"> <level1 xmlns:xi="http://www.w3.org/2001/XInclude"> <level2> <content/> </level2> </level1> <level2> <content/> </level2> </parent> parent /home/rpazdera/work/lxml_bug/parent.xml 1 level1 /home/rpazdera/work/lxml_bug/parent.xml 1 level2 /home/rpazdera/work/lxml_bug/parent.xml 1 content /home/rpazdera/work/lxml_bug/parent.xml 2 level2 /home/rpazdera/work/lxml_bug/parent.xml 1 content /home/rpazdera/work/lxml_bug/parent.xml 2 doc None 1 foo None 2 level2 None 1 content None 2 Python : sys.version_info(major=2, minor=7, micro=5, releaselevel='final', serial=0) lxml.etree : (3, 2, 1, 0) libxml used : (2, 9, 1) libxml compiled : (2, 9, 1) libxslt used : (1, 1, 28) libxslt compiled : (1, 1, 28) Thanks, Radek Pazdera

2 1

Surprising slicing behaviour
by Frank Millman 26 Aug '13

26 Aug '13

Hi all I noticed a behaviour in lxml that surprised me, so I thought I would mention it. Versions - python 3.3.2, lxml 3.2.1. Consider the following - standard python slicing - >>> a = [1, 2, 3, 4] >>> b = [5, 6, 7, 8] >>> a[:0] = b[0:] >>> a [5, 6, 7, 8, 1, 2, 3, 4] >>> b [5, 6, 7, 8] This is what I expected. Then look at this - lxml slicing - >>> from lxml import etree >>> a = etree.fromstring('<a1><a1.1/><a1.2/></a1>') >>> b = etree.fromstring('<b1><b1.1/><b1.2/></b1>') >>> a[:0] = b[0:] >>> etree.tostring(a) b'<a1><b1.1/><b1.2/><a1.1/><a1.2/></a1>' >>> etree.tostring(b) b'<b1/>' >>> The slicing worked, but it removed the contents of b. Is this intentional? Frank Millman

2 2

Possible bug with lxml.etree.fromstring('<value>]]></value>').text
by Kees Bos 16 Aug '13

16 Aug '13

Hi all, I get an error when I try to parse a text element like this: <value>]]></value> >>> lxml.etree.fromstring('<value>]]></value>').text Traceback (most recent call last): File "<stdin>", line 1, in <module> File "lxml.etree.pyx", line 2756, in lxml.etree.fromstring (src/lxml/lxml.etree.c:54726) File "parser.pxi", line 1578, in lxml.etree._parseMemoryDocument (src/lxml/lxml.etree.c:82843) File "parser.pxi", line 1457, in lxml.etree._parseDoc (src/lxml/lxml.etree.c:81641) File "parser.pxi", line 965, in lxml.etree._BaseParser._parseDoc (src/lxml/lxml.etree.c:78311) File "parser.pxi", line 569, in lxml.etree._ParserContext._handleParseResultDoc (src/lxml/lxml.etree.c:74567) File "parser.pxi", line 650, in lxml.etree._handleParseResult (src/lxml/lxml.etree.c:75458) File "parser.pxi", line 590, in lxml.etree._raiseParseError (src/lxml/lxml.etree.c:74791) lxml.etree.XMLSyntaxError: Sequence ']]>' not allowed in content, line 1, column 8 I'm not an expert, but from: http://www.w3.org/TR/REC-xml/#NT-AttValue AttValue ::= '"' ([^<&"] | Reference)* '"' | "'" ([^<&'] | Reference)* "'" which I read as: Any Reference character is valid, except & and <, which are used for escaping and closing the element. The sequence <value>]]></value> also validates as well-formed at http://www.xmlvalidation.com/ The sequence <value>]></value> parses OK (So, it's only with a double ] and > ) It's probably related to parsing <![CDATA[ ... ]]> (i.e. I guess when the parser detects ]]> it assumes / requires the state of <![CDATA[ which is, of course, not true) The sequence <value><![CDATA[foo]]></value> is parsed correctly: >>> lxml.etree.fromstring('<value><![CDATA[foo]]></value>').text 'foo' Any ideas whether this is really a bug? - Kees

2 1

Problems using LXML with Mod WSGI
by Lee E. Brown 12 Aug '13

12 Aug '13

Greetings! I am using Apache 2.2.25, Mod WSGI 3.3, LXML 3.2.1, and Python 2.7.2 My web site scheme stores web pages as XML documents which are transformed into XHTML on-the-fly using XSLT. Previously I had been using a mod_python output filter and I am now trying to update my system to use mod_wsgi. I have converted the mod_python output filter into an equivalent mod_wsgi script as shown below. This script when run outside of mod_wsgi functions perfectly. When it is run inside mod_wsgi it goes into the etree.XSLT() method on line 57 and there it hangs up until Apache times out the request. No information is written to the errorlog files. I have gone through all the known application issues and debugging suggestions given on the mod_wsgi wikki to no avail. Any help at this point would be greatly appreciated. """ Transform an XML document into an XHTML document """ import sys, os, time from cStringIO import StringIO sys.path.append('C:/webroot/sites/wsgitest/wsgiscripts') sys.path.append('C:/webroot/sites/wsgitest/content') sys.path.append('C:/Python27/lib/site-packages/lxml') from lxml import etree from fileutils import * from nsutils import * ns = {'pysite': 'http://www.leebrown.org/pysite'} def paramify (parent, target={}): """ A very quick and dirty objectification function. Given an arbitrary parent etree element, creates a dictionary of {local-name:text} from each child element. Also adds necessary embedded quotes to make the dictionary suitable to pass as XSLT parameters. Requires function lname() from module nsutils.py """ for child in parent.getchildren(): target[lname(child.tag)] = "'%s'" % (child.text) return target def parse_source (streambuffer): """ Converts the source XML held in streambuffer into an lxml Element Tree. Source includes are also processed at this time. """ streambuffer.seek(0) xml_tree = etree.parse(streambuffer, etree.XMLParser(remove_blank_text=True)) xml_tree.xinclude() return xml_tree def parse_template (xml_tree): """ Attempts to create an XSLT transform object from an xml-stylesheet PI in the source tree. Returns a None object if no xml-stylesheet PI is found; this causes the transform() function below to return untransformed XML to the client. """ xsl_pi = xml_tree.getroot().getprevious() xsl_tree = xsl_pi.parseXSL() if xsl_tree: return etree.XSLT(xsl_tree) else: return None def get_params (xml_tree): """ If the source XML contains a <page-params> element in the target namespace, convert all child elements of this element into an XLST parameter dictionary. Other parameters may be added on as desired. """ try: params = paramify(xml_tree.xpath('//pysite:page-params', namespaces=ns)[0]) except IndexError: params = {} return params def transform (streambuffer): """ This is where most of the fun stuff happens. If parse_template() returns a transformer object, the source XML tree is transformed, and any xincludes added in the transform are then processed. If parse template() does not return a tranformer object, this function returns the unparsed, untransformed XML source. """ xml_tree = parse_source(streambuffer) transformer = parse_template(xml_tree) if transformer: params = get_params(xml_tree) result_tree = transformer(xml_tree, **params) result_tree.xinclude() return result_tree else: return streambuffer.getvalue() def parse_request(environ): path = environ['PATH_INFO'].lstrip('/') filespec = smoothpath('C:/webroot/sites/wsgitest/content', path, 'page.xml') return filespec def application(environ,start_response): start = time.clock() buffer = StringIO() try: buffer.write(readfile(parse_request(environ))) result_tree = transform(buffer) end = time.clock() try: # Looks for %f and %i in output, inserts transform time # and pages per second if found result = str(result_tree) % (float(end - start), int(1.0/(end - start))) except: result = str(result_tree) status = '200 OK' except IOError: status = '404 FILE NOT FOUND' result = '404 FILE NOT FOUND' finally: # Prepare HTTP response response_headers = [('Content-Type','text/html'),('Content-Length',str(len(result)))] start_response(status,response_headers) return([result]) if __name__ == '__main__': def start_response(status, response_headers): pass environ = {} environ['PATH_INFO'] = '/' result = application(environ, start_response) for line in result: print line -- Best Regards, Lee E. Brown

3 2