Mailman 3 September 2010 - lxml - The Python XML Toolkit

[lxml-dev] lxml has its page on launchpad
by Stefan Behnel 11 Apr '23

11 Apr '23

Hi all, I added the lxml project to launchpad, the Ubuntu Bug-Tracker. It also has a FAQ engine and a couple of other goodies. https://launchpad.net/lxml It's easy to sign up for launchpad, BTW, no 90%-footnotes-contract. Have fun, Stefan

9 9

[lxml-dev] Checking whether a node is a comment/element
by Geoffrey Sneddon 10 Apr '23

10 Apr '23

Hi, What's the best way to check whether a given node is a comment or an element? For the former, I'm currently using isinstance(node, etree._Comment), which is rather obviously sub-optimal. -- Geoffrey Sneddon <http://gsnedders.com/>

6 6

[lxml-dev] Reparenting a node
by Lawrence Oluyede 30 Jan '23

30 Jan '23

I have a doc A and a doc B, I'd like to put a node extracted from A in the document B but I always get a ValueError: ValueError: Element is not a child of this node. I didn't find any "setparent" in the API. How can I do this? -- Lawrence, oluyede.org - neropercaso.it "It is difficult to get a man to understand something when his salary depends on not understanding it" - Upton Sinclair

3 2

[lxml-dev] lxml 2.0.5 released
by Stefan Behnel 11 Jan '23

11 Jan '23

Hi all, lxml 2.0.5 is on PyPI. This is a bug-fix-only release of the stable 2.0 series. Have fun, Stefan 2.0.5 (2008-05-01) Bugs fixed * Resolving to a filename in custom resolvers didn't work. * lxml did not honour libxslt's second error state "STOPPED", which let some XSLT errors pass silently. * Memory leak in Schematron with libxml2 >= 2.6.31.

3 4

[lxml-dev] Building LXML Trunk
by Sidnei da Silva 31 Aug '22

31 Aug '22

Hi, I've tried to build lxml from trunk today, on Win32. Got the following error: src\lxml\etree.c(880) : error C2059: syntax error : ')' src\lxml\etree.c(881) : error C2059: syntax error : ')' src\lxml\etree.c(882) : error C2059: syntax error : ')' src\lxml\etree.c(883) : error C2059: syntax error : ')' Any clue? Smells like a Pyrex issue? -- Sidnei da Silva Enfold Systems http://enfoldsystems.com Fax +1 832 201 8856 Office +1 713 942 2377 Ext 214

4 4

[lxml-dev] confusing xpath performance characteristics
by jholg＠gmx.de 24 Aug '12

24 Aug '12

Hi, I ran into some performance characteristics of lxml/libxml2 xpath that I find rather confusing: I try to find the @type attribute of a certain element in an XML Schema (which contains lots of complexType definitions with lots of elements in them; unfortunately I can't post the schema): >>> timeit.Timer(stmt="""xpath(schema)""", setup="""from lxml import etree, objectify; schema=etree.parse('NDM.xsd').getroot(); xpath = etree.XPath('//xs:element[@name="equity"]/@type', namespaces={'xs': 'http://www.w3.org/2001/XMLSchema'})""").repeat(number=10) [0.095885038375854492, 0.096823930740356445, 0.096174955368041992] So I think I'm being smart and give a little more path information - reckoning that this should *improve* performance: >>> timeit.Timer(stmt="""xpath(schema)""", setup="""from lxml import etree, objectify; schema=etree.parse('NDM.xsd').getroot(); xpath = etree.XPath('/xs:schema//xs:element[@name="equity"]/@type', namespaces={'xs': 'http://www.w3.org/2001/XMLSchema'})""").repeat(number=10) [0.1770780086517334, 0.1775970458984375, 0.17748594284057617] Hm. Performance degrades slightly. I'm adding even more of the path to where my desired elements live in the schema: >>> timeit.Timer(stmt="""xpath(schema)""", setup="""from lxml import etree, objectify; schema=etree.parse('xsd/NDM.xsd').getroot(); xpath = etree.XPath('/xs:schema/xs:complexType//xs:element[@name="equity"]/@type', namespaces={'xs': 'http://www.w3.org/2001/XMLSchema'})""").repeat(number=10) [103.79744100570679, 103.83671712875366, 103.61817717552185] What??? >>> timeit.Timer(stmt="""xpath(schema)""", setup="""from lxml import etree, objectify; schema=etree.parse('/ae/data/pydev/hjoukl/NDM/SVN_CO/TRUNK/ndm/reference/xsd/NDM.xsd').getroot(); xpath = etree.XPath('/xs:schema/xs:complexType/*/xs:element[@name="equity"]/@type', namespaces={'xs': 'http://www.w3.org/2001/XMLSchema'})""").repeat(number=10) [0.044407129287719727, 0.044126987457275391, 0.044229030609130859] >>> Ok, this version's better than my naive approach, which seems logical to me. But why would '/xs:schema/xs:complexType//xs:element[@name="equity"]/@type' perform drastically slower than '/xs:schema/xs:complexType//xs:element[@name="equity"]/@type' ? libxml2 problem? Running the same xpaths in Oxygen I don't notice performance differences (can't profile this). Holger -- GRATIS für alle GMX-Mitglieder: Die maxdome Movie-FLAT! Jetzt freischalten unter http://portal.gmx.net/de/go/maxdome01

2 5

[lxml-dev] DLL issue with win32 static builds of lxml
by Scott Smith 04 Apr '11

04 Apr '11

Hi I'm looking at upgrading our python installations from 2.6(.2) to 2.7 and upgrading the packages we use to the latest versions. We use python and the mod_wsgi module in Apache 2.2 for our application server and easy_install to install egg packages. I'm currently testing on a Windows 7 machine. The current egg version of lxml that we're using (before upgrading) is: lxml-2.2.4-py2.6-win32.egg In trying to upgrade things, I've found that all the later builds of lxml cause a DLL problem through apache. For example, all of these builds (from pipy) cause the problem: lxml-2.2.8-py2.6-win32.egg lxml-2.2.8-py2.7-win32.egg lxml-2.2beta1-py2.6-win32.egg lxml-2.3beta1-py2.6-win32.egg lxml-2.3beta1-py2.7-win32.egg (Note that there are no static win32 builds available for 2.2.5-2.2.7, or I can't find them) The error thrown in the Apache logs is: mod_wsgi (pid=4812): Target WSGI script 'C:/wsgi/handler.py' cannot be loaded as Python module. mod_wsgi (pid=4812): Exception occurred processing WSGI script 'C:/wsgi/handler.py'. Traceback (most recent call last): File "C:/wsgi/handler.py", line 2, in <module> from lxml import etree ImportError: DLL load failed: The specified module could not be found. The same file works in Python 2.6 if lxml 2.2.4 is installed, but fails for any other higher version of lxml (either in python 2.6 or 2.7). I've been able to trace the problem down to the builds not being able to find the msvcr90.dll file properly. I've tried modifying path variables and other changes, but can't figure out what the problem is. A couple of other things to note: - Using just "import lxml" doesn't cause the error, just "from lxml import etree" - The problem only occurs through Apache. If I write a script that references etree and call it through the command line, the problem doesn't occur. Does anyone know what I need to do? What changed after the 2.2.4 lxml builds? Thanks for any help you can give. Scott Smith

5 5

[lxml-dev] SyntaxErrors with Python 3
by Arfrever Frehtes Taifersar Arahesis 23 Nov '10

23 Nov '10

LXML r76211 generally supports Python 3, but there are still some SyntaxErrors. (The problem with ez_setup.py can be ignored.) $ python3.1 -m compileall -q . *** Error compiling ./benchmark/bench_etree.py ... File "./benchmark/bench_etree.py", line 10 UTEXT = u"some klingon: \F8D2" ^ SyntaxError: invalid syntax *** Error compiling ./benchmark/benchbase.py ... File "./benchmark/benchbase.py", line 10 _UTEXT = u"some klingon: \F8D2" * TREE_FACTOR ^ SyntaxError: invalid syntax *** Error compiling ./doc/mklatex.py ... File "./doc/mklatex.py", line 247 print "Creating %s" % outname ^ SyntaxError: invalid syntax *** Error compiling ./doc/rest2html.py ... File "./doc/rest2html.py", line 41 except ValueError, e: ^ SyntaxError: invalid syntax *** Error compiling ./doc/rest2latex.py ... File "./doc/rest2latex.py", line 44 except ValueError, e: ^ SyntaxError: invalid syntax *** Error compiling ./doc/s5/rst2s5.py ... File "./doc/s5/rst2s5.py", line 82 parsed = highlight(u'\n'.join(self.content), lexer, formatter) ^ SyntaxError: invalid syntax *** Error compiling ./ez_setup.py ... File "./ez_setup.py", line 106 except pkg_resources.VersionConflict, e: ^ SyntaxError: invalid syntax *** Error compiling ./src/local_doctest.py ... File "./src/local_doctest.py", line 372 raise TypeError, 'Expected a module: %r' % module ^ SyntaxError: invalid syntax *** Error compiling ./src/lxml/html/_diffcommand.py ... File "./src/lxml/html/_diffcommand.py", line 85 print "Not yet implemented" ^ SyntaxError: invalid syntax *** Error compiling ./src/lxml/html/tests/transform_feedparser_data.py ... File "./src/lxml/html/tests/transform_feedparser_data.py", line 91 print 'Bad data in %s:' % filename ^ SyntaxError: invalid syntax *** Error compiling ./src/lxml/tests/test_errors.py ... File "./src/lxml/tests/test_errors.py", line 1 # -*- coding: utf-8 -*- ^ SyntaxError: invalid character in identifier *** Error compiling ./tools/xpathgrep.py ... File "./tools/xpathgrep.py", line 5 except ImportError, e: ^ SyntaxError: invalid syntax -- Arfrever Frehtes Taifersar Arahesis

2 4

[lxml-dev] XSLT: Issues encountered when transforming docbook
by Jérôme Carretero 01 Nov '10

01 Nov '10

Hi all, Using a python-based build tool, I decided to use lxml directly instead of calling xsltproc in order to transform docbook sources. The docbook sources contain olinks (links between documents). Everything works fine when using xsltproc. I tried the following : def xsltproc(xml_filename, xslt_filename, **kw): parser = etree.XMLParser() xml_doc = etree.parse(xml_filename, parser) print xml_doc.getroot().base xml_doc.xinclude() xslt_doc = etree.parse(xslt_filename) ac = etree.XSLTAccessControl(read_network=True, write_file=True, read_file=True, create_dir=True) transform = etree.XSLT(xslt_doc, access_control=ac) result_tree = transform(xml_doc, **kw) res = etree.tostring(result_tree, pretty_print=True, encoding="utf-8", xml_declaration=True) print "Transform\n", transform.error_log print "Parser\n", parser.error_log if not res: raise Exception("pouet") return res kw = { "olink.base.uri" : "doc.html", "collect.xref.targets" : "yes", "targets.filename" : "doc.html.db", "target.database.document" : "olinkdb-html.xml", } res = xsltproc("doc.xml", ".../xsl/xhtml/docbook.xsl", **kw) with open("pouet", "w") as f: f.write(res) This code fails silently (Exception("pouet") is raised). If I comment the line with remove collect.xref.targets, a document is correctly output. But the olinks are screwed. Somehow the xslt transformer is not able to read the olink information of other documents, by reading the pointers in olinkdb-html.xml. It seems that the silent failure is a bug, and I may be missing some stuff in order to convert the documents properly. More complete test cases available upon request. Thank you all, -- cJ

3 8

[lxml-dev] lxml.html.submit_form and unicode values
by Eugene Van den Bulke 11 Oct '10

11 Oct '10

Hi, Slowly but surely learning more about lxml.html by using it to do some scrapping. I encountered a unicode problem trying to submit the following form. <form name="Lien1" method="POST" action="http://recherche2.assemblee-nationale.fr/resultats_tribun.jsp" id="Lien1"> <input type="hidden" name="id_auteur" value="Aboud Élie"> <input type="hidden" name="nom_auteur" value="Élie Aboud"> <input type="hidden" name="legislature" value="13"> <input type="hidden" name="typedoc" value="Questions"> </form> Which can be found under the Questions link of http://www.assemblee-nationale.fr/13/tribun/fiches_id/267457.asp#P3 ===== UnicodeEncodeError Traceback (most recent call last) /Users/eugene/Documents/Dev/parlorama/code/<ipython console> in <module>() /opt/local/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/site-packages/lxml/html/__init__.pyc in submit_form(form, extra_values, open_http) 819 if open_http is None: 820 open_http = open_http_urllib --> 821 return open_http(form.method, form.action, values) 822 823 def open_http_urllib(method, url, values): /opt/local/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/site-packages/lxml/html/__init__.pyc in open_http_urllib(method, url, values) 836 data = None 837 else: --> 838 data = urlencode(values) 839 return urlopen(url, data) 840 /opt/local/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/urllib.pyc in urlencode(query, doseq) 1267 for k, v in query: 1268 k = quote_plus(str(k)) -> 1269 v = quote_plus(str(v)) 1270 l.append(k + '=' + v) 1271 else: UnicodeEncodeError: 'ascii' codec can't encode character u'\xc9' in position 6: ordinal not in range(128) ===== I tried to address the problem by encoding the values in the form fields as suggested here : http://mail.python.org/pipermail/tutor/2007-May/054340.html but in a python shell doing >>> form.fields['id_auteur'] u'Aboud \xc9lie' >>> form.fields['id_auteur'] = form.fields['id_auteur'].encode('utf-8') [...] ValueError: All strings must be XML compatible: Unicode or ASCII, no NULL bytes Would welcome advice or guidance ... if I want to make urlopen "happy" I am "displeasing" ElementTree :( Thanks for your help, -- EuGeNe -- I lend my books on COlivri http://www.colivri.org/user/eugene, do you?

3 4