Mailman 3 April 2007 - lxml - The Python XML Toolkit

[lxml-dev] Pickling objectified trees
by Christian Zagrodnick 09 Jun '08

09 Jun '08

Hi, the other day I had to pickle objectified trees. I just thought to share my findings. Pickling is about serialization. IMHO the natural serialization of an objectified tree is its XML representation. So the following basically does that: -------------------------- import copy_reg import lxml.etree import lxml.objectify def treeFactory(state): """Un-Pickle factory.""" return lxml.objectify.fromstring(state) copy_reg.constructor(treeFactory) def reduceObjectifiedElement(object): """Reduce function for lxml.objectify trees. See http://docs.python.org/lib/pickle-protocol.html for details. """ return (treeFactory, (lxml.etree.tostring(object), )) copy_reg.pickle(lxml.objectify.ObjectifiedElement, reduceObjectifiedElement, treeFactory) ----------------------------------------- You might consider just registering the reduce function in lxml itself. Shouldn't hurt, should it. -- Christian Zagrodnick gocept gmbh & co. kg · forsterstrasse 29 · 06112 halle/saale www.gocept.com · fon. +49 345 12298894 · fax. +49 345 12298891

2 4

[lxml-dev] lxml namespaces
by Maxim Sloyko 20 Feb '08

20 Feb '08

Hi All! I have a little problem with XML namespaces. In my application I have two XML processors, that process the same document, one after the other. The first one looks for nodes in 'ns1' namespace, and substitutes them, according to some algorithm. After this processor is finished, it is guaranteed that there are no more 'ns1' nodes left in the tree. 'ns1' namespace dclaration is still there, in the root node (well, I put it there manually). Now, when this namespace is no longer needed, I want to get rid of it, because it confuses some other processors (namely, my browser) So, the question is, how do I do that? del tree.getroot().nsmap['ns1'] does not seem to do the trick :( Thanks in advance and Happy Holidays! -- Maxim Sloyko

3 5

[lxml-dev] [objectify] schema type registry: QNames for xsi:type?
by jholg＠gmx.de 14 May '07

14 May '07

Hi, I just detected a problem with the xsi-types in objectify type registry in that they are no QNames: >>> schematree = etree.fromstring(""" ... <xsd:schema xmlns:xsd="http://www.w3.org/2001/XMLSchema"> ... ... <xsd:element name="root" type="RootType"/> ... ... <xsd:complexType name="RootType"> ... <xsd:sequence> ... <xsd:element name="s" type="xsd:string"/> ... </xsd:sequence> ... </xsd:complexType> ... </xsd:schema> ... """) >>> schema = etree.XMLSchema(schematree) >>> msg = etree.fromstring("""<root xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:xsd="http://www.w3.org/2001/XMLSchema"><s xsi:type="xsd:string">2387</s></root>""") >>> print schema.validate(msg) 1 >>> print objectify.dump(msg) root = None [ObjectifiedElement] s = 2387 [IntElement] * xsi:type = 'xsd:string' >>> Note that s is an IntElement wherease it should be a StringElement. This goes away if changing its xsi:type to "string"; however, the doc instance then isn't valid against the schema anymore: >>> msg = etree.fromstring("""<root xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:xsd="http://www.w3.org/2001/XMLSchema"><s xsi:type="string">2387</s></root>""") >>> print schema.validate(msg) 0 >>> print objectify.dump(msg) root = None [ObjectifiedElement] s = '2387' [StringElement] * xsi:type = 'string' >>> Is it easily possible to use QNames in the xsi-type lookup system? Holger -- "Feel free" - 10 GB Mailbox, 100 FreeSMS/Monat ... Jetzt GMX TopMail testen: http://www.gmx.net/de/go/topmail

2 3

[lxml-dev] Whoops, Internal Error
by martin＠martinthomas.net 01 May '07

01 May '07

Using the lxml rpm for FC6 and Python 2.4, I get an internal error when I try validating a document against a XMLschema document. The xml document that I am trying to validate and the XMLschema which I am validating against both came from NIST (contained in the 'Complete 1.1.3 Schema Bundle .zip' at http://nvd.nist.gov/scap/xccdf/xccdf.cfm). The error message reads Internal error: xmlSchemaIDCRegisterMatchers, Could not find an augmented IDC item for an IDC definition. I'll write this up properly tonight and send in an error log, along with all the schema documents etc unless someone tells me otherwise. Cheers // Martin

3 2

[lxml-dev] Call for contribution towards lxml 1.3
by Stefan Behnel 30 Apr '07

30 Apr '07

Hi all, lxml 1.3 is nearing completion. There were some major changes under the hood, but the most visible part of the new release is actually the new layout of the documentation site, which should make it much more accessible. As usual, the preview is here: http://codespeak.net/lxml/dev/ Some of you have mentioned their impression that it's hard to help out on lxml as it's written in Pyrex, not Python. Although the current code looks very C-ish in many places, this is more of a performance optimisation than a real requirement. Pyrex actually makes it possible to work on the code in a very Python-like style, and to make the C-ification a matter of later improvement. So Python(-like) implementations of new features are definitely welcome. A non-optimised implementation of an interesting feature is much better than the lack of this feature would be. So, everyone is invited to get involved in making the code even better than it is today. But there is another area where help is appreciated. A very important area in fact: *documentation*. While there is quite a bit of documentation both on ElementTree and lxml, there are certainly places where lxml's API and its way of doing XML are hard to access, especially for new users and those who have a fixed (should I say: Java-ish?) mindset on XML. If you want to contribute, helping out in this area is warmly appreciated. Here are a few ideas that would be truely helpful for lxml's user base. * I would love to see lxml's own tutorial that gets the main ideas and the most useful features across without caring too much about ElementTree (which already has a tutorial). * Some statistics: what /are/ the most useful features of lxml? What do people like or use most? What parts of lxml should be more accessible? Which parts are so well done that people grasp their usage immediately (and should therefore be promoted as an eye-catcher)? * We could benefit from a Wiki where users could contribute code examples, best practices, work-arounds or tool snippets. We should also start linking to external pages, blogs, presentations on lxml or ElementTree that others might find interesting. Obviously, this list is not complete, so if you want to contribute, I hope you will easily find places to do so. Please help us in making lxml 1.3 the best release ever - and the most accessible one! Have fun, Stefan

5 9

[lxml-dev] Error installing lxml
by Róger Aguilar 26 Apr '07

26 Apr '07

4 5

[lxml-dev] lxml and binary eggs on Linux
by Martijn Faassen 26 Apr '07

26 Apr '07

Hi there, In the past we've been in the habit of providing binary eggs for lxml on Linux. We've been less diligent about this recently, which is actually a good thing. I would in fact ask everybody to stop uploading binary eggs for Linux, and only do so for Windows. Why? Python interpreters on the Linux world are compiled with different options. Prominent here is UCS2 versus UCS4 for the internal unicode encoding. An egg compiled for a UCS4 python doesn't work on a UCS2 python and vice versa. There are other potential issues, such as the location of various shared libraries that might differ per platform. Uploading a binary egg means that we risk making life worse for some users, as they'll be stuck with a non-working egg. If we only upload the source (including the generated C code), lxml will compile and install itself and this should be reliable on all Linux boxes. This does however mean that people need to install the libxml2 and libxslt headers on their system (libxml2-dev and libxstl-dev on debian/ubuntu), otherwise the compile would fail. It would also mean we need to modify our installation instructions. Unfortunately I don't see any other way to make lxml installation more reliable on Linux, though. On Windows, because nobody has a compiler and the platform is more uniform (practically everybody runs the same compiled version of Python), we don't have this problem. In fact we have the problem that nobody has a compiler, so we certainly need to continue uploading the binary eggs. Comments? Regards, Martijn

2 1

[lxml-dev] Building lxml on PPC Mac 10.4
by David Pratt 26 Apr '07

26 Apr '07

Hi. I am having trouble building lxml on PPC Mac with my buildouts and also easy_setup. It seems to build with warnings and killed my python as soon as it was accessed. The following is what happens with the build. I compiled and ran previously on PPC python without trouble. I am now running a Universally build python 2.4.4 with OSX 10.4.9 on a PPC. Many thanks. Regards, David Buildout build ============== zc.buildout.easy_install: Getting new distribution for lxml Building lxml version 1.3.beta warning: no previously-included files found matching 'doc/pyrex.txt' warning: no previously-included files found matching 'src/lxml/etree.pxi' /usr/bin/ld: for architecture i386 /usr/bin/ld: warning /opt/local/lib/libxslt.dylib cputype (18, architecture ppc) does not match cputype (7) for specified -arch flag: i386 (file not loaded) /usr/bin/ld: warning /opt/local/lib/libexslt.dylib cputype (18, architecture ppc) does not match cputype (7) for specified -arch flag: i386 (file not loaded) /usr/bin/ld: warning /opt/local/lib/libxml2.dylib cputype (18, architecture ppc) does not match cputype (7) for specified -arch flag: i386 (file not loaded) /usr/bin/ld: warning /opt/local/lib/libz.dylib cputype (18, architecture ppc) does not match cputype (7) for specified -arch flag: i386 (file not loaded) /usr/bin/ld: for architecture ppc /usr/bin/ld: warning can't open dynamic library: /Developer/SDKs/MacOSX10.4u.sdk/opt/local/lib/libiconv.2.dylib referenced from: /opt/local/lib/libxslt.dylib (checking for undefined symbols may be affected) (No such file or directory, errno = 2) /usr/bin/ld: for architecture i386 /usr/bin/ld: warning /opt/local/lib/libxslt.dylib cputype (18, architecture ppc) does not match cputype (7) for specified -arch flag: i386 (file not loaded) /usr/bin/ld: warning /opt/local/lib/libexslt.dylib cputype (18, architecture ppc) does not match cputype (7) for specified -arch flag: i386 (file not loaded) /usr/bin/ld: warning /opt/local/lib/libxml2.dylib cputype (18, architecture ppc) does not match cputype (7) for specified -arch flag: i386 (file not loaded) /usr/bin/ld: warning /opt/local/lib/libz.dylib cputype (18, architecture ppc) does not match cputype (7) for specified -arch flag: i386 (file not loaded) /usr/bin/ld: for architecture ppc /usr/bin/ld: warning can't open dynamic library: /Developer/SDKs/MacOSX10.4u.sdk/opt/local/lib/libiconv.2.dylib referenced from: /opt/local/lib/libxslt.dylib (checking for undefined symbols may be affected) (No such file or directory, errno = 2) zc.buildout.easy_install: Got lxml 1.3beta An easy_setup build =================== Searching for lxml Reading http://cheeseshop.python.org/pypi/lxml/ Reading http://cheeseshop.python.org/pypi/lxml/1.3beta Reading http://codespeak.net/lxml Reading http://cheeseshop.python.org/pypi/lxml/1.2.1 Best match: lxml 1.3beta Downloading http://cheeseshop.python.org/packages/source/l/lxml/lxml-1.3beta.tar.gz Processing lxml-1.3beta.tar.gz Running lxml-1.3beta/setup.py -q bdist_egg --dist-dir /tmp/easy_install-uCUEox/lxml-1.3beta/egg-dist-tmp-tOf7Pb Building lxml version 1.3.beta warning: no previously-included files found matching 'doc/pyrex.txt' warning: no previously-included files found matching 'src/lxml/etree.pxi' /usr/bin/ld: for architecture ppc /usr/bin/ld: warning can't open dynamic library: /Developer/SDKs/MacOSX10.4u.sdk/opt/local/lib/libiconv.2.dylib referenced from: /opt/local/lib/libxslt.dylib (checking for undefined symbols may be affected) (No such file or directory, errno = 2) /usr/bin/ld: for architecture i386 /usr/bin/ld: warning /opt/local/lib/libxslt.dylib cputype (18, architecture ppc) does not match cputype (7) for specified -arch flag: i386 (file not loaded) /usr/bin/ld: warning /opt/local/lib/libexslt.dylib cputype (18, architecture ppc) does not match cputype (7) for specified -arch flag: i386 (file not loaded) /usr/bin/ld: warning /opt/local/lib/libxml2.dylib cputype (18, architecture ppc) does not match cputype (7) for specified -arch flag: i386 (file not loaded) /usr/bin/ld: warning /opt/local/lib/libz.dylib cputype (18, architecture ppc) does not match cputype (7) for specified -arch flag: i386 (file not loaded) /usr/bin/ld: for architecture ppc /usr/bin/ld: warning can't open dynamic library: /Developer/SDKs/MacOSX10.4u.sdk/opt/local/lib/libiconv.2.dylib referenced from: /opt/local/lib/libxslt.dylib (checking for undefined symbols may be affected) (No such file or directory, errno = 2) /usr/bin/ld: for architecture i386 /usr/bin/ld: warning /opt/local/lib/libxslt.dylib cputype (18, architecture ppc) does not match cputype (7) for specified -arch flag: i386 (file not loaded) /usr/bin/ld: warning /opt/local/lib/libexslt.dylib cputype (18, architecture ppc) does not match cputype (7) for specified -arch flag: i386 (file not loaded) /usr/bin/ld: warning /opt/local/lib/libxml2.dylib cputype (18, architecture ppc) does not match cputype (7) for specified -arch flag: i386 (file not loaded) /usr/bin/ld: warning /opt/local/lib/libz.dylib cputype (18, architecture ppc) does not match cputype (7) for specified -arch flag: i386 (file not loaded) Adding lxml 1.3beta to easy-install.pth file Installed /Library/Frameworks/Python.framework/Versions/2.4/lib/python2.4/site-packages/lxml-1.3beta-py2.4-macosx-10.3-fat.egg Processing dependencies for lxml From my logs: ============ Apr 25 14:32:05 Mac-PG crashdump[27220]: Python crashed Apr 25 14:32:15 Mac-PG crashdump[27220]: crash report written to: /Users/davidpratt/Library/Logs/CrashReporter/Python.crash.log

1 2

[lxml-dev] lxml Mac OS X probs?
by Gary Poster 24 Apr '07

24 Apr '07

Hi all. I saw here http://www.openplans.org/projects/bbq-sprint/nudgenudge the following text at the bottom: """Note that the Deliverance middleware requires lxml to do the theming which is known to have problems on certain platforms, e.g. Mac OS X.""" Googling for such only found problems in 2005 and 2006, and I didn't see an obvious reference to these on the lxml main page or FAQ. Can anyone give me an idea of why that comment might have been made, and what the current issues are, if any? Pointing to a web page would be fine... Thanks Gary

4 4

[lxml-dev] Bug in XPath evaluation
by Torsten Rehn 24 Apr '07

24 Apr '07

Hi list, here's what I have: poc.xml: <?xml version="1.0" encoding="utf-8" ?> <myrootnode> <myns:mynode xmlns:myns="http.//www.example.com/myns"> <myns:mysubnode>some text</myns:mysubnode> </myns:mynode> </myrootnode> poc.py: #!/usr/bin/env python from lxml import etree DocTree = etree.parse("poc.xml") QueryResult = DocTree.xpath("//myns:mynode") The result (with added version info): [gentop][scel@/home/scel/workspace/lxmlbug] > ./poc.py lxml.etree: (1, 2, 1, 0) libxml used: (2, 6, 27) libxml compiled: (2, 6, 27) libxslt used: (1, 1, 17) libxslt compiled: (1, 1, 17) Traceback (most recent call last): File "./poc.py", line 9, in ? QueryResult = DocTree.xpath("//myns:mynode") File "etree.pyx", line 1256, in etree._ElementTree.xpath File "xpath.pxi", line 75, in etree._XPathEvaluatorBase.evaluate File "xpath.pxi", line 212, in etree.XPathDocumentEvaluator.__call__ File "xpath.pxi", line 105, in etree._XPathEvaluatorBase._handle_result File "xpath.pxi", line 93, in etree._XPathEvaluatorBase._raise_parse_error etree.XPathSyntaxError: error in xpath expression The expression however, is valid (or I'm just insanely stupid). I tested the same query on the same data using http://dmag.upf.edu/contorsion/query.jsp and it worked just as it should. Strangely, //*[name()='myns:mynode'] works with lxml. Regards, Torsten -- Torsten Rehn <scel(a)users.sourceforge.net>

4 10