Mailman 3 June 2006 - lxml - The Python XML Toolkit

[lxml-dev] Resolving <xsl:include>
by Petr van Blokland 05 Jun '06

05 Jun '06

Hi, I get a very consistent error when including an XSL stylesheet from another XSL stylesheet using <xsl:include> or <xsl:import> But the problem is so basic that I cannot believe it could be something inside lxml. So I am overlooking something in my code. When there is a template file named "template.xsl" with ------------------------------------------------ <?xml version="1.0" encoding="UTF-8"?> <xsl:stylesheet version="1.0" xmlns:xsl="http://www.w3.org/1999/XSL/Transform" > <xsl:include href="para.xsl"/> <xsl:template match="*"/> <xsl:template match="/"> <foo>...</foo> </xsl:template> </xsl:stylesheet> ------------------------------------------------ and another template file named: ------------------------------------------------ <?xml version="1.0" encoding="UTF-8"?> <xsl:stylesheet version="1.0" xmlns:xsl="http://www.w3.org/1999/XSL/Transform" > <xsl:template match="para"> This is a paragraph </xsl:template> </xsl:stylesheet> ------------------------------------------------ and then executing the code below ------------------------------------------------ import lxml f = open('template.xsl', 'rb') xslttree = lxml.etree.parse(f) f.close() transformer = lxml.etree.XSLT(xslttree) ------------------------------------------------ the following error is raised: File "xslt.pxi", line 261, in etree.XSLT.__init__ File "etree.pyx", line 133, in etree._ExceptionContext._raise_if_stored etree.XSLTParseError: Cannot resolve URI XSLT://para.xsl All 3 files are in the same directory. And I am in that directory when executing the Python code. What do I do wrong that XSLT cannot resolve the para.xsl file? Petr ---------------------------------------------- Petr van Blokland buro(a)petr.com | www.petr.com | +31 15 219 10 40 ----------------------------------------------

2 3

[lxml-dev] Axis iteration
by Stefan Behnel 04 Jun '06

04 Jun '06

Hi all, I thought a bit more about the axis issues that Andrey raised. While list-like access to them is not a viable solution IMHO (and already covered by XPath calls anyway), I think it would still be nice to provide them as iterators. I therefore added a few new methods to Element: itersiblings(preceding=False) iterancestors() iterdescendants() Note that iterdescendants() is almost like getiterator(), except that it does not include the element itself, which is consistent with XPath's descendant axis. getiterator() is therefore the equivalent of the descendant-or-self axis. I think these methods are pretty Pythonic and make tree navigation fairly simple. Stefan

1 0

[lxml-dev] element of an xpath evaluation
by Andreas Pakulat 03 Jun '06

03 Jun '06

Hi, very recently I needed a program that evaluates a xpath and displays the "result" graphically in a tree of the xml file that was used. I found XPath Explorer a java application, however it does quite some more stuff than I need and there certain things that just don't work as I want. I figured I could very easily provide something similar using lxml and PyQt4. However if I want to highlight the tree node that the xpath matches I have a "problem" when the xpath matches attributes or text nodes. So the question is: Is there a way using lxml to find out to which element a certain non-element result of an xpath evaluation belongs? Andreas -- You now have Asian Flu.

2 7

[lxml-dev] "Extract" namespace from tag
by Andreas Pakulat 03 Jun '06

03 Jun '06

Hi, I know I had some discussions about this already but in a different context, I think. I'd like to be able to extract the namespace and namespace prefix for an element and it seems I can't do that with the current API (unless I'm overlooking something). Is there a way to make this work? Of course I can just "parse" the tag name for the namespace, but this doesn't give me the namespace prefix used. Andreas -- You will forget that you ever knew me.

2 5

[lxml-dev] Defining name spaces in lxml/xslt
by Petr van Blokland 02 Jun '06

02 Jun '06

Hi, new to lxml I may be overlooking some hints in the doc files. I am trying to define a name space (myspace) in Python, but I don't know by which method my name space object is getting called from lxml/xslt. class AbcElement(ElementBase): def ###METHODNAME###(self): return 'Output of abc' ns = Namespace('http://xml.petr.com/xpyth3/ns/myspace') ns['abc'] = AbcElement xslt = '''<xsl:stylesheet version="1.0" xmlns:xsl="http://www.w3.org/1999/XSL/Transform" xmlns:abc="http://xml.petr.com/xpyth3/xpath/ns/myspace" xsl:extension-element-prefixes='myspace' > <xsl:template match="/"> <foo> < myspace:abc/> </foo> </xsl:template> </xsl:stylesheet>''' Where ###METHODNAME### I don't know. I am using lxml 1.0 beta Suggestions? Kind regards, Petr van Blokland ---------------------------------------------- Petr van Blokland buro(a)petr.com | www.petr.com | +31 15 219 10 40 ----------------------------------------------

2 5

[lxml-dev] Element.getnext() and Element.getprevious() ?
by Stefan Behnel 02 Jun '06

02 Jun '06

Hi all, when I wrote the constant updater script, I noticed that navigating through an ElementTree to find the preceding sibling of an element is not trivial. However, that's not an unusual thing to do in HTML, where you might want to find a specific heading in the body, for example, and then look through the paragraphs belonging to the heading. It's ok as long as you stick with ET and traverse the tree yourself to find the heading. However, if you find the heading with XPath, you're lost as you can't easily find out how the XML structure continues at the same level... I'm therefore tempted to add the (trivially implemented) methods getnext() and getprevious() to Element, in the style of getparent(), getchildren() and gettreeroot(), but I wanted to ask here first if there are any objections to this extension. I think, we already have opened up the ET API towards a document based structure, so these would actually match the other extensions rather nicely. Stefan

3 12

Re: [lxml-dev] windows compiler error: string too big
by Stefan Behnel 02 Jun '06

02 Jun '06

Hi Steve, Steve Howe wrote: > I was trying to build the Windows eggs, when got the following: > > C:\Program Files\Microsoft Visual Studio .NET 2003\Vc7\bin\cl.exe /c /nologo /Ox /MD /W3 /GX /DNDEBUG -Iz:/xml/include - > IZ:\python24\include -IZ:\python24\PC /Tcsrc/lxml/etree.c /Fobuild\temp.win32-2.4\Release\src/lxml/etree.obj -w > cl : Command line warning D4025 : overriding '/W3' with '/w' > etree.c > src\lxml\etree.c(964) : error C2026: string too big, trailing characters truncated > src\lxml\etree.c(964) : error C2026: string too big, trailing characters truncated > src\lxml\etree.c(964) : error C2026: string too big, trailing characters truncated > src\lxml\etree.c(964) : error C2026: string too big, trailing characters truncated > src\lxml\etree.c(964) : error C2026: string too big, trailing characters truncated > src\lxml\etree.c(964) : error C2026: string too big, trailing characters truncated > src\lxml\etree.c(964) : error C2026: string too big, trailing characters truncated > src\lxml\etree.c(964) : error C2026: string too big, trailing characters truncated > src\lxml\etree.c(964) : error C2026: string too big, trailing characters truncated > src\lxml\etree.c(964) : error C2026: string too big, trailing characters truncated > src\lxml\etree.c(964) : error C2026: string too big, trailing characters truncated > src\lxml\etree.c(964) : error C2026: string too big, trailing characters truncated > src\lxml\etree.c(964) : error C2026: string too big, trailing characters truncated > src\lxml\etree.c(964) : error C2026: string too big, trailing characters truncated > src\lxml\etree.c(964) : error C2026: string too big, trailing characters truncated > error: command '"C:\Program Files\Microsoft Visual Studio .NET 2003\Vc7\bin\cl.exe"' failed with exit status 2 > > And indeed, that line is quite large (~32kb)... what should I do ? Oh well, how is that for a bug. :) > That compiler is the same which compiled Python, so its the "official" one. Then it's the right one to use. > Could that string be broken apart ? No problem. Here's a patch that does that. Compiles nicely on my machine. Likely adds nothing but a microsecond to the setup time, but is a little harder to maintain as we have to remember to apply something alike when we update the error code constants. Just apply the patch to the official version and compile that. I'll see if I can find a better patch for the trunk. As we know, most people won't compile under windows themselves, so that's not really something we need corrected in the source distribution. Sorry for the hassle. Stefan Index: src/lxml/xmlerror.pxi =================================================================== --- src/lxml/xmlerror.pxi (Revision 28057) +++ src/lxml/xmlerror.pxi (Arbeitskopie) @@ -561,6 +561,8 @@ XML_NS_ERR_QNAME = 202 : 202 XML_NS_ERR_ATTRIBUTE_REDEFINED = 203 : 203 XML_NS_ERR_EMPTY = 204 : 204 +""" + \ +""" XML_DTD_ATTRIBUTE_DEFAULT = 500 XML_DTD_ATTRIBUTE_REDEFINED = 501 : 501 XML_DTD_ATTRIBUTE_VALUE = 502 : 502 @@ -727,6 +729,8 @@ XML_RNGP_VALUE_NO_CONTENT = 1120 : 1120 XML_RNGP_XMLNS_NAME = 1121 : 1121 XML_RNGP_XML_NS = 1122 : 1122 +""" + \ +""" XML_XPATH_EXPRESSION_OK = 1200 XML_XPATH_NUMBER_ERROR = 1201 : 1201 XML_XPATH_UNFINISHED_LITERAL_ERROR = 1202 : 1202 @@ -1017,6 +1021,8 @@ XML_SCHEMAV_CVC_TYPE_2 = 1876 : 1876 XML_SCHEMAV_CVC_IDC = 1877 : 1877 XML_SCHEMAV_CVC_WILDCARD = 1878 : 1878 +""" + \ +""" XML_XPTR_UNKNOWN_SCHEME = 1900 XML_XPTR_CHILDSEQ_START = 1901 : 1901 XML_XPTR_EVAL_FAILED = 1902 : 1902

2 3

Re: [lxml-dev] etree.c
by Stefan Behnel 02 Jun '06

02 Jun '06

Hi Steve, Steve Howe wrote: > Friday, June 2, 2006, 3:56:45 AM, you wrote: >> here's a new etree.c that /should/ compile. All lines below 8k. > I was wrong about the max line size. It seems max size is 2k (argh!) 2k for a string? That's nothing! How can this thing call itself a C compiler? > The platform (MSVC6) docs for C2026: > > After adjacent strings are concatenated, a string cannot be longer > than 2048 characters. > > However, this should work: > > "huge part 1" "huge part 1" > > The compiler only gripes about a single string literal > 2K. Hu? But it says "after adjacent strings are concatenated"... Anyway, I can't do that from Pyrex code. Ok, so that means I have to revert the change that introduced these long strings, which are essentially long lists of constants, one per line. I did that to reduce the time it takes to compile. If we use Python lists to store them or Python objects that have them as attributes, Pyrex will generate tons of setup code that builds the objects or lists and that bloats the code and raises the compile time to the sky. A quick loop over a splitted string is just the fastest and smallest thing I could imagine. I'll see what I can come up with. I guess the best way is an auto-generator for these constants from the libxml2 source I currently cut&pasted them from. Maybe we should store them in C arrays and then have our own loop to generate Python strings from them. Stefan

1 0

[lxml-dev] namespaces
by Petr van Blokland 01 Jun '06

01 Jun '06

Hi lxml-developers, I have another question about the way lxml handles namespaces in combination with Python based elements: Think of an example derived from the doc/namespace_extension.txt from lxml.etree import Namespace, ElementBase namespace = Namespace('http://hui.de/honk') class HonkElement(ElementBase): def __str__(self): return 'String of this element' tree = XML('<a><b><c><honk xmlns="http://hui.de/honk" honking="true"/ ></c></b></a>') The question is: is it possible to make the tree build to xml by using lxml.etree.tostring(tree) where during the evaluation the __str__ (or another method) of the Python element is called. Otherwise the example as given in the doc will only work when the Python element is root of a tree as in: honk_element = XML('<honk xmlns="http://hui.de/honk" honking="true"><a><b/></a></honk>') Kind regards, Petr van Blokland ---------------------------------------------- Petr van Blokland buro(a)petr.com | www.petr.com | +31 15 219 10 40 ----------------------------------------------

2 1

[lxml-dev] lxml 1.0 is on cheeseshop!
by Stefan Behnel 01 Jun '06

01 Jun '06

Hallo everyone, I have the honour to announce the availability of lxml 1.0 on cheeseshop. While the list of features added since the beta version (1.0.beta) is rather small, this version contains a pretty large number of bug fixes found by various users and testers. Thank you all for your help! Features added since 0.9.2: * Element.getiterator() and the findall() methods support finding arbitrary elements from a namespace (pattern {namespace}*) * Another speedup in tree iteration code * General speedup of Python Element object creation and deallocation * Writing C14N no longer serializes in memory (reduced memory footprint) * PyErrorLog for error logging through the Python logging module * element.getroottree() returns an ElementTree for the root node of the document that contains the element. * ElementTree.getpath(element) returns a simple, absolute XPath expression to find the element in the tree structure * Error logs have a last_error attribute for convenience * Comment texts can be changed through the API * Formatted output via pretty_print keyword to serialization functions * XSLT can block access to file system and network via XSLTAccessControl * ElementTree.write() no longer serializes in memory (reduced memory footprint) * Speedup of Element.findall(tag) and Element.getiterator(tag) * Support for writing the XML representation of Elements and ElementTrees to Python unicode strings via etree.tounicode() * Support for writing XSLT results to Python unicode strings via unicode() * Parsing a unicode string no longer copies the string (reduced memory footprint) * Parsing file-like objects now reads chunks rather than the whole file (reduced memory footprint) * Parsing StringIO objects from the start avoids copying the string (reduced memory footprint) * Read-only 'docinfo' attribute in ElementTree class holds DOCTYPE information, original encoding and XML version as seen by the parser * etree module can be compiled without libxslt by commenting out the line include "xslt.pxi" near the end of the etree.pyx source file * Better error messages in parser exceptions * Error reporting now also works in XSLT * Support for custom document loaders (URI resolvers) in parsers and XSLT, resolvers are registered at parser level * Implementation of exslt:regexp for XSLT based on the Python 're' module, enabled by default, can be switched off with 'regexp=False' keyword argument * Support for exslt extensions (libexslt) and libxslt extra functions (node-set, document, write, output) * Substantial speedup in XPath.evaluate() * HTMLParser for parsing (broken) HTML * XMLDTDID function parses XML into tuple (root node, ID dict) based on xml:id implementation of libxml2 (as opposed to ET compatible XMLID) Bugs fixed since 0.9.2: * Memory leak in Element.__setitem__ * Memory leak in Element.attrib.items() and Element.attrib.values() * Memory leak in XPath extension functions * Memory leak in unicode related setup code * Element now raises ValueError on empty tag names * Namespace fixing after moving elements between documents could fail if the source document was freed too early * Setting namespace-less tag names on namespaced elements ('{ns}t' -> 't') didn't reset the namespace * Unknown constants from newer libxml2 versions could raise exceptions in the error handlers * lxml.etree compiles much faster * On libxml2 <= 2.6.22, parsing strings with encoding declaration could fail in certain cases * Document reference in ElementTree objects was not updated when the root element was moved to a different document * Running absolute XPath expressions on an Element now evaluates against the root tree * Evaluating absolute XPath expressions (/*) on an ElementTree could fail * Crashes when calling XSLT, RelaxNG, etc. with uninitialized ElementTree objects * Removed public function initThreadLogging(), replaced by more general initThread() which fixes a number of setup problems in threads * Memory leak when using iconv encoders in tostring/write * Deep copying Elements and ElementTrees maintains the document information * Serialization functions raise LookupError for unknown encodings * Memory deallocation crash resulting from deep copying elements * Some ElementTree methods could crash if the root node was not initialized (neither file nor element passed to the constructor) * Element/SubElement failed to set attribute namespaces from passed attrib dictionary * tostring() now adds an XML declaration for non-ASCII encodings * tostring() failed to serialize encodings that contain 0-bytes * ElementTree.xpath() and XPathDocumentEvaluator were not using the ElementTree root node as reference point * Calling document('') in XSLT failed to return the stylesheet I feel a certain fascination when I look back on the relatively short time it took Martijn and me (and several other contributors) to implement the large set of features that this version has and to bring it to this level of maturity. A big "Thank you!" to all code contributors, egg builders, bug finders, testers, users and everyone else who helped in bringing lxml towards 1.0! Stefan

2 1