Mailman 3 March 2007 - lxml - The Python XML Toolkit

Re: [lxml-dev] Adding a Stylesheet PI when creating new XML documents
by Stefan Behnel 26 Mar '07

26 Mar '07

Hi Lee, Lee Brown wrote: > As a work-around, I tried creating my XML document from a prototype > string that contained a stylesheet PI and an empty root element. That > part worked O.K. (tree.getroot().getprevious() returned the PI) but then > the PI was promptly discarded upon reserialization by etree.tostring()!!! Right, I noticed this also. It's not as simple as "here's the fix", though. The right way to deal with this would be to use the "xmlsave" API in libxml2. However, this API had a couple of problems with serialisation in the past, so this would require us to depend on a rather recent version of libxml2 (I think it was at least 2.6.24). I'm actually quite happy that we currently have lxml working across the last ten versions of libxml2 without major problems. You can see the current behaviour as a compromise between the targeted behaviour of Fredrik's ElementTree (which, for example, avoids XML declarations wherever possible) and the backwards compatibility required to work around libxml2's bugs. We could drop either of the two, i.e., serialise different (though equivalent) XML or start requiring libxml2 2.6.24. Given these two possibilities, I'd personally vote for keeping it the way it is. Still, I admit that this is annoying. While you can serialise to a StringIO object by using ET.write() - which should not suffer from this problem - it would definitely be better to provide the same behaviour for string serialisation, too. Maybe I could try to fix it at least for more recent libxml2 versions - if I find the time. Stefan

1 0

Re: [lxml-dev] Adding a Stylesheet PI when creating new XML documents
by Stefan Behnel 23 Mar '07

23 Mar '07

Hi, Lee Brown wrote: >> From: Stefan Behnel [mailto:stefan_ml@behnel.de] >> I personally liked the "addToTree()" method better as it >> allows us to accompany it with a "copyToTree()" method that >> would not remove the PI from its current location. Maybe >> "prependToTree()" and "prependCopyToTree()" would be even >> more descriptive names. > > Fair enough. I like "addToTree" and "copyToTree." "prependCopyToTree" is > starting to look too much like JavaScript :-) :) sure. Still time to find a better alternative, though. >>> Doesn't "tree.docinfo.stylesheet" or >> "tree.docinfo.PI['stylesheet']" >>> make more sense than the way we're doing it now? >> >> So, regarding your proposal: it will not quite work as >> nothing prevents an XML document from having 20 processing >> instructions of the same type. The meme above sort of >> suggests that there can be only one (note that we are talking >> about etree here, not objectify, which has a very different >> look-and-feel). >> >> Is there any obvious way that would allow us to handle the >> normal case of zero or one PIs per type as nicely as >> possible, while allowing us to deal with the possible case of >> having an arbitrary number of them? > > Well, we could do what the Xpath method does: Always return a list object, even > if it contains zero members. If multiple elements are returned, it would be up > to the consumer to determine which one to use. Sure, that's the obvious solution. (not quite like xpath() though, which can return loads of stuff, not only lists) > One possible way might be to give the ElementTree object a means to encapsulate > the entire document in a "phantom" element when needed. Tree.getroot(), > tree.xpath(), and tree.docinfo could still work the way they've always worked, > but a call to, say, document.something() could encapsulate the entire document > in a phantom <document> element and then all the child and sibling oriented > methods could be applied to the phantom element (document.getchildren(), > document.firstchild(), etc.) Validating parsers would want to mark any DOCTYPE > declaration as read-only, but non-validating parsers wouldn't care. Something like a "ET.gettoplevelelements()" method. Which sounds like it would rather return a list than an object, I'd say. > For all I know, this might be a complete programming nightmare. But it would > give us a very nice, clean, API. Well, "clean" looks different in my eyes. Having a special object 'above' the normal objects you'd use in the ET API sounds a bit fishy. But then, from the point of view of the API, PIs are kinda fishy, too. > I'd love to contribute patches, but I'm C-illiterate. Can LXML be ported over > into FORTRAN77? :-) Have you actually looked at the code? It's written in Pyrex - most of it is still looking rather Python-like, so if you know Python, you should at least be able to read it. Stefan

1 0

Re: [lxml-dev] Adding a Stylesheet PI when creating new XML documents
by Stefan Behnel 23 Mar '07

23 Mar '07

Hi Lee, Lee Brown wrote: > I think the prependPI() is slightly better, as it is reminiscent of the append() > method. I personally liked the "addToTree()" method better as it allows us to accompany it with a "copyToTree()" method that would not remove the PI from its current location. Maybe "prependToTree()" and "prependCopyToTree()" would be even more descriptive names. > Here's a silly thought: > > Couldn't we just save any PIs preceeding the root element as members of docinfo? > That's where the doctype, xml version, and other "pre-root" stuff lives. > Doesn't "tree.docinfo.stylesheet" or "tree.docinfo.PI['stylesheet']" make more > sense than the way we're doing it now? > > (And wouldn't it have been nice if I'd thought of this several months ago?) Not that silly at all, and not too late either, since we do not yet have an API for this (apart from the generic 'getprevious()' method). So, regarding your proposal: it will not quite work as nothing prevents an XML document from having 20 processing instructions of the same type. The meme above sort of suggests that there can be only one (note that we are talking about etree here, not objectify, which has a very different look-and-feel). Is there any obvious way that would allow us to handle the normal case of zero or one PIs per type as nicely as possible, while allowing us to deal with the possible case of having an arbitrary number of them? I mean, since PIs can be accessed with the normal API, the main problem we are facing is that we currently cannot append or prepend siblings - only children. And the API is pretty much geared towards child handling (because that's how things work in ET), which is usually enough - except for document level processing instructions that *are* siblings. I already said it was time to think a bit more about the docinfo property. I'm still not sure it is the best way of accessing this information. The main problem with docinfo is that it currently is read-only. If we start using it for adding processing instructions (i.e. for modifying trees), it would be hard to explain why you can't change or add a DOCTYPE declaration the same way (e.g. by throwing in a DTD object). So, starting work here implies a complete rewrite of the DocInfo class - and I would prefer having a good idea about the way it should look afterwards before wasting my time. As usual, any patches are welcome and I'm very much open for suggestions here. I just won't be the one fighting this through as I'm not too interested in getting this in. (it's not something that can't currently be done, just something that could need a good API to make it's usage a bit more efficient and a lot more obvious). Regards, Stefan

2 1

Re: [lxml-dev] Adding a Stylesheet PI when creating new XML documents
by Stefan Behnel 22 Mar '07

22 Mar '07

Hi! Lee Brown wrote: > Parsing an XSLT stylesheet from an xml-stylesheet PI is working just > fine, but now I'm trying to add an xml-stylesheet PI to a new XML tree > created from within lxml. I've looked back through the list traffic and > done some code introspection but I can't find anything. > > Did we ever decide upon / create an API for adding a stylesheet PI to a > new tree? No, not yet. :) I see two possibilities: add a "prependPI(pi)" method to ElementTree, or add a method "addToTree(tree)" method to ProcessingInstruction. We might then also need a way to get rid of them again - although, well, you could always move them to a different tree, which would obviously remove them from the current one. Ok, so any preferences? Stefan

2 1

[lxml-dev] Adding a Stylesheet PI when creating new XML documents
by Lee Brown 22 Mar '07

22 Mar '07

Greetings! Parsing an XSLT stylesheet from an xml-stylesheet PI is working just fine, but now I'm trying to add an xml-stylesheet PI to a new XML tree created from within lxml. I've looked back through the list traffic and done some code introspection but I can't find anything. Did we ever decide upon / create an API for adding a stylesheet PI to a new tree? Best Regards, Lee E. Brown Electronics Development Corporation 9055-F Guilford Road Columbia MD 21046 410-312-6678 lee.brown(a)elecdev.com www.elecdev.com

1 1

Re: [lxml-dev] Overriding whitespace normalization under XSLT
by Stefan Behnel 22 Mar '07

22 Mar '07

Nathan R. Yergler schrieb: > Thanks to everyone for their suggestions. Sounds like there's not > some secret knob to turn (or at least not one that the group knows > about). I did find that passing the result through tidy formats it > correctly, so that's the interim fix for me. The suggestion by Lee Brown could be implemented by canonicalising (c14n_write) the XML output, in case you want to drop a dependency here. Stefan > On 3/17/07, Lee Brown <leebrown(a)leebrown.org> wrote: >> Greetings! >> >> With regards to the snippet below, the "missing" space is valid XHTML but >> not valid HTML according specifications preceding HTML 4.01. >> Provided that it won't break your consumer's browser agents, setting the >> DOCTYPE to HTML 4.01 Strict or to XHTML 1.0 Strict and omitting the >> leading >> XML declaration should cause this to render correctly in most modern >> browsers. >> >> If you must support older browsers, try adding a closing </img> tag. >> Technically, that's invalid HTML as well, but most of the older browsers >> will parse it anyway. >> >> Best Regards, >> Lee E. Brown >> (leebrown(a)leebrown.org) >> >> -----Original Message----- >> From: lxml-dev-bounces(a)codespeak.net >> [mailto:lxml-dev-bounces@codespeak.net] >> On Behalf Of Stefan Behnel >> Sent: Saturday, March 17, 2007 1:33 AM >> To: Nathan R. Yergler >> Cc: lxml-dev(a)codespeak.net >> Subject: Re: [lxml-dev] Overriding whitespace normalization under XSLT >> >> Nathan R. Yergler wrote: >> > The part that's problematic is this line: >> > >> > <a rel="license" href="{$license-uri}"><img alt="Creative >> Commons >> > License" style="border-width:0" src="{$license-button}" /></a> >> > >> > Note that there is a space between the closing quote of the src >> > attribute on the image tag and the "/>" closing bracket. When we >> > process the transform, we consistently end up with >> > >> > <a rel="license" href="..."><img alt="Creative Commons >> License" >> > style="border-width:0" src="..."/></a> >> > >> > (note the space has been removed) >> >> That's perfectly well-formed XHTML. But rumour has it that some browsers >> can't handle that. It's just not old-style HTML-ish enough. >> >> Stefan >> _______________________________________________ >> lxml-dev mailing list >> lxml-dev(a)codespeak.net >> http://codespeak.net/mailman/listinfo/lxml-dev >> >>

2 1

Re: [lxml-dev] Custom Elements
by Stefan Behnel 21 Mar '07

21 Mar '07

Hi again, Stefan Behnel wrote: > Stephan Richter wrote: >> class DirectiveLookup(etree.CustomElementClassLookup): >> >> def lookup(self, type, doc, namespace, name): >> import pdb; pdb.set_trace() >> if name == 'foo1': >> return Directive1 >> elif name == 'foo2': >> return Directive2 >> return None >> >> However, in my case, the lookup also depends on the parent element in my case, >> because a tag with the same name can have different meanings in different >> contexts. (Don't ask, this is a specification requirement of RML. ;-) >> >> So here my question: How do I access the parent element? > > That's a tricky one. Correcting myself: wasn't that tricky after all. The trunk now has a new module called "lxml.pyclasslookup" that provides a new lookup mechanism: PythonElementClassLookup You can subclass it and implement the method "lookup(self, doc, element)" to find a suitable element class for "element". This last argument is an instance of a lightweight Element proxy that implements most of the API of the standard Element class of lxml.etree. The main differences are the lack of iteration support and the bigger things like XPath etc. Just take that object, traverse and analyse the tree as you see fit and return the element class that suits your needs. Have fun, Stefan

1 0

Re: [lxml-dev] Custom Elements
by Stefan Behnel 21 Mar '07

21 Mar '07

Stephan Richter wrote: > class DirectiveLookup(etree.CustomElementClassLookup): > > def lookup(self, type, doc, namespace, name): > import pdb; pdb.set_trace() > if name == 'foo1': > return Directive1 > elif name == 'foo2': > return Directive2 > return None > > However, in my case, the lookup also depends on the parent element in my case, > because a tag with the same name can have different meanings in different > contexts. (Don't ask, this is a specification requirement of RML. ;-) > > So here my question: How do I access the parent element? That's a tricky one. Providing the parent at the Python level would require its instantiation as Element, i.e. another lookup of the target class. So we end up with a recursive chicken-and-egg problem. Providing its tag name might be enough, but then, others might need to check its attributes or other parts of the tree, too - no end in sight. So, all I can suggest is to write a separate Pyrex module against etree's public C-API to provide a C-implemented lookup mechanism that works on the libxml2 tree. You can take a look at objectify's ObjectifyElementClassLookup implementation to see how this can be done. It's actually not difficult at all, but you'd have to write it in Pyrex. Another possible solution is to actually use the same class, but with a different configuration based on its parent. You can do the setup in Element._init(). Hope it helps, Stefan

1 0

[lxml-dev] Custom Elements
by Stephan Richter 20 Mar '07

20 Mar '07

Hi everyone, I did some initial trials with creating custom elements and the following script works as expected: from lxml import etree class Directive1(etree.ElementBase): foo = '1' class Directive2(etree.ElementBase): foo = '2' class DirectiveLookup(etree.CustomElementClassLookup): def lookup(self, type, doc, namespace, name): import pdb; pdb.set_trace() if name == 'foo1': return Directive1 elif name == 'foo2': return Directive2 return None parser = etree.XMLParser() parser.setElementClassLookup(DirectiveLookup()) elm = etree.fromstring( '<bar><foo1 /><foo2 /><foo /></bar>', parser) # Do inspection import pdb; pdb.set_trace() However, in my case, the lookup also depends on the parent element in my case, because a tag with the same name can have different meanings in different contexts. (Don't ask, this is a specification requirement of RML. ;-) So here my question: How do I access the parent element? Regards, Stephan -- Stephan Richter CBU Physics & Chemistry (B.S.) / Tufts Physics (Ph.D. student) Web2k - Web Software Design, Development and Training

1 0

[lxml-dev] Overriding whitespace normalization under XSLT
by Nathan R. Yergler 20 Mar '07

20 Mar '07

I'm not sure this is really the right list for this, but we're using lxml, and I'm hoping that someone will have the requisite expertise to enlighten me. Creative Commons has an XSLT transformation we use for generating license engine output; it's divided into a few templates, and one final template that assembles the pieces. The problem is with respect to the "img" tag in the human readable copy-n-paste output. Here's an excerpt from the transform: <xsl:template name="html"> <xsl:param name="license-uri"/> <xsl:param name="license-name"/> <xsl:param name="rdf"/> <xsl:variable name="license-button"> <xsl:choose> <xsl:when test="contains($license-uri,'GPL')">http://i.creativecommons.org/l<xsl:value-of select="substring-after($license-uri,'http://creativecommons.org/licenses')"/>88x62.png</xsl:when> <xsl:otherwise>http://i.creativecommons.org/l<xsl:value-of select="substring-after($license-uri,'http://creativecommons.org/licenses')"/>88x31.png</xsl:otherwise> </xsl:choose> </xsl:variable> <a rel="license" href="{$license-uri}"><img alt="Creative Commons License" style="border-width:0" src="{$license-button}" /></a> <xsl:call-template name="thiswork"> <xsl:with-param name="license_name" select="$license-name" /> <xsl:with-param name="license_url" select="$license-uri" /> </xsl:call-template> </xsl:template> The part that's problematic is this line: <a rel="license" href="{$license-uri}"><img alt="Creative Commons License" style="border-width:0" src="{$license-button}" /></a> Note that there is a space between the closing quote of the src attribute on the image tag and the "/>" closing bracket. When we process the transform, we consistently end up with <a rel="license" href="..."><img alt="Creative Commons License" style="border-width:0" src="..."/></a> (note the space has been removed) I've done some experiments and it seems any amount of whitespace (including carriage returns) is collapsed during processing. This may be valid markup (I'm not certain), but some consumers of our copy-and-paste markup barf when that space doesn't exist. I feel like I've reached the end of my XSLT expertise, so does anyone have any idea a) how to maintain that whitespace or b) whom to ask who might know? Thanks, and thanks for lxml, Nathan ----- Nathan R. Yergler Senior Software Engineer Creative Commons http://wiki.creativecommons.org/User:NathanYergler

3 4