Mailman 3 October 2009 - lxml - The Python XML Toolkit

[lxml-dev] xpath check, selective xslt
by Emanuele D'Arrigo Oct. 28, 2009

Oct. 28, 2009

Hi everybody, a couple of questions: 1) is it possible to check if an element is selected by a particular xpath expression -without- checking if the element is part of the whole node-set returned by the expression evaluation? 2) let's assume we have an xslt-transformed ElementTree and that we add an untransformed branch to it. Is it possible to apply the same xslt transformation selectively, only to the added branch but not to the rest of the tree? Of course this wouldn't help in those situation where transformed(tree+branch) != transformed(tree) + transformed(branch). But excluding those cases, is there a way to do it? Thanks for your help! Manu

2 12

[lxml-dev] Comments before or after the root element
by Bob Brandt Oct. 28, 2009

Oct. 28, 2009

First let me say I love lxml! I have noticed one problem though... (Although I am unsure whether the problem is with lxml or PEBCAK) I need to insert comments both before and after the root element. For what I can see this is not only perfectly legal but specifically allowed in the RFCs (although for the life of me I can find the relevant sections again) I can not figure out how to perform this. And from my testing it appears that even if I import (either tostring or parse) ignores a document which has comments before and after, it just ignores them... For now I am running a "workaround" with just outputs the comments, then the xml, then the comments, but it would be much better if lxml could do it since there could be problems with the XML Declaration. Thanks Bob -- The problem with socialism is that you eventually run out of other people's money. - Margaret Thatcher

2 2

[lxml-dev] etree._Element.items(): Really "arbitrary order" or rather "in document order"?
by Praktikant3 - SAG Oct. 28, 2009

Oct. 28, 2009

Hi, I have designed this Loop element that determines its loop counter variable from the first attribute. Example: <xm:Loop i="0x16..0x21" format="0x%02X" j="i+1"> <something base="{i}" incremented="{j}"/> </xm:Loop> I implemented this loop a year ago, finding the loop counter attribute using the following code: # 'el' is the etree._Element <xm:Loop/> from above loop_counter = None format = None variables = {} for name, value in el.attrib.iteritems(): if name == "format": format = value continue if loop_counter is None: # the first attribute that is not 'format' loop_counter = name variables[name] = value I looked at the lxml implementation and it comes down to the _collectAttributes() function, defined in src/lxml/apihelpers.pxi. At that point I had to give up, I found it to be quicker to write this email than to find my way around libxml2 docs and code. I wanted to determine whether attributes parsed by libxml2 will remain in document order, regardless of what lxml's API documents (lxml.etree._Element.items() -> "arbitrary order"). Because, the code above worked without a hitch so far, always delivering attributes in document order. Is there an lxml or libxml2 version that will *not* keep attributes in document order? Why is it documented as being in "arbitrary order"? Do you want to be API-compatible for future changes that might break the order? Is libxml2 silent about this? What does its parser do? It was just observation that led me to believe document order is maintained, but I'd like to have the "proof" behind that observation. I don't know where to continue. Thanks, Felix Rabe

2 2

[lxml-dev] Handling processing instructions
by Emanuele D'Arrigo Oct. 27, 2009

Oct. 27, 2009

A couple of questions on processing instructions: 1) short of iterating from the root element using repeatedly getprevious(), is there a way to obtain the processing instructions that are listed before the root element? 2) is it possible to delete a processing instruction from a tree/element? Manu

1 0

[lxml-dev] etree._Element.items(): Really "arbitrary order" or rather "in document order"? (try #2 w/ formatting)
by Praktikant3 - SAG Oct. 27, 2009

Oct. 27, 2009

Hi, I have designed this Loop element that determines its loop counter variable from the first attribute. Example: <xm:Loop i="0x16..0x21" format="0x%02X" j="i+1"> <something base="{i}" incremented="{j}"/> </xm:Loop> I implemented this loop a year ago, finding the loop counter attribute using the following code: # 'el' is the etree._Element <xm:Loop/> from above loop_counter = None format = None variables = {} for name, value in el.attrib.iteritems(): if name == "format": format = value continue if loop_counter is None: # the first attribute that is not 'format' loop_counter = name variables[name] = value I looked at the lxml implementation and it comes down to the _collectAttributes() function, defined in src/lxml/apihelpers.pxi. At that point I had to give up, I found it to be quicker to write this email than to find my way around libxml2 docs and code. I wanted to determine whether attributes parsed by libxml2 will remain in document order, regardless of what lxml's API documents (lxml.etree._Element.items() -> "arbitrary order"). Because, the code above worked without a hitch so far, always delivering attributes in document order. Is there an lxml or libxml2 version that will *not* keep attributes in document order? Why is it documented as being in "arbitrary order"? Do you want to be API-compatible for future changes that might break the order? Is libxml2 silent about this? What does its parser do? It was just observation that led me to believe document order is maintained, but I'd like to have the "proof" behind that observation. I don't know where to continue. Thanks, Felix Rabe

1 0

[lxml-dev] Availibilty of nsmap
by Nicolas Delaby Oct. 23, 2009

Oct. 23, 2009

Hi all, I would like to know if the sample of code bellow is an expected behaviour of nsmap handling. >>> from lxml import etree >>> string = '<root><node><sub:node xmlns:sub="any_uri"/></node></root>' >>> doc = etree.fromstring(string) >>> doc.find('node/{any_uri}node').nsmap {'sub': 'any_uri'} >>> doc.nsmap {} >>> Why nsmap at root Element level doesn't return nsmap of is children ? Regards, Nicolas -- Nicolas Delaby Nexedi: Consulting and Development of Libre / Open Source Software http://www.nexedi.com/

2 2

Re: [lxml-dev] CSSSelector namespace issues
by Stefan Behnel Oct. 21, 2009

Oct. 21, 2009

Hi, please keep the list involved in your replies. David Baird wrote: > On Sun, Oct 11, 2009 at 10:30 AM, Stefan Behnel wrote: >> Thanks a lot for the patch! I'll look through it ASAP. > > I think my test cases would have fit better into the test_css.txt > framework rather than into test_css.py. Yep. Could you move them there? > Too bad Python's batteries-included standard library doesn't include > *parsing tools* :-P (well, besides regex). True. pyparsing would be nice, although there are certainly others that deserve a similar standing. >>> Also, while I was digging around, I found some inconsistent rules for >>> converting to lower case in cssselect.py: >> I'm not sure this is intended. It might make sense for HTML, but I actually >> doubt that lower casing tag names is a good idea in the general case. > > For the situation of HTML (case-insensitive) versus XML > (case-sensitive), I think a good way to handle this is allow the user > to choose if they want lower-case conversion or not. I added some of > the hooks for doing this (especially by adding the "make_lower" > property in Element), but I didn't do much with these hooks yet. > Unfortunately, just lower-casing the selector doesn't do too much good > all by itself - the whole HTML tree would need to be lower cased as > well, and I guess the user would just have to be diligent to do that. >>> import lxml.html as h >>> [el.tag for el in ... h.fromstring("<HTML><body><BR></BODY></HTML>").iter()] ['html', 'body', 'br'] So the HTML parser actually does it for you. Still, there is no reason why users can't just write their CSS expressions in lower case when searching HTML. Lower casing the expression internally breaks non-HTML traversal. I think that's just one more thing to clean up in lxml 2.3. Stefan

2 1

[lxml-dev] Objectify python lists
by Matteo Malosio Oct. 20, 2009

Oct. 20, 2009

I'd like to interpret XML Schema lists (i.e. "/<position>1.0 2.0 3.0 4.0 5.0 6.0</position>/" defined with something like "/<list itemType="double" />/" ) as python lists (a list made up of 6 doubles). I can't find any reference in the manual about python list supporting. I'm able only to use a sequence of positions (one per each number) but to me it seems too verbose. XMLschema /list/ would be better / Thank you! /

2 1

[lxml-dev] How to delete mutibple attributes (parsed from HTML)
by Mateusz Korniak Oct. 16, 2009

Oct. 16, 2009

I am parsing[1] simple HTML having two times defined <html xmlns= > I wish I could delete both xmlns definitions but: del root_elem.attrib["xmlns"] deletes only second one for both setups I used[2] and later there are no more "xmlns" ... :/ Any way I could do it ? Thanks in advance, regards ! [1]: # -*- coding: utf-8 -* import lxml.html import lxml.etree from lxml import etree print "lxml.etree: ", etree.LXML_VERSION print "libxml used: ", etree.LIBXML_VERSION print "libxml compiled: ", etree.LIBXML_COMPILED_VERSION print "libxslt used: ", etree.LIBXSLT_VERSION print "libxslt compiled: ", etree.LIBXSLT_COMPILED_VERSION content = """<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd"> <html xmlns="http://www.w3.org/1999/xhtml" xmlns="http://www.w3.org/1999/xhtml"> Foo </html>""" parser = lxml.html.HTMLParser(remove_comments=True,remove_pis=True) # Removing comments as later xml output with them is broken root_elem = lxml.html.document_fromstring(str(content),parser=parser) print "DEBUG: root_elem: %r/%s" % (root_elem,root_elem,) content_xml = lxml.etree.tostring(root_elem,pretty_print=True,with_tail=False,method="xml") # This produces non-parsable XML print "DEBUG: content_xml: %r" % (content_xml,) ## lxml.etree.fromstring(content_xml) # lxml.etree.XMLSyntaxError: Attribute xmlns redefined, line 1, column 18 print "DEBUG: root_elem.attrib: %r" % (root_elem.attrib,) del root_elem.attrib["xmlns"] print "DEBUG: root_elem.attrib after first delete: %r" % (root_elem.attrib,) content_xml = lxml.etree.tostring(root_elem,pretty_print=True,with_tail=False,method="xml") # This produces non-parsable XML print "DEBUG: content_xml: %r" % (content_xml,) lxml.etree.fromstring(content_xml) # This parses, but still has <html xmlns="http://www.w3.org/1999/xhtml"> [2]: lxml.etree: (2, 2, 0, 0) libxml used: (2, 7, 4) libxml compiled: (2, 7, 3) libxslt used: (1, 1, 24) libxslt compiled: (1, 1, 24) and: lxml.etree: (2, 1, 5, 0) libxml used: (2, 7, 2) libxml compiled: (2, 7, 2) libxslt used: (1, 1, 24) libxslt compiled: (1, 1, 24) -- Mateusz Korniak

1 0

[lxml-dev] Parsing xml ignoring xml namesapces ?
by Mateusz Korniak Oct. 16, 2009

Oct. 16, 2009

Hi ! Is it possible to parse XML completly ignoring xml ns declarations ? Code [1] provies element names like: {http://www.w3.org/1999/xhtml}html and I would prefer just html . Thanks in advance , regards ! [1]: # -*- coding: utf-8 -* import StringIO import lxml.html import lxml.etree content_xml = '<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="cs" lang="cs" >\n <body>\n <p> Foo </p>\n </body>\n</html>\n' parser = lxml.etree.XMLParser(ns_clean=True) tree = lxml.etree.parse(StringIO.StringIO(content_xml), parser) # root_elem = lxml.etree.fromstring(content_xml) root_elem = tree.getroot() for elem in root_elem.iter(): print "elem: %r/%s" % (elem,elem,) -- Mateusz Korniak

2 1