Mailman 3 August 2009 - lxml - The Python XML Toolkit

[lxml-dev] a solution to file upload
by Mary Lei 24 Aug '09

24 Aug '09

8/20/09 I posted the following: I have to support form post. Since I am using lxml for form requests, how do I set up the file upload part ? I now have a solution for python 2.6: 1. I adopted Fabien SEISEN's urllib2_file.py. When my code detects that the form request is a file upload request, instead of using submit_form, my code switches to calling urllib2.urlopen http handler in urllib2_file.py and thus puts out the same multipart form/data format for file upload as done via the web browser. I did have to include an error handler for urllib2_file.py: # Special case for StringIO try: <--- catch if fd.__module__ in ("StringIO", "cStringIO"): <-- has no __module__ name = k fd.seek(0, 2) # EOF file_size = fd.tell() fd.seek(0) # START else: file_size = os.fstat(fd.fileno())[stat.ST_SIZE] except AttributeError: <-- catch exception here file_size = os.fstat(fd.fileno())[stat.ST_SIZE] -- Mary Lei Software Testing IPAC-NExScl Rm: KS-233 MS: 220-6 Phone: 395-1998

2 2

[lxml-dev] getname() method on 'smart' attribute string values
by Jamie Norrish 23 Aug '09

23 Aug '09

I have a situation where I want to find any attributes in a document that contain a certain value, and then change that value and record the fact that the new value is the result of such a change. In order to track these changes, I am populating a dictionary keyed by the parent element, with each value being another dictionary keyed by attribute name with a value of the list of parts of the attribute that have been changed. So: get_attrs = etree.XPath('//@*[contains(concat(" ", ., " "), concat(" #", $old_id, " "))]') for attribute in get_attrs(element, old_id=some_string): element = attribute.getparent() And then I discover that in fact I have to use element.attrib.items() and search through for which attribute matched. It would be much easier if the attribute 'smart' string result from the XPath evaluation had a method which would specify *which* attribute of the parent it was the value of. Or am I missing a better, existing way of doing this? Note that while there is in fact a finite set of attribute names I need to check, it's a potentially expanding set, and I'd rather not have to touch the code when that expansion happens. Jamie

2 4

Re: [lxml-dev] problems trying to install lxml 2.2 on Mac OS X
by Stefan Behnel 21 Aug '09

21 Aug '09

Hi, Pascal Oberndoerfer wrote: > Sorry for sending this large file to you directly, but I thought > it might be of use. And I don't have a clue with regard to pypi... > > Built on i386, running 10.4.11, with MacPython.org distro, using: > "python setup.py bdist_egg --static-deps". > > Untested, as I did this after "python setup.py install --static-deps". Thanks a lot! I put it here: http://codespeak.net/lxml/lxml-2.2.2-py2.5-macosx-10.3-i386.egg so that others can test it before I move it over to PyPI. Stefan

4 3

[lxml-dev] how to do file upload via submit_form
by Mary Lei 20 Aug '09

20 Aug '09

I have to support form post. Since I am using lxml for form requests, how do I set up the file upload part ? Thanks. -- Mary Lei Software Testing IPAC-NExScl Rm: KS-233 MS: 220-6 Phone: 395-1998

1 0

[lxml-dev] Validating against an empty element
by Bryan Hughes 14 Aug '09

14 Aug '09

I have the following empty SubElement in my XML file: <State/> This value stores one of the 50 US states. When I attempt to validate it against my XSD, I receive the following error: Element 'State': [facet 'pattern'] The value '' is not accepted by the pattern Here is the code in my XSD. Can someone shed some light on why this might be failing? The "State" type is listed as a non-required field (minOccurs=0), so I'm stumped why this error is popping up. <xsd:simpleType name="State"> <xsd:restriction base="xsd:string"> <xsd:pattern value="AK|AL|AR|AZ|CA|CO|CT|DC|DE|FL|GA|HI|IA|ID|IL|IN|KS|KY|LA|MA|MD|ME|MI|MN|MO|MS|MT|NC|ND|NE|NH|NJ|NM|NV|NY|OH|OK|OR|PA|RI|SC|SD|TN|TX|UT|VA|VT|WA|WI|WV|WY"/> </xsd:restriction> </xsd:simpleType> FWIW, I've ever tried adding "\s" to my regex pattern, but it still raises the same exception.

2 1

[lxml-dev] lxml bug
by commissar wu 13 Aug '09

13 Aug '09

Hi:everyone,lxml is very good, I like it .But I recently encountered a little trouble.I use lxml to parse the contents of the url( http://www.dtzww.cn/files/article/fulltext/23/23208.html),the lxml is been blocking,and don't rasie exception. The CPU utilization rate is 100%. My environment is lxml-2.2.2. ubutnu-8.04-amd64-server python-2.5.2 My code is fellow: import lxml.html as htmltool import urlib url = "http://www.dtzww.cn/files/article/fulltext/23/23208.html" f = urllib.urlopen(url) data = f.read() doc = htmltool.document_fromstring(data) ## <--- Block this Looking forward to your reply. commissar your friend

2 3

[lxml-dev] Behaviour of the push parser in recover mode on encountering errors
by Laurence Rowe 13 Aug '09

13 Aug '09

Over at http://bugzilla.gnome.org/show_bug.cgi?id=569131 I reported what I thought was a bug in HTMLParser but on closer inspection appears to be an incorrect assumption on my part (and that of lxml) when dealing with errors returned by the push parser interface. With the libxml2 bindings, I am able to parse invalid html using the push parser: >>> import libxml2 >>> options = libxml2.HTML_PARSE_RECOVER | libxml2.HTML_PARSE_NONET >>> p = libxml2.htmlCreatePushParser(None, "", 0, "test") >>> p.ctxtUseOptions(options) 0 >>> bad1 = '''<p><pre></pre></p>\n''' >>> p.htmlParseChunk(bad1, len(bad1), 0) test:1: HTML parser error : Unexpected end tag : p <p><pre></pre></p> ^ 76 >>> good = '''<div>foo</div>\n''' >>> p.htmlParseChunk(good, len(good), 0) 76 >>> p.htmlParseChunk("", 0, 1) 76 >>> print p.doc().serialize() <!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.0 Transitional//EN" "http://www.w3.org/TR/REC-html40/loose.dtd"> <html><body><p></p><pre></pre> <div>foo</div></body></html> But with lxml, the parser is reset on encountering an error: >>> from lxml.etree import HTMLParser, dump >>> p = HTMLParser(recover=True) >>> bad1 = '''<p><pre></pre></p>\n''' >>> p.feed(bad1) Traceback (most recent call last): File "<console>", line 1, in ? File "parser.pxi", line 1093, in lxml.etree._FeedParser.feed (src/lxml/lxml.etree.c:61114) File "parser.pxi", line 534, in lxml.etree._ParserContext._handleParseResult (src/lxml/lxml.etree.c:56605) File "parser.pxi", line 628, in lxml.etree._handleParseResult (src/lxml/lxml.etree.c:57504) File "parser.pxi", line 568, in lxml.etree._raiseParseError (src/lxml/lxml.etree.c:56902) XMLSyntaxError: Unexpected end tag : p, line 1, column 19 >>> good = '''<div>foo</div>\n''' >>> p.feed(good) >>> elem = p.close() And previous state is lost: >>> dump(elem) <html> <body> <div>foo</div> </body> </html> In fact, I'm unable to retrieve any state from the parser unless it is reset: >>> p.feed(bad1) Traceback (most recent call last): File "<console>", line 1, in ? File "parser.pxi", line 1093, in lxml.etree._FeedParser.feed (src/lxml/lxml.etree.c:61114) File "parser.pxi", line 534, in lxml.etree._ParserContext._handleParseResult (src/lxml/lxml.etree.c:56605) File "parser.pxi", line 628, in lxml.etree._handleParseResult (src/lxml/lxml.etree.c:57504) File "parser.pxi", line 568, in lxml.etree._raiseParseError (src/lxml/lxml.etree.c:56902) XMLSyntaxError: Unexpected end tag : p, line 1, column 19 >>> p.close() Traceback (most recent call last): File "<console>", line 1, in ? File "parser.pxi", line 1113, in lxml.etree._FeedParser.close (src/lxml/lxml.etree.c:61239) XMLSyntaxError: no element found So in my view, the behaviour here is not helpful. When a parser is created with recover=True it should not raise errors, so allowing incremental parsing of invalid html. Laurence

2 1

[lxml-dev] Inserting Element before Tails
by Nick Lang 12 Aug '09

12 Aug '09

Hello, I am new to lxml. I see from the website that it says it supports Mixed-Content. Though I have question with this mixed content support. Say for example I have the following xml: <root> <a>this is <bold>BOLD</bold></a> </root> If I were to get a list of the children of <a> it would _only_ contain the element <bold> correct? The .tail of <a> would be: "this is", right? So the question I have is this: If I wanted to add an element before <bold> and after <a> I would do so like this (or similarly): <a>.insert(0, etree.Element("new") The result of this insert would leave me with: <a>This is <new /><bold>BOLD</bold></a> Is it possible to insert before the tail of <a>? IE, I would want something like this. before: <a>this is <bold>BOLD</bold></a> after: <a><new />this is <bold>BOLD</bold></a> I hope this is clear. Thanks Nick

2 1

[lxml-dev] default namespace and xpath evaluation
by Emanuele D'Arrigo 12 Aug '09

12 Aug '09

Hi everybody, I can't seem to find a more compact way to do this: nsDict = {"default":anElement.nsmap[None]} aChildElement = anElement.xpath("/default:elem", namespaces=nsDict )[0] Specifically, I would have thought that if the element <elem> is in the default namespace the simple string: aChildElement = anElement.xpath("/elem")[0] would be sufficient, but the element is not found - probably appropriately. However, passing the element's nsmap such as in: aChildElement = anElement.xpath("/elem" ,namespaces=anElement.nsmap)[0] results in the following error message: Traceback (most recent call last): File "<stdin>", line 1, in <module> File "lxml.etree.pyx", line 1314, in lxml.etree._Element.xpath (src/lxml/lxml.etree.c:38871) File "xpath.pxi", line 245, in lxml.etree.XPathElementEvaluator.__init__ (src/lxml/lxml.etree.c:106924) File "xpath.pxi", line 117, in lxml.etree._XPathEvaluatorBase.__init__ (src/lxml/lxml.etree.c:105514) File "xpath.pxi", line 55, in lxml.etree._XPathContext.__init__ (src/lxml/lxml.etree.c:104808) File "extensions.pxi", line 77, in lxml.etree._BaseContext.__init__ (src/lxml/lxml.etree.c:96771) TypeError: empty namespace prefix is not supported in XPath Am I missing a nicer way of doing this? Manu

2 4

[lxml-dev] libxml2 crash on 64bit Ubuntu and solution
by Christian Heimes 12 Aug '09

12 Aug '09

Dear lxml users, My blog post may safe some of you several hours of debugging. If you compile libxml2 yourself on a 64bit Ubuntu system you are going to run into the same problem. http://lipyrary.blogspot.com/2009/08/libxml2-crash-on-64bit-ubuntu.html HTH Christian

1 0