Mailman 3 September 2010 - lxml - The Python XML Toolkit

[lxml-dev] Hashing objectify.StringElement
by Saúl Ibarra Corretgé 21 Sep '10

21 Sep '10

Hi all, It's my first question on the list and I tried to find the answer but couldn't so if my question has indeed been answered before just point me there please :-) I'm updating some API which used to use etree.Element objects to objectify because I need to pickle this items and from what I've read so far, this seems like the way to go. At some point in the code elements are used as dictionary keys and this used to work before, but now I get a TypeError telling me that 'lxml.objectify.StringElement' is unhashable. However, 'lxml.objectify.ObjectifiedElement' class is hashable. Also, I found that the __hash__ function seems to be implemented, but for some reason calling hash() or trying to use StringElement as a dictionary key will give this error. Is this a bug? A feature? Can I get around this somehow? Thanks in advance and kind regards, PS: I'm using version 2.2.6-1 on Debian squeeze. -- Saúl Ibarra Corretgé AG Projects

1 1

[lxml-dev] Concurrent Programming and lxml but recipe for Newbies like me - any solution?
by Dimitrios Pritsos 20 Sep '10

20 Sep '10

Hello to all, I am working for a while with lxml in a Concurrent Programming Environment. I have tested to pass an HtmlElement tree and failed in both case of MultiProcessing and MultiThreading setup. I use "threading" and "multiprocessing" native modules. I found in FAQ that at least in MultiThreading case it should work properly when passing a tree or element between Threads. However it is not working. More specifically it seems that is passing the HtmlElement Part of python but not the part of C because (or something like that anyway): writing this : isinstance(xhtml_tree, lxml.html.HtmlElement) => returns True in both Threads 1.the one that Got the Tree from a Queue.Queue() and 2. the one that puts the Puts the Tree to the Queue() but few lines later writing: xhtml.xpath("//text()") => does nothing, while writing : xpath(xhtml) where xpath = lxml.etree.XPath("//text()") => raises an error that the input is not an HtmlElement Even if Python says that it is ( isinstance() ). So, I guess there is a problem with python and C parts of the etree objects. Is it? however returning the tree from an Evenlet's greenThread it is working fine ( I guess that is because greenThread is nothing more than a Coroutine) Do you have any Idea why lxml is not working in the concurrent setup (even in Threading) while it should? However I would prefer to have it working on MultiProcessing env Best Regards, Dimitrios

1 0

[lxml-dev] Access to ElementTree for XML schema
by Dave Kuhlman 14 Sep '10

14 Sep '10

I'm looking for a way to get access to an etree._ElementTree that represents an XML schema document in which the xsd:include and xsd:import elements have been recursively expanded. When I create an instance of etree.XMLSchema, libxml2 expands the underlying C tree for the schema. Am I right about that? If so, is there a way for me to get an etree._ElementTree that wraps that underlying C tree? Or, perhaps to have a way to create an etree._ElementTree from the XMLSchema object? If that document is not available, I suppose that I am asking for a new feature that enables us to retrieve the processed and expanded etree Document from an etree.XMLSchema object. Or, is there already some other way to get an XML schema document tree in which the include and import elements have been (recursively) expanded? The reason I'm asking for this -- I process XML schema documents, and I think we should encourage other Python hackers to do so, too. This (new) feature would enable lxml to support that. I'm trying to implement this capability myself in Python using lxml, but my implementation still has bugs and I'm sure that libxml2 does it better than I can. Thanks for help. - Dave -- Dave Kuhlman http://www.rexx.com/~dkuhlman

2 2

[lxml-dev] lxml can't work with html5lib
by flya flya 14 Sep '10

14 Sep '10

I installed lxml 2.3beta1 and html5lib 0.90 on python2.6.6, when try code from here http://codespeak.net/lxml/html5parser.html >>> from lxml.html import tostring, html5parser >>> tostring(html5parser.fromstring("<table><td>foo")) get the follow error message: Traceback (most recent call last): File "<pyshell#9>", line 1, in <module> tostring(html5parser.fromstring("<table><td>foo")) File "C:\Python26\lib\site-packages\lxml\html\html5parser.py", line 137, in fromstring guess_charset=guess_charset) File "C:\Python26\lib\site-packages\lxml\html\html5parser.py", line 54, in document_fromstring return parser.parse(html, useChardet=guess_charset).getroot() File "build\bdist.win32\egg\html5lib\html5parser.py", line 211, in parse parseMeta=parseMeta, useChardet=useChardet) File "build\bdist.win32\egg\html5lib\html5parser.py", line 111, in _parse self.mainLoop() File "build\bdist.win32\egg\html5lib\html5parser.py", line 179, in mainLoop self.phase.processStartTag(token) File "build\bdist.win32\egg\html5lib\html5parser.py", line 578, in processStartTag self.parser.phase.processStartTag(token) File "build\bdist.win32\egg\html5lib\html5parser.py", line 616, in processStartTag self.insertHtmlElement() File "build\bdist.win32\egg\html5lib\html5parser.py", line 595, in insertHtmlElement self.tree.insertRoot(impliedTagToken("html", "StartTag")) File "C:\Python26\lib\site-packages\lxml\html\_html5builder.py", line 91, in insertRoot root_element = self.elementClass(name) File "build\bdist.win32\egg\html5lib\treebuilders\etree.py", line 31, in __init__ namespace)) File "C:\Python26\lib\site-packages\lxml\html\__init__.py", line 1563, in Element v = html_parser.makeelement(*args, **kw) File "parser.pxi", line 861, in lxml.etree._BaseParser.makeelement (src/lxml/lxml.etree.c:74517) File "apihelpers.pxi", line 120, in lxml.etree._makeElement (src/lxml/lxml.etree.c:12246) File "apihelpers.pxi", line 1464, in lxml.etree._getNsTag (src/lxml/lxml.etree.c:23600) File "apihelpers.pxi", line 1358, in lxml.etree._utf8 (src/lxml/lxml.etree.c:22519) TypeError: Argument must be string or unicode.

1 0

[lxml-dev] Finding the media-type and method of an XSLT
by Laurence Rowe 13 Sep '10

13 Sep '10

Thanks for your earlier help. Now that I'm suing str(result) to output the result tree, I'd like to set an appropriate Content-Type header. The <xsl:output> element allows one to specify the output method (xml or html) and media-type of the result tree. From libxslt one can interrogate these values on the transform with: xsltStylesheetPtr transform; transform->mediaType; transform->encoding; transform->method; In lxml I can find the encoding from the resultTree with result.docinfo.encoding, but how do I find the method and mediaType? Laurence

1 0

[lxml-dev] lxml 2.3 beta 1 released
by Stefan Behnel 13 Sep '10

13 Sep '10

Hi, I'm happy to announce the first beta release of lxml 2.3. This is a bug-fix release for the upcoming 2.3 series that mostly follows the fixes in lxml 2.2.8. Please consider testing this version if you currently use 2.2.x, which will continue to receive only important bug fixes. Note that the source distribution is available on PyPI, so your default lxml installation process may choose to use it. If you do not want that, choose an explicit version, e.g. "easy_install lxml==2.2.8". Sidnei, Pascal, please provide binary builds for this release, preferably using libxml2 2.7.7 (or 2.7.3 alternatively) and libxslt 1.1.26. Based on the current state of affairs, I'm expecting to release lxml 2.3 final in mid October, so there's still enough time to find bugs. :) This release was built using Cython 0.13 final. Have fun, Stefan 2.3beta1 (2010-09-06) Bugs fixed * Crash in newer libxml2 versions when moving elements between documents that had attributes on replaced XInclude nodes. * XMLID() function was missing the optional parser and base_url parameters. * Searching for wildcard tags in iterparse() was broken in Py3. * lxml.html.open_in_browser() didn't work in Python 3 due to the use of os.tempnam. It now takes an optional 'encoding' parameter.

4 3

[lxml-dev] install problems in Windows
by Terry Brown 12 Sep '10

12 Sep '10

Hi, thanks for the wonderful product, use it all the time. Given the inability of Python's etree to find a parent from an element (am I missing something?), I recently used in lxml in simple program, not thinking people would try and run it on Windows. But they did. c:\python26\python\scripts\easy_install lxml==2.2.4 seemed to work fine for python 2.6. We got nothing working for python 2.7 and 3.1. Without installing a C compiler in Windows... is there an easy way to get lxml in windows in 2.7/3.1 currently? Is it just a matter of waiting for the right binary installer to be made? Current plan is to rewrite the program to not use lxml, it's trivial enough that that's no big deal, but a shame if it's unnecessary work. Thanks, Terry

3 4

[lxml-dev] xsl:output method and encoding ignored
by Laurence Rowe 07 Sep '10

07 Sep '10

XSLT enables you to set output method (xml or html) and encoding using the xsl:output element. However, there seems to be no way to use this information from lxml, the default settings from the write method or the tostring function seem to take precedence. For example with the following XSL: <?xml version="1.0" encoding="UTF-8"?> <xsl:stylesheet version="1.0" xmlns:xsl="http://www.w3.org/1999/XSL/Transform"> <xsl:output method="html" omit-xml-declaration="yes" media-type="text/html" encoding="utf-8"/> <xsl:template match="/"> <xsl:copy-of select="."/> </xsl:template> </xsl:stylesheet> >>> from lxml import etree >>> import sys >>> identity = etree.XSLT(etree.parse('identity.xsl')) >>> identity(etree.HTML('<br>')).write(sys.stdout) <html><body><br/></body></html> Whereas with xsltproc: $ echo "<br>" | xsltproc --html identity.xsl - <html><body><br></body></html> Does anyone have any suggestions on how to deal with this? Laurence

2 1

[lxml-dev] Code hangs when calling etree.XMLSchema
by Bryan Hughes 04 Sep '10

04 Sep '10

When I pass in a valid XSD file into this line of code, the application hangs: logging.debug("start") xmlschema_ = etree.XMLSchema(xsd_file) logging.debug("finish") I've wrapped that piece of code with debug logging to confirm that it hangs. In other words, my logs will print "start", but do not print "finish". Unfortunately, no exceptions are thrown -- it simply does nothing once that line of code executes. Any thoughts on what may be happening?

2 1

[lxml-dev] lxml 2.2.8 released
by Stefan Behnel 02 Sep '10

02 Sep '10

Hi, lxml 2.2.8 is up on PyPI. It is a minor bugfix release for the stable 2.2 series. Have fun, Stefan 2.2.8 (2010-09-02) Bugs fixed * Crash in newer libxml2 versions when moving elements between documents that had attributes on replaced XInclude nodes. * Import fix for urljoin in Python 3.1+.

1 0