Mailman 3 November 2009 - lxml - The Python XML Toolkit

[lxml-dev] Installing on Mac OS X 10.6.2: "lipo: can't figure out the architecture type of"
by Esben Bugge 25 Nov '09

25 Nov '09

I have tried to install lxml on Mac OS X 10.6.2 using both sudo python setup.py build --static-deps and sudo STATIC_DEPS=true easy_install lxml In both cases I get lipo: can't figure out the architecture type of: /var/tmp//ccdcpY3e.out make[2]: *** [SAX.lo] Error 1 make[1]: *** [all-recursive] Error 1 make: *** [all] Error 2 Exception: Command "make" returned code 2 What can I do to resolve this error?

2 3

[lxml-dev] elementtree as a hierarchical object tree
by cgnmm 24 Nov '09

24 Nov '09

Hi, I'm new of lxml and XML programming. I would to use etree as a hierarchical object tree: every node element should have a property associated to an object. >>> import lxml.etree as etree >>> element=etree.fromstring('<TAG>some text</TAG>') >>> element.text="a string" that's all ok, but: >>> element.text={'a':'real', 'python':'object'} TypeError: Argument must be string or unicode. OK, the property name is "text" so some text is expected! My question is: if write a custom element with a property called "object", may I store inside it a reference to an object?

2 1

[lxml-dev] Why can't I turn all xpath node types into string?
by Kevin Ar18 24 Nov '09

24 Nov '09

XML defines several node types that get returned by XPATH: http://www.w3.org/TR/xpath#data-model When I use the xpath feature in lxml and then try to read out the data, it won't work for certain node types. How can I get a string version of all node types? Examples follow. The following works: Domain/HTML: http://www.citiesxl.com/ XPATH: //div//a from lxml import etree self.xpath_list = self.tree.xpath(xpath) for entry in self.xpath_list: tmp = "" try: tmp = etree.tostring(entry) except: tmp = str(entry) print tmp In this example, xpath_list is of type: <class 'lxml.etree._Element'> The etree.tostring() function works for this type. The following will NOT work for etree.tostring() XPATH: //div//a/attribute::href It says it can't handle the type or something. In this case, xpath_list is of type: <class 'lxml.etree._ElementStringResult'> or something like that (I forget the actual type). The str() function will work on this one. The following will NOT work at all using either str() or etree.tostring() XPATH: //self::text() One of the types is: <type 'lxml.etree._ElementUnicodeResult'> and I get this error: UnicodeEncodeError: 'ascii' codec can't encode character u'\xa0' in position 38: ordinal not in range(128) Is there an lxml function that will convert all types to strings properly instead of trying to use this try/except hack to handle all the types? _________________________________________________________________ Bing brings you maps, menus, and reviews organized in one place. http://www.bing.com/search?q=restaurants&form=MFESRP&publ=WLHMTAG&crea=TEXT…

2 1

[lxml-dev] lxml 2.2.4 for Python 2.6
by srijit＠aim.com 24 Nov '09

24 Nov '09

Is there any reason why lxml-2.2.4-py2.6-win32.egg or lxml-2.2.4.win32-py2.6.exe is not available in http://pypi.python.org/pypi/lxml/2.2.4? Best regards, /Srijit

3 2

Re: [lxml-dev] lxml slower than ElementTree?
by Stefan Behnel 24 Nov '09

24 Nov '09

Hi, John Machin, 24.11.2009 14:00: > Thanks for your response, Stefan. You're welcome. > On 24/11/2009 7:48 PM, Stefan Behnel wrote: >> In general, ET can't compete with lxml.etree, whereas cET can, especially >> when you stay with code that supports both ET and lxml.etree. > > As it stands, the code tries to import xml.etree.cElementTree, then > cElementTree, then ElementTree. I plan to allow an override option, > where the caller can import any minimal-ET-subset-compliant module (e.g. > lxml) and pass it in. The mentioned speed comparison was using cET. >> - lxml parses much faster from file names than from open file(-like) >> objects, especially multi-threaded. ET handles them exactly the same. > > The MS Excel 2007 file format is a ZIP file containing compressed XML > documents. Hence one ends up parsing file-like objects. Sure, I expected something like that. The thing is just that parsing from file-like object from lxml means that it needs to walk all the way up the Python stack to look up and call the object's .read() method for each chunk of data, which then builds and returns a new Python string object. Parsing from a file path implies straight calls to C's file reading function, which works completely outside of the GIL. So there is a pretty huge performance penalty for file-like objects. This penalty applies to (c)ET as well, though, so that's not a reason for lxml to be slower than them. > For better or worse, ET comes with Python. Design goal was to avoid > requiring Nth party modules where appropriate I totally understand that design goal. Being in the stdlib is clearly a major advantage of ET. >> - ET creates the tree as Python objects once and for all, lxml creates >> Python proxies only at request. > > I acknowledge that ET may start swapping much sooner than lxml That's not what I was referring to, though. It's ok to call ET an elephant, but cET is actually *very* memory friendly. I meant to say that lxml.etree implies a performance penalty if you access many elements in the tree, where cET can just return their reference. The penalty is not large, but it's certainly worth being a bit more selective when walking the tree and searching elements. That's what XPath and getiterator(tagname) are great for. >> - Tree iteration using getiterator() in lxml.etree is much faster than in >> cET, and also much, much faster than using .find(). This strikes even >> more >> when searching specific tags, because fewer proxies have to be created. >> (c)ET doesn't show a difference here. > > A base class provides a controller method that uses > getiterator(tag=None) and a mapping from tags to methods. Each type of > document (about 5 types) has a subclass. I would expect that to be pretty fast on cET. For lxml.etree, however, it might still be faster to traverse the document independently for each interesting tag name - as long as the document fits into memory completely, and as long as the methods are independent from each other. The benefit certainly depends on the ratio of interesting versus ignored elements in the traversal, but given that lxml's tight traversal loop is easily an order of magnitude faster than even cET, being selective can really turn the vane here. > The area where heavy lifting is required is the worksheet document, > which contains cell elements as children of row elements. Max 2**20 rows > and max 2**14 cells per row. If iterparse is available, a specialised > controller method is used; it uses iterparse to iterate over row > elements (only "end" events), clearing each row as it is finished. This > should solve most of the memory problem for [c]ET. Extending this to > clear the root element (i.e. avoid leaving empty row elements lying > about) is a possibility. Too bad that a) lxml's iterparse() is slightly slower than the one in cET and b) clearing the root element doesn't work in lxml.etree I guess a) is where your initial comment mainly originated from. Regarding b), getting your hands at the root element in (c)ET would require you to also accept 'start' events, which usually results in such a huge performance drop for larger documents that it's almost always better to just leave the dead elements around instead. Also note that lxml.etree would allow you to drop them during the iteration by removing the preceding siblings of the current row element through their common parent element. So you can have the cake and eat it, too. :) Stefan

1 0

[lxml-dev] lxml slower than ElementTree?
by Chris Withers 24 Nov '09

24 Nov '09

Background: John maintains xlrd, the python package for reading Excel files, and is looking to support Microsoft's newer xml based format... John Machin wrote: <snip> > Note that it needs an ElementTree implementation (supplied with more > recent Pythons), and tries to find one in various places. Limited > testing with lxml gave identical results, but slightly slower, so you > could try that instead if you wanted to (would require fiddling with > imports in xlsxrd.py) That surprised me. Would anyone here be interested in taking a look at John's code to see what's tripping up lxml and causing it to be slower? cheers, Chris -- Simplistix - Content Management, Batch Processing & Python Consulting - http://www.simplistix.co.uk

2 1

[lxml-dev] lxml Win32 egg for Python 2.6
by Celvin 23 Nov '09

23 Nov '09

Hi, I just went to my local cheese shop to upgrade lxml for Python 2.6 but noticed that there are only lxml 2.2.4 / Python 2.6 packages for AMD64 - is this intentional or a mere oversight? Cheers, Celvin

2 1

[lxml-dev] smart_strings
by Nicolas Delaby 23 Nov '09

23 Nov '09

Hi, I would like to know if exists a way to reach an attribute name with an xpath result from _ElementStringResult instance. I see that we can reach is parent, but i need to know to which attribute this result belong, without parsing xpath expression with regex or something similar. root = etree.XML('<root><a a="aaa">TEXT</a></root>') result = root.xpath('/root/a/attribute::a')[0] assert result.is_attribute result.getparent().???? Regards, Nicolas -- Nicolas Delaby Nexedi: Consulting and Development of Libre / Open Source Software http://www.nexedi.com/

2 2

[lxml-dev] etree not printing pretty :(
by Martin Seiler 19 Nov '09

19 Nov '09

Hi all, I am rather new to python and lxml and I wonder about the output of my tree. I add an Element, which contains some childs to a tree. When I write the output everything is in pretty print, but the elements I appended. They show up in one line. Here is a snippet of my code: > eggs = et.fromstring(xmltransformed) > for each in eggs: > if each.tag == "someElement": > > each.append(anotherElement) > > else: > pass > > > et.ElementTree(element=eggs).write(xmlout, pretty_print=True) When I open the file in firefox its formated all right, but when I view the code 'anotherElement' with all its childs is just one line... what am I doing wrong here? python: 2.6 python-lxml: 2.1.5-1ubuntu2 OS: Ubuntu 9.04 Thanks for you help! Cheers, martin

2 1

[lxml-dev] Python 2.6 Win-32 binary?
by Mike Naglee 18 Nov '09

18 Nov '09

Hey folks, Am I just missing something or is there no Python 2.6 Win32 binary for lxml? Thanks, and have an interesting day! Mike

1 0