Mailman 3 December 2008 - lxml - The Python XML Toolkit

[lxml-dev] lxml 2.0.11, lxml 2.1.4 and 2.2beta1 released - a note on future releases
by Stefan Behnel 18 Dec '08

18 Dec '08

Hi, lxml 2.0.11, 2.1.4 and 2.2beta1 are on PyPI. The first two are pure bug fix releases for a crash bug that can occur when using a single XPath evaluator concurrently from multiple threads. lxml 2.2beta1 adds a couple of further fixes to this. Changelog below. In its current state, I consider lxml stable and mature enough not to require separate long-living release branches. The current release is the end of the 2.0 series. Unless we find major reasons for users not to switch after the release of 2.2 final, 2.1 will soon go out of maintenance. Stefan 2.2beta1 (2008-12-12) Features added * Allow lxml.html.diff.htmldiff to accept Element objects, not just HTML strings. Bugs fixed * Crash when using an XPath evaluator in multiple threads. * Fixed missing whitespace before Link:... in lxml.html.diff. Other changes * Export lxml.html.parse. 2.1.4 (2008-12-12) Bugs fixed * Crash when using an XPath evaluator in multiple threads. 2.0.11 (2008-12-12) Bugs fixed * Crash when using an XPath evaluator in multiple threads.

2 1

[lxml-dev] xslt coverage
by Dirk Rothe 16 Dec '08

16 Dec '08

Hi, are there any possibilities to get something like a line coverage from a xsl-transformation. That would be pretty useful, if integrated in a test-code-coverage run. --dirk

2 2

[lxml-dev] Strange ValueError
by pythonalex＠email.it 15 Dec '08

15 Dec '08

Hello everyone. I'm Alessio Esposito and collaborate with the Institute of Cybernetics of the CNR in Pozzuoli (NA) - Italy, I have the latest version of lxml (lxml 2.1.4). I need to have something like this: ... <octapyelement xsi:schemaLocation="http://octapycms.remuna.org/schema.xsd"> <octapy:first> something </octapy:first> <octapy:second> something </octapy:second> ... ... </octapyelement> ... And this is the python code: ... e_octapy = SubElement(rootelement, 'octapyelement') e_octapy.set('{%s}schemaLocation' % NS_XSI, 'http://octapycms.remuna.org/schema.xsd') for sub_elem in struct: e = SubElement( e_octapy, sub_elem ) e.text = 'something' ... The variable sub_elem contain strings octapy:first, octapy:second etc. The problem is that the latest version of lxml does not accept that sub_elem contains ':' . This is a part of the terminal output: Module lxml.etree, line 2342, in lxml.etree.SubElement (src/lxml/lxml.etree.c:%u) Module lxml.etree, line 160, in lxml.etree._makeSubElement (src/lxml/lxml.etree.c:%u) Module lxml.etree, line 1311, in lxml.etree._tagValidOrRaise (src/lxml/lxml.etree.c:%u) ValueError: Invalid tag name u'octapy:first' The previous version of lxml (lxml 2.1) whit the same code gave no problems. How can I fix this? Kind regards, Alessio. -- Caselle da 1GB, trasmetti allegati fino a 3GB e in piu' IMAP, POP3 e SMTP autenticato? GRATIS solo con Email.it: http://www.email.it/f Sponsor: CheBanca! La prima banca che ti dà gli interessi in anticipo. Fino al 4,70% sul Conto Deposito, zero spese e interessi subito. Aprilo! Clicca qui: http://adv.email.it/cgi-bin/foclick.cgi?mid=8435&d=20081215

3 2

[lxml-dev] pyquery
by Olivier Lauzanne 14 Dec '08

14 Dec '08

Hello, First thanks for lxml it's great. But I miss an interface on top of it. Something like jquery<http://jquery.com>or hpricot <http://code.whytheluckystiff.net/hpricot/>. Is there any work in progress to go toward something like that in python ? Missing a jquery like API in python, I started reproducing the jquery API in python by using lxml and released it a few days ago : pyquery<http://pypi.python.org/pypi/pyquery> If anyone is interested in contributing the code is clean and small. If anyone is working toward the same idea but in a slightly different direction I would be very interested. Olivier

3 8

[lxml-dev] buildlibxml problems
by Gael Pasgrimaud 12 Dec '08

12 Dec '08

Hi, I have a problem to install lxml with Cython and --static-deps. The problem is that you dont keep the original os.environ in place during subprocess so gcc is not found during ./configure of libxml2. I've found a workaround and you can find my patch attached (buildlibxml.py.patch) Regards, -- Gael

2 1

[lxml-dev] Yet (another) XML-verifier utility, Xpectador
by I Putu Alberto Lee 12 Dec '08

12 Dec '08

Hi, I just implemented an XML-verifier utility for the purpose of my work. It's written in python, and uses lxml library. With pleasure I'd like to share with you; hopefully you will find it useful. Basically it allows you to state your expectations over "what should appear" and "what should not appear" in the XML-document that you'd like to validate, using the same format as that XML-document-to-be-verified. I make the source code available here: http://www.box.net/shared/5hs5ynq75k (a .py file) -- just three functions that make up the recursive checkings. More info is available at: http://jananuraga.blogspot.com/2008/12/xpectador-10.html Feel free to use and modify it under creative-common attribution generic 2.5 license: http://creativecommons.org/licenses/by/2.5/ . At least please put a link to my blog in your version / port of xpectador. Saludos, Raka

2 1

[lxml-dev] [Fwd: [Ian Bicking: a blog] Comment: "lxml: an underappreciated web scraping library"]
by Ian Bicking 12 Dec '08

12 Dec '08

Any ideas on this guy's installation issues? -------- Original Message ------- New comment on your post #85 "lxml: an underappreciated web scraping library" Author : john aman Thanks to all. @Ian, @Tres: The first real problem? My stupidity. python2.5-dev not installed. I should have landed here much sooner but for Ubuntu makes installing packages so damn easy. 2nd problem - running at 64 bit is a bit like warp speed to this MSDOS 2.0 old timer. Years of DOS and windoze development may also have corrupted my neural network. Now I get to this: (env)john@ibex:~$ STATIC_DEPS=true easy_install 'lxml>=2.2alpha1' ... Downloading libxslt into libs/libxslt-1.1.24.tar.gz Unpacking libxslt-1.1.24.tar.gz into build/tmp Running "./configure --without-python --disable-dependency-tracking --disable-shared --prefix=/tmp/easy_install-Am4yKQ/lxml-2.2alpha1/build/tmp/libxml2" in build/tmp/libxml2-2.7.2 checking build system type... x86_64-unknown-linux-gnu checking host system type... x86_64-unknown-linux-gnu ... lxml-2.2alpha1/build/tmp/libxml2/lib/pkgconfig" /usr/bin/install -c -m 644 'libxslt.pc' '/tmp/easy_install-6H0Ks9/lxml-2.2alpha1/build/tmp/libxml2/lib/pkgconfig/libxslt.pc' /usr/bin/install -c -m 644 'libexslt.pc' '/tmp/easy_install-6H0Ks9/lxml-2.2alpha1/build/tmp/libxml2/lib/pkgconfig/libexslt.pc' make[2]: Leaving directory `/tmp/easy_install-6H0Ks9/lxml-2.2alpha1/build/tmp/libxslt-1.1.24' make[1]: Leaving directory `/tmp/easy_install-6H0Ks9/lxml-2.2alpha1/build/tmp/libxslt-1.1.24' NOTE: Trying to build without Cython, pre-generated 'src/lxml/lxml.etree.c' needs to be available. Using build configuration of libxml2 2.7.2 and libxslt 1.1.24 Building against libxml2/libxslt in the following directory: /tmp/easy_install-6H0Ks9/lxml-2.2alpha1/build/tmp/libxml2/lib ************************ /usr/bin/ld: /tmp/easy_install-6H0Ks9/lxml-2.2alpha1/build/tmp/libxml2/lib/libexslt.a(exslt.o): relocation R_X86_64_32 against `a local symbol' can not be used when making a shared object; recompile with -fPIC ************************ /tmp/easy_install-6H0Ks9/lxml-2.2alpha1/build/tmp/libxml2/lib/libexslt.a: could not read symbols: Bad value collect2: ld returned 1 exit status error: Setup script exited with error: command 'gcc' failed with exit status 1 where error is marked *********** It seems the linker is failing to link exslt.o into libexslt.a: (exslt.o): relocation R_X86_64_32 against `a local symbol' can not be used when making a shared object; recompile with -fPIC Oh, did I mention I'm new to 64 bit OS? Perhaps someone can point out how to do all the steps the wget/tar/.configure/make way. Too old to try to figure out something so easy_install._ Not knocking easy_install - this is a first time failure for me with that program.

2 1

Re: [lxml-dev] Yet (another) XML-verifier utility, Xpectador
by I Putu Alberto Lee 10 Dec '08

10 Dec '08

Hi, Sorry I made a mistake, the correct link to the py file is http://www.box.net/shared/u55s51p9hf Best regards, Raka --- On Wed, 12/10/08, I Putu Alberto Lee <putualbertolee(a)yahoo.com> wrote: From: I Putu Alberto Lee <putualbertolee(a)yahoo.com> Subject: Yet (another) XML-verifier utility, Xpectador To: lxml-dev(a)codespeak.net Date: Wednesday, December 10, 2008, 2:39 PM Hi, I just implemented an XML-verifier utility for the purpose of my work. It's written in python, and uses lxml library. With pleasure I'd like to share with you; hopefully you will find it useful. Basically it allows you to state your expectations over "what should appear" and "what should not appear" in the XML-document that you'd like to validate, using the same format as that XML-document-to-be-verified. I make the source code available here: http://www.box.net/shared/5hs5ynq75k (a .py file) -- just three functions that make up the recursive checkings. More info is available at: http://jananuraga.blogspot.com/2008/12/xpectador-10.html Feel free to use and modify it under creative-common attribution generic 2.5 license: http://creativecommons.org/licenses/by/2.5/ . At least please put a link to my blog in your version / port of xpectador. Saludos, Raka

1 0

[lxml-dev] html encoding
by Daniel Jirku 09 Dec '08

09 Dec '08

hi... My problem is i suppose well known, but i couldnt find any soultion through my searches... I have a regular html link with ? and an &. When i print the variable in pyhton, it looks fine... (like: http://www.somelink.com/site.html?param1=test&param2=hello), BUT when i add it to my root xml element with: adId1 = etree.SubElement(tagAd, "originalAdUrl") adId1.text = adUrl and then later write the xml to a file with this: toStringValue = etree.tostring(xmlTagRoot, encoding="utf-8", method="xml", xml_declaration=True, pretty_print=True) ... the tag has as its value the link with an & instead of & !! How can i use the correct signs for persistant storage in a xml file...? thank you very much.. -- Psssst! Schon vom neuen GMX MultiMessenger gehört? Der kann`s mit allen: http://www.gmx.net/de/go/multimessenger

3 2

[lxml-dev] extracting .text strings systematically in unicode
by Jean Daniel 09 Dec '08

09 Dec '08

Hello, I am working on a small XML to SQL application. Input attribute values and text fields usually are unicode but not always. They are fed into the attributes of an object which only accepts unicode input and raise an exception if the data is an 'str' instead (said object is a storm persisted class). My problem seems to be that lxml extracts text element either as an 'str' or a 'unicode', depending on the text element, as shown on the code snippets : from lxml.etree import XML type( XML('<tag>element</tag>').text ) <type 'str'> type( XML('<tag>élément</tag>').text ) <type 'unicode'> So far, it seems that my only choice is to 'cast' every extraction of the xml doc to unicode, which is cumbersome and does not seem necessary. Example : self.name = unicode( element.get('name') ) for child in element: setattr(self, child.tag, unicode( child.text ) ) Is there a switch in the lxml module to make the strings of the xml document appears predictably as unicode even is the string can be represented a simple 'str'? Thank you,

4 5