Mailman 3 December 2010 - lxml - The Python XML Toolkit

[lxml-dev] lxml has its page on launchpad
by Stefan Behnel 11 Apr '23

11 Apr '23

Hi all, I added the lxml project to launchpad, the Ubuntu Bug-Tracker. It also has a FAQ engine and a couple of other goodies. https://launchpad.net/lxml It's easy to sign up for launchpad, BTW, no 90%-footnotes-contract. Have fun, Stefan

9 9

[lxml-dev] Checking whether a node is a comment/element
by Geoffrey Sneddon 10 Apr '23

10 Apr '23

Hi, What's the best way to check whether a given node is a comment or an element? For the former, I'm currently using isinstance(node, etree._Comment), which is rather obviously sub-optimal. -- Geoffrey Sneddon <http://gsnedders.com/>

6 6

[lxml-dev] Reparenting a node
by Lawrence Oluyede 30 Jan '23

30 Jan '23

I have a doc A and a doc B, I'd like to put a node extracted from A in the document B but I always get a ValueError: ValueError: Element is not a child of this node. I didn't find any "setparent" in the API. How can I do this? -- Lawrence, oluyede.org - neropercaso.it "It is difficult to get a man to understand something when his salary depends on not understanding it" - Upton Sinclair

3 2

[lxml-dev] lxml 2.0.5 released
by Stefan Behnel 11 Jan '23

11 Jan '23

Hi all, lxml 2.0.5 is on PyPI. This is a bug-fix-only release of the stable 2.0 series. Have fun, Stefan 2.0.5 (2008-05-01) Bugs fixed * Resolving to a filename in custom resolvers didn't work. * lxml did not honour libxslt's second error state "STOPPED", which let some XSLT errors pass silently. * Memory leak in Schematron with libxml2 >= 2.6.31.

3 4

[lxml-dev] Building LXML Trunk
by Sidnei da Silva 31 Aug '22

31 Aug '22

Hi, I've tried to build lxml from trunk today, on Win32. Got the following error: src\lxml\etree.c(880) : error C2059: syntax error : ')' src\lxml\etree.c(881) : error C2059: syntax error : ')' src\lxml\etree.c(882) : error C2059: syntax error : ')' src\lxml\etree.c(883) : error C2059: syntax error : ')' Any clue? Smells like a Pyrex issue? -- Sidnei da Silva Enfold Systems http://enfoldsystems.com Fax +1 832 201 8856 Office +1 713 942 2377 Ext 214

4 4

[lxml-dev] confusing xpath performance characteristics
by jholg＠gmx.de 24 Aug '12

24 Aug '12

Hi, I ran into some performance characteristics of lxml/libxml2 xpath that I find rather confusing: I try to find the @type attribute of a certain element in an XML Schema (which contains lots of complexType definitions with lots of elements in them; unfortunately I can't post the schema): >>> timeit.Timer(stmt="""xpath(schema)""", setup="""from lxml import etree, objectify; schema=etree.parse('NDM.xsd').getroot(); xpath = etree.XPath('//xs:element[@name="equity"]/@type', namespaces={'xs': 'http://www.w3.org/2001/XMLSchema'})""").repeat(number=10) [0.095885038375854492, 0.096823930740356445, 0.096174955368041992] So I think I'm being smart and give a little more path information - reckoning that this should *improve* performance: >>> timeit.Timer(stmt="""xpath(schema)""", setup="""from lxml import etree, objectify; schema=etree.parse('NDM.xsd').getroot(); xpath = etree.XPath('/xs:schema//xs:element[@name="equity"]/@type', namespaces={'xs': 'http://www.w3.org/2001/XMLSchema'})""").repeat(number=10) [0.1770780086517334, 0.1775970458984375, 0.17748594284057617] Hm. Performance degrades slightly. I'm adding even more of the path to where my desired elements live in the schema: >>> timeit.Timer(stmt="""xpath(schema)""", setup="""from lxml import etree, objectify; schema=etree.parse('xsd/NDM.xsd').getroot(); xpath = etree.XPath('/xs:schema/xs:complexType//xs:element[@name="equity"]/@type', namespaces={'xs': 'http://www.w3.org/2001/XMLSchema'})""").repeat(number=10) [103.79744100570679, 103.83671712875366, 103.61817717552185] What??? >>> timeit.Timer(stmt="""xpath(schema)""", setup="""from lxml import etree, objectify; schema=etree.parse('/ae/data/pydev/hjoukl/NDM/SVN_CO/TRUNK/ndm/reference/xsd/NDM.xsd').getroot(); xpath = etree.XPath('/xs:schema/xs:complexType/*/xs:element[@name="equity"]/@type', namespaces={'xs': 'http://www.w3.org/2001/XMLSchema'})""").repeat(number=10) [0.044407129287719727, 0.044126987457275391, 0.044229030609130859] >>> Ok, this version's better than my naive approach, which seems logical to me. But why would '/xs:schema/xs:complexType//xs:element[@name="equity"]/@type' perform drastically slower than '/xs:schema/xs:complexType//xs:element[@name="equity"]/@type' ? libxml2 problem? Running the same xpaths in Oxygen I don't notice performance differences (can't profile this). Holger -- GRATIS für alle GMX-Mitglieder: Die maxdome Movie-FLAT! Jetzt freischalten unter http://portal.gmx.net/de/go/maxdome01

2 5

[lxml-dev] DLL issue with win32 static builds of lxml
by Scott Smith 04 Apr '11

04 Apr '11

Hi I'm looking at upgrading our python installations from 2.6(.2) to 2.7 and upgrading the packages we use to the latest versions. We use python and the mod_wsgi module in Apache 2.2 for our application server and easy_install to install egg packages. I'm currently testing on a Windows 7 machine. The current egg version of lxml that we're using (before upgrading) is: lxml-2.2.4-py2.6-win32.egg In trying to upgrade things, I've found that all the later builds of lxml cause a DLL problem through apache. For example, all of these builds (from pipy) cause the problem: lxml-2.2.8-py2.6-win32.egg lxml-2.2.8-py2.7-win32.egg lxml-2.2beta1-py2.6-win32.egg lxml-2.3beta1-py2.6-win32.egg lxml-2.3beta1-py2.7-win32.egg (Note that there are no static win32 builds available for 2.2.5-2.2.7, or I can't find them) The error thrown in the Apache logs is: mod_wsgi (pid=4812): Target WSGI script 'C:/wsgi/handler.py' cannot be loaded as Python module. mod_wsgi (pid=4812): Exception occurred processing WSGI script 'C:/wsgi/handler.py'. Traceback (most recent call last): File "C:/wsgi/handler.py", line 2, in <module> from lxml import etree ImportError: DLL load failed: The specified module could not be found. The same file works in Python 2.6 if lxml 2.2.4 is installed, but fails for any other higher version of lxml (either in python 2.6 or 2.7). I've been able to trace the problem down to the builds not being able to find the msvcr90.dll file properly. I've tried modifying path variables and other changes, but can't figure out what the problem is. A couple of other things to note: - Using just "import lxml" doesn't cause the error, just "from lxml import etree" - The problem only occurs through Apache. If I write a script that references etree and call it through the command line, the problem doesn't occur. Does anyone know what I need to do? What changed after the 2.2.4 lxml builds? Thanks for any help you can give. Scott Smith

5 5

Re: [lxml-dev] lxml-dev Digest, Vol 75, Issue 1
by Dimitrios Pritsos 06 Jan '11

06 Jan '11

Hello guys, I am sorry that I am sending this as a response but there is two issues I d like to point out: 1. There is a memory leakage using lxml.html.parse (or etree) while you do that constantly in a loop. In particular creating etrees in a loop does let the trees there and is not deleting the properly when you reuse the same python variable to store the resutls. For now I haven't tryed to resolve it because module re (regular expression) is just fine for URL extraction, however I would prefer the use of XPath for extracting a variate of links more easily in Coding point of view. Plus I think that the overhead of Tree Building is not so much (I dont know for sure thought). 2. Speaking of XPath for url extraction, I think that lxml.html has some issues in url extraction (this is what I think reading the Code of this module). And the question is why not to use the XPath for making the code twice smaller and twice neater (I cleaner and well formed - I hope my vocabulary is correct), maybe faster too. Best Regards, Dimitrios On 12/02/2010 11:35 AM, lxml-dev-request(a)codespeak.net wrote: > Send lxml-dev mailing list submissions to > lxml-dev(a)codespeak.net > > To subscribe or unsubscribe via the World Wide Web, visit > http://codespeak.net/mailman/listinfo/lxml-dev > or, via email, send a message with subject or body 'help' to > lxml-dev-request(a)codespeak.net > > You can reach the person managing the list at > lxml-dev-owner(a)codespeak.net > > When replying, please edit your Subject line so it is more specific > than "Re: Contents of lxml-dev digest..." > > > Today's Topics: > > 1. Re: Compile failure (felix) > 2. Re: Compile failure (jholg(a)gmx.de) > 3. Re: SyntaxErrors with Python 3 (Stefan Behnel) > 4. Re: Schema validation - no file position (Stefan Behnel) > 5. Re: Schema validation - no file position (Krzysztof Jakubczyk) > 6. Re: Schema validation - no file position (Stefan Behnel) > 7. Re: read .xlsx spreadsheets with lxml ? (Chris Withers) > 8. How to register extension functions for XSL transformations > (Shaung) > 9. Re: How to register extension functions for XSL > transformations (jholg(a)gmx.de) > > > ---------------------------------------------------------------------- > > Message: 1 > Date: Tue, 23 Nov 2010 11:14:19 +0100 > From: felix<crucialfelix(a)gmail.com> > Subject: Re: [lxml-dev] Compile failure > To: Stefan Behnel<stefan_ml(a)behnel.de> > Cc: lxml-dev(a)codespeak.net > Message-ID: > <AANLkTinBaMukcy=pCVHjmmmAq=kiPN=nGE_Gd27_z5sT(a)mail.gmail.com> > Content-Type: text/plain; charset="utf-8" > > On Sat, Oct 30, 2010 at 10:49 AM, Stefan Behnel<stefan_ml(a)behnel.de> wrote: > >> felix, 26.10.2010 15:28: >> >> According to this: >>> http://codespeak.net/lxml/build.html >>> >>> we should avoid installing Cython >>> >>> but using easy_install to build fails saying the cython generated file is >>> missing >>> >> I doubt that it's failing because of that. However, you didn't provide the >> output of the build, so I can't guess what happened that actually made the >> build fail. > > sorry, that output had scrolled off by the time I realized I should submit a > report. I have another server so fortunately I can fail there and show you. > > crucial@crucial-systems:~/working/lxml$ python setup.py build > /home/crucial/working/lxml/versioninfo.py:53: UserWarning: unrecognized > .svn/entries format; skipping /home/crucial/working/lxml/ > warn("unrecognized .svn/entries format; skipping "+base) > Building lxml version 2.3.beta1. > *NOTE: Trying to build without Cython, pre-generated 'src/lxml/lxml.etree.c' > needs to be available.* > Using build configuration of libxslt 1.1.26 > Building against libxml2/libxslt in the following directory: /usr/lib > running build > running build_py > running build_ext > building 'lxml.etree' extension > gcc -pthread -fno-strict-aliasing -DNDEBUG -g -fwrapv -O2 -Wall > -Wstrict-prototypes -fPIC -I/usr/local/include/libxml2 > -I/usr/include/python2.6 -c src/lxml/lxml.etree.c -o > build/temp.linux-x86_64-2.6/src/lxml/lxml.etree.o -w > gcc: src/lxml/lxml.etree.c: No such file or directory > gcc: no input files > error: command 'gcc' failed with exit status 1 > > > >> The latest build instructions for the SVN trunk are in the SVN trunk as >> "doc/build.txt", or *(not always completely up-to-date)* here: >> > exactly > > > > >> *but then I succeeded with the old sudo easy_install lxml* >>> >>> because now I have Cython >>> >> Again, I doubt that this is the reason. >> > sudo easy_install lxml > failed before > > after installing Cython it says it uses Cython (not Trying to build without > Cython) and it worked. > > nothing else having changed I thought it was a reasonable guess that it > worked because it used Cython because Cython is installed. > > * > * > >> Stefan >> >

3 9

[lxml-dev] no error position or line number when walidating against XMLSchema using iterparse
by Grzegorz Ślusarek 30 Dec '10

30 Dec '10

Hello I'm trying to use iterparse to validate big xml document using XmlSchema. Generally it works, but when walidate malformed xml, the exception received doesn't contain any information about file position. Specific Exception attributes like position is (0,0), offset is None and linenumber is just None. In that case is very hard say what's wrong with xml, especially when is'a big xml file (300 MB). Is it a bug or this is correct behaviour ? I found that this question was allready asked on this mailing list, but there wasn't any response. Btw. this situation happen on lxml 2.2.8 and python2.6 (I must say that I try to used lxml 2.3 beta and get the same case). Below I attached some piece of code that I use to do validation schema = XMLSchema(file=schemapath) context = iterparse(file(filepath,'r'), events=('end',), schema=schema) try: for event, elem in context: elem.clear() while elem.getprevious() is not None: if elem.getparent() is not None: del elem.getparent()[0] except XMLSyntaxError, e: ....... Regards Grzegorz Ślusarek

1 0

[lxml-dev] xpath: retrieve attribute names
by jholg＠gmx.de 29 Dec '10

29 Dec '10

Hi, is there a convenient way to retrieve element attribute values and *names* using xpath? I haven't yet found an XPath (1.0) expression and believe this is impossible in XPath 1.0. It is of course trivial to get at the attribute values: >>> print etree.__version__ 2.2.6 >>> print etree.LIBXML_VERSION (2, 6, 32) >>> print etree.LIBXSLT_VERSION (1, 1, 23) >>> >>> root = objectify.Element('root') >>> root.a = 3 >>> root.a.set('foo', 'bar') >>> root.a.set('doo', 'woop') >>> print etree.tostring(root, pretty_print=True) <root xmlns:py="http://codespeak.net/lxml/objectify/pytype" xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" py:pytype="TREE"> <a py:pytype="int" foo="bar" doo="woop">3</a> </root> >>> root.xpath('//@*') ['TREE', 'int', 'bar', 'woop'] >>> All I can come up with for getting at attribute names are ugly beasts like >>> [ (elt.xpath('name(@*[$i])', i=i+1), elt.xpath('@*[$i]', i=i+1)[0]) for i in range(elt.xpath('count(@*)')) ] [('py:pytype', 'int'), ('foo', 'bar'), ('doo', 'woop')] I also thought about using exslt trickery, but now I'm a bit confused about its possible usage in XPath: >>> root.xpath("dyn:map(//*, name())") 'root' >>> root.xpath('str:split("12, 13", ",")') ',' Seems like this isn't possible? Anyhow: How about adding a getname() method to XPath smart string results, analogous to the getparent() functionality? This would * return the (ns-qualified) attribute name for attribute results * return the (ns-qualified) element tag for element results It seems to me that having getparent() isn't of much use for attribute results if you want to get at the attribute's name. Holger -- Neu: GMX De-Mail - Einfach wie E-Mail, sicher wie ein Brief! Jetzt De-Mail-Adresse reservieren: http://portal.gmx.net/de/go/demail

2 2