lxml - The Python XML Toolkit December 2009

lxml@python.org

21 participants
30 discussions

[lxml-dev] Checking whether a node is a comment/element
by Geoffrey Sneddon 11 Jan '23

11 Jan '23

Hi, What's the best way to check whether a given node is a comment or an element? For the former, I'm currently using isinstance(node, etree._Comment), which is rather obviously sub-optimal. -- Geoffrey Sneddon <http://gsnedders.com/>

4 4

[lxml-dev] lxml 2.0.5 released
by Stefan Behnel 11 Jan '23

11 Jan '23

Hi all, lxml 2.0.5 is on PyPI. This is a bug-fix-only release of the stable 2.0 series. Have fun, Stefan 2.0.5 (2008-05-01) Bugs fixed * Resolving to a filename in custom resolvers didn't work. * lxml did not honour libxslt's second error state "STOPPED", which let some XSLT errors pass silently. * Memory leak in Schematron with libxml2 >= 2.6.31.

3 4

[lxml-dev] lxml has its page on launchpad
by Stefan Behnel 21 Sep '22

21 Sep '22

Hi all, I added the lxml project to launchpad, the Ubuntu Bug-Tracker. It also has a FAQ engine and a couple of other goodies. https://launchpad.net/lxml It's easy to sign up for launchpad, BTW, no 90%-footnotes-contract. Have fun, Stefan

3 2

[lxml-dev] Building LXML Trunk
by Sidnei da Silva 31 Aug '22

31 Aug '22

Hi, I've tried to build lxml from trunk today, on Win32. Got the following error: src\lxml\etree.c(880) : error C2059: syntax error : ')' src\lxml\etree.c(881) : error C2059: syntax error : ')' src\lxml\etree.c(882) : error C2059: syntax error : ')' src\lxml\etree.c(883) : error C2059: syntax error : ')' Any clue? Smells like a Pyrex issue? -- Sidnei da Silva Enfold Systems http://enfoldsystems.com Fax +1 832 201 8856 Office +1 713 942 2377 Ext 214

4 4

[lxml-dev] confusing xpath performance characteristics
by jholg＠gmx.de 24 Aug '12

24 Aug '12

Hi, I ran into some performance characteristics of lxml/libxml2 xpath that I find rather confusing: I try to find the @type attribute of a certain element in an XML Schema (which contains lots of complexType definitions with lots of elements in them; unfortunately I can't post the schema): >>> timeit.Timer(stmt="""xpath(schema)""", setup="""from lxml import etree, objectify; schema=etree.parse('NDM.xsd').getroot(); xpath = etree.XPath('//xs:element[@name="equity"]/@type', namespaces={'xs': 'http://www.w3.org/2001/XMLSchema'})""").repeat(number=10) [0.095885038375854492, 0.096823930740356445, 0.096174955368041992] So I think I'm being smart and give a little more path information - reckoning that this should *improve* performance: >>> timeit.Timer(stmt="""xpath(schema)""", setup="""from lxml import etree, objectify; schema=etree.parse('NDM.xsd').getroot(); xpath = etree.XPath('/xs:schema//xs:element[@name="equity"]/@type', namespaces={'xs': 'http://www.w3.org/2001/XMLSchema'})""").repeat(number=10) [0.1770780086517334, 0.1775970458984375, 0.17748594284057617] Hm. Performance degrades slightly. I'm adding even more of the path to where my desired elements live in the schema: >>> timeit.Timer(stmt="""xpath(schema)""", setup="""from lxml import etree, objectify; schema=etree.parse('xsd/NDM.xsd').getroot(); xpath = etree.XPath('/xs:schema/xs:complexType//xs:element[@name="equity"]/@type', namespaces={'xs': 'http://www.w3.org/2001/XMLSchema'})""").repeat(number=10) [103.79744100570679, 103.83671712875366, 103.61817717552185] What??? >>> timeit.Timer(stmt="""xpath(schema)""", setup="""from lxml import etree, objectify; schema=etree.parse('/ae/data/pydev/hjoukl/NDM/SVN_CO/TRUNK/ndm/reference/xsd/NDM.xsd').getroot(); xpath = etree.XPath('/xs:schema/xs:complexType/*/xs:element[@name="equity"]/@type', namespaces={'xs': 'http://www.w3.org/2001/XMLSchema'})""").repeat(number=10) [0.044407129287719727, 0.044126987457275391, 0.044229030609130859] >>> Ok, this version's better than my naive approach, which seems logical to me. But why would '/xs:schema/xs:complexType//xs:element[@name="equity"]/@type' perform drastically slower than '/xs:schema/xs:complexType//xs:element[@name="equity"]/@type' ? libxml2 problem? Running the same xpaths in Oxygen I don't notice performance differences (can't profile this). Holger -- GRATIS für alle GMX-Mitglieder: Die maxdome Movie-FLAT! Jetzt freischalten unter http://portal.gmx.net/de/go/maxdome01

2 5

[lxml-dev] Evaluate XSLT directives inside custom XSLT extension elements
by Marat Dakota 25 Jan '10

25 Jan '10

Hi, I have a problem I couldn't find the way to solve. Sadly if it's not possible. That's the code from XSLT extensions tutorial: <xsl:template match="*"> <my:python-extension> <some-content /> </my:python-extension> </xsl:template> Everything's working cool, but I need a bit more: <xsl:template match="*"> <my:python-extension> <something><xsl:attribute name="test"><xsl:value-of select="111" /></xsl:attribute></something> </my:python-extension> </xsl:template> Let's assume that's my execute function's signature: execute(self, context, self_node, input_node, output_parent) When it's started by XSLT processor, self_node contains <something> tag with <xsl:attribute> and <xsl:value-of> tags inside it. What should I do to ask XSLT processor to evaluate it? I need to make self_node containing <something test="111" />. And even more. What if my XSLT file looks like: <xsl:template match="*"> <my:python-extension> <something> <xsl:attribute name="test"><xsl:value-of select="111" /></xsl:attribute> <my:python-extension> <xsl:attribute name="test2"><xsl:value-of select="222" /></xsl:attribute> </my:python-extension> </something> </my:python-extension> </xsl:template> I need to have execute function called for deepest <my:python-extension> first, I need to evaluate <xsl:attribute> inside it to have <my:python-extension test2="222">... And to process the rest <my:python-extension> having the result of processed <my:python-extension test2="222"> and all XSLT instructions inside. Is there a way to do it? Or maybe there is some other way to get such possibilities? Thanks. -- Marat

2 17

[lxml-dev] Instructions to build on Windows 64-bit?
by Sridhar Ratnakumar 12 Jan '10

12 Jan '10

I noticed that the lxml PyPI page provides 64-bit Windows installers [http://pypi.python.org/pypi/lxml/2.2.4 ; lxml-*amd64.exe]. I assume they are statically linked with the libxml/xslt libraries. In the interest of providing 64-bit binaries in PyPM [pypm.activestate.com], may I know how these binaries are built? I tried buildlibxml.py which fails at several steps; and the compiled libraries provided at ftp.zlatkovic.com are 32-bit only. -srid

2 2

[lxml-dev] Looking for performance tips for soupparser
by Yannick Gingras 10 Jan '10

10 Jan '10

Hi, first of all, I have to say that I really like soupparser. Thanks a lot for it. I use it a lot data mining on a somewhat large document collection that I often revisit to try new ideas. Soupparser is fast but I put a lot of strain on it so I was looking for ways to speed things up. My first idea was to use beaker to cache the root Element object of every document to disk. Unfortunately, Element instances are not pickleable so I have to look for something else. Would any of you have some tips to share on speeding things up with soupparser? How hard would it be to make elements conform to the pickling protocol? -- Yannick Gingras http://ygingras.net http://confoo.ca -- track coordinator http://montrealpython.org -- lead organizer

2 2

[lxml-dev] Is there a 'Generic' XML-to-Relational Database Program?
by Dattatreya Mellacheruvu 31 Dec '09

31 Dec '09

Hi All, 1. Is there a python (UTILITY) program that converts 'any' XML to Relational Database program (which uses, say, lxml)? So, the utility program should not only parse the xml, but also understand the most general 'hierarchical' structure of the elements, create tables (say in sqlite) accordingly and insert the data. I have seen many relational DB to xml programs, but I haven't come across one that does the opposite. 2. Is there a UTILITY (program) whtat diplays the most general hierarchical structure of the elements as a tree (or as an expandable/collapsable list). 3. Is there a program which helps me to filter elements (in an xml file)? Supposing we were able to see the results of 2 (above), we then should be able ask for something like -- "discard all elements which do not have xyz and create a new file." Is there a pre-built UTILITY for doing such a thing? If nothing pre-build is available, I am gonna build them now. But I would love NOT to reinvent the wheel. Guru's on the forum, please advise. Regards, Datta. Graduate Student, UMICH.

2 1

Re: [lxml-dev] etree not printing pretty :(
by Stefan Behnel 29 Dec '09

29 Dec '09

Martin Seiler, 19.11.2009 12:05: >>> I am rather new to python and lxml and I wonder about the output of my >>> tree. I add an Element, which contains some childs to a tree. When I >>> write the output everything is in pretty print, but the elements I >>> appended. They show up in one line. >> did you read this? >> >> http://codespeak.net/lxml/FAQ.html#why-doesn-t-the-pretty-print-option-refo… > > Yes, I did and applied before. If I print out the Element before > appending it, it works fine, just after appending it to an existing tree > it shows this behavior. libxml2 uses a heuristic to handle document-style content. When it finds tail text on an element, it switches to keeping text content intact within the respective subtree. That's likely the problem here. Setting the .tail property of each element to None should fix this, i.e. for el in root.iter(): el.tail = None > Also changing the call to: > >> et.ElementTree(element=eggs, parser=parser).write(xmlout, pretty_print=True) > > doesn't help? No. Setting a different parser here doesn't do anything (probably worth an exception...) Stefan

1 0

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

2005

2004

lxml - The Python XML Toolkit December 2009