Mailman 3 November 2009 - lxml - The Python XML Toolkit

[lxml-dev] lxml has its page on launchpad
by Stefan Behnel 11 Apr '23

11 Apr '23

Hi all, I added the lxml project to launchpad, the Ubuntu Bug-Tracker. It also has a FAQ engine and a couple of other goodies. https://launchpad.net/lxml It's easy to sign up for launchpad, BTW, no 90%-footnotes-contract. Have fun, Stefan

9 9

[lxml-dev] Checking whether a node is a comment/element
by Geoffrey Sneddon 10 Apr '23

10 Apr '23

Hi, What's the best way to check whether a given node is a comment or an element? For the former, I'm currently using isinstance(node, etree._Comment), which is rather obviously sub-optimal. -- Geoffrey Sneddon <http://gsnedders.com/>

6 6

[lxml-dev] Reparenting a node
by Lawrence Oluyede 30 Jan '23

30 Jan '23

I have a doc A and a doc B, I'd like to put a node extracted from A in the document B but I always get a ValueError: ValueError: Element is not a child of this node. I didn't find any "setparent" in the API. How can I do this? -- Lawrence, oluyede.org - neropercaso.it "It is difficult to get a man to understand something when his salary depends on not understanding it" - Upton Sinclair

3 2

[lxml-dev] lxml 2.0.5 released
by Stefan Behnel 11 Jan '23

11 Jan '23

Hi all, lxml 2.0.5 is on PyPI. This is a bug-fix-only release of the stable 2.0 series. Have fun, Stefan 2.0.5 (2008-05-01) Bugs fixed * Resolving to a filename in custom resolvers didn't work. * lxml did not honour libxslt's second error state "STOPPED", which let some XSLT errors pass silently. * Memory leak in Schematron with libxml2 >= 2.6.31.

3 4

[lxml-dev] Building LXML Trunk
by Sidnei da Silva 31 Aug '22

31 Aug '22

Hi, I've tried to build lxml from trunk today, on Win32. Got the following error: src\lxml\etree.c(880) : error C2059: syntax error : ')' src\lxml\etree.c(881) : error C2059: syntax error : ')' src\lxml\etree.c(882) : error C2059: syntax error : ')' src\lxml\etree.c(883) : error C2059: syntax error : ')' Any clue? Smells like a Pyrex issue? -- Sidnei da Silva Enfold Systems http://enfoldsystems.com Fax +1 832 201 8856 Office +1 713 942 2377 Ext 214

4 4

[lxml-dev] confusing xpath performance characteristics
by jholg＠gmx.de 24 Aug '12

24 Aug '12

Hi, I ran into some performance characteristics of lxml/libxml2 xpath that I find rather confusing: I try to find the @type attribute of a certain element in an XML Schema (which contains lots of complexType definitions with lots of elements in them; unfortunately I can't post the schema): >>> timeit.Timer(stmt="""xpath(schema)""", setup="""from lxml import etree, objectify; schema=etree.parse('NDM.xsd').getroot(); xpath = etree.XPath('//xs:element[@name="equity"]/@type', namespaces={'xs': 'http://www.w3.org/2001/XMLSchema'})""").repeat(number=10) [0.095885038375854492, 0.096823930740356445, 0.096174955368041992] So I think I'm being smart and give a little more path information - reckoning that this should *improve* performance: >>> timeit.Timer(stmt="""xpath(schema)""", setup="""from lxml import etree, objectify; schema=etree.parse('NDM.xsd').getroot(); xpath = etree.XPath('/xs:schema//xs:element[@name="equity"]/@type', namespaces={'xs': 'http://www.w3.org/2001/XMLSchema'})""").repeat(number=10) [0.1770780086517334, 0.1775970458984375, 0.17748594284057617] Hm. Performance degrades slightly. I'm adding even more of the path to where my desired elements live in the schema: >>> timeit.Timer(stmt="""xpath(schema)""", setup="""from lxml import etree, objectify; schema=etree.parse('xsd/NDM.xsd').getroot(); xpath = etree.XPath('/xs:schema/xs:complexType//xs:element[@name="equity"]/@type', namespaces={'xs': 'http://www.w3.org/2001/XMLSchema'})""").repeat(number=10) [103.79744100570679, 103.83671712875366, 103.61817717552185] What??? >>> timeit.Timer(stmt="""xpath(schema)""", setup="""from lxml import etree, objectify; schema=etree.parse('/ae/data/pydev/hjoukl/NDM/SVN_CO/TRUNK/ndm/reference/xsd/NDM.xsd').getroot(); xpath = etree.XPath('/xs:schema/xs:complexType/*/xs:element[@name="equity"]/@type', namespaces={'xs': 'http://www.w3.org/2001/XMLSchema'})""").repeat(number=10) [0.044407129287719727, 0.044126987457275391, 0.044229030609130859] >>> Ok, this version's better than my naive approach, which seems logical to me. But why would '/xs:schema/xs:complexType//xs:element[@name="equity"]/@type' perform drastically slower than '/xs:schema/xs:complexType//xs:element[@name="equity"]/@type' ? libxml2 problem? Running the same xpaths in Oxygen I don't notice performance differences (can't profile this). Holger -- GRATIS für alle GMX-Mitglieder: Die maxdome Movie-FLAT! Jetzt freischalten unter http://portal.gmx.net/de/go/maxdome01

2 5

[lxml-dev] Evaluate XSLT directives inside custom XSLT extension elements
by Marat Dakota 25 Jan '10

25 Jan '10

Hi, I have a problem I couldn't find the way to solve. Sadly if it's not possible. That's the code from XSLT extensions tutorial: <xsl:template match="*"> <my:python-extension> <some-content /> </my:python-extension> </xsl:template> Everything's working cool, but I need a bit more: <xsl:template match="*"> <my:python-extension> <something><xsl:attribute name="test"><xsl:value-of select="111" /></xsl:attribute></something> </my:python-extension> </xsl:template> Let's assume that's my execute function's signature: execute(self, context, self_node, input_node, output_parent) When it's started by XSLT processor, self_node contains <something> tag with <xsl:attribute> and <xsl:value-of> tags inside it. What should I do to ask XSLT processor to evaluate it? I need to make self_node containing <something test="111" />. And even more. What if my XSLT file looks like: <xsl:template match="*"> <my:python-extension> <something> <xsl:attribute name="test"><xsl:value-of select="111" /></xsl:attribute> <my:python-extension> <xsl:attribute name="test2"><xsl:value-of select="222" /></xsl:attribute> </my:python-extension> </something> </my:python-extension> </xsl:template> I need to have execute function called for deepest <my:python-extension> first, I need to evaluate <xsl:attribute> inside it to have <my:python-extension test2="222">... And to process the rest <my:python-extension> having the result of processed <my:python-extension test2="222"> and all XSLT instructions inside. Is there a way to do it? Or maybe there is some other way to get such possibilities? Thanks. -- Marat

2 17

[lxml-dev] better schematron support
by jholg＠gmx.de 22 Dec '09

22 Dec '09

Hi, schematron support in lxml is currently a second-class citizen due to libxml2 restrictions, see e.g. http://mail.gnome.org/archives/xml/2007-August/msg00016.html where Stefan commented on error reporting deficiencies or http://mail.gnome.org/archives/xml/2009-September/msg00022.html where Daniel of libxml2 fame comments on a feature request to support schematron embedded in XML Schema, basically stating that the implementation is incomplete. However, there is a pure-XSLT implementation of the now-ISO-standardized schematron by its inventor (and editor of the standard) Rick Jelliffe, the so-called skeleton implementation: http://www.schematron.com/ (Daniel also mentions this in his comment) Basically, the "skeleton" toolchain creates an xslt that is used for validation. Indeed, schematron itself is just a well-defined way of using xslt for validation, which I was looking into but really wasn't aware that schematron does this exactly. The "skeleton" implementation is available in both xslt 1 and 2 notions. The toolchain steps are (taken from www.schematron.com, modifications by me) 0) [Extract from XML Schema/RelaxNG schema)] 1) Process inclusions 2) Process abstract patterns 3) Compile the schema 4) Validate which translates to xsltproc XSD2Schtrn.xsl XMLSchema.xsd > theSchema.sch or xsltproc RNG2Schtrn.xsl RelaxNGSchema.rng > theSchema.sch xsltproc iso_dsdl_include.xsl theSchema.sch > theSchema1.sch xsltproc iso_abstract_expand.xsl theSchema1.sch > theSchema2.sch xsltproc iso_svrl_for_xsltn.xsl theSchema2.sch > theSchema.xsl xsltproc theSchema.xsl myDocument.xml > myResult.xml Enter libxslt aka lxml's xslt capabilities: It looks pretty easy to integrate this xslt-based toolchain into lxml, effectively enabling full ISO schematron support. I suggest complementing the current lxml schematron support using this approach: - add the necessary stylesheets (extraction and skeleton implementation) to the lxml codebase - add a convenient API to support xslt-based schematron validation to lxml that - hides toolchain steps, at least in default mode - fits in with the current validators' API - provides support for the parameters used for the separate toolchain steps And finally: Maybe somebody has already done this with lxml + schematron. Care to step forward? Any Comments? Holger -- Jetzt kostenlos herunterladen: Internet Explorer 8 und Mozilla Firefox 3.5 - sicherer, schneller und einfacher! http://portal.gmx.net/de/go/chbrowser

2 19

[lxml-dev] Critical crashes on Windows under high load
by Martin Aspeli 05 Dec '09

05 Dec '09

Hi folks, We have an incredibly frustrating, show-stopping problem using lxml (under Deliverance, in front of a repoze.zope2 pipeline serving up a Plone site) on Windows. Under high load, the Python process crashes. There is no traceback in the log, so I can't identify where it actually happens, but we get a Windows error dialogue saying python.exe (or pythonservice.exe if running as a Windows service) has crashed in etree.pyd (at some binary address, no line numbers or function references). The Deliverance (0.3/trunk) rules use fairly complex xpath expressions. We're trying to simplify these, but there's nothing obviously wrong, and in any case it shouldn't crash. We've tried to run both multi-threaded and single-threaded 'paster' processes: the problem happens with both. I did read somewhere that it's possible to build a single-threaded lxml egg (?), but I haven't found one. We would be incredibly grateful for any help with (a) debugging and (b) resolving this. At present, we're having to fight a lot of nervousness regarding the production-worthiness of our Deliverance/lxml based solution, which is rather unfortunate. :-( Cheers, Martin

2 6

[lxml-dev] XPath optimization troubles.
by John Krukoff 27 Nov '09

27 Nov '09

Hello, I expect this is properly a libxml2 question, but it's weird enough I wanted to check here first to make sure that lxml isn't effecting the results. I have equivalent XPath expressions, one using prefixes to do the selection, and one using namespace-uri to do the check. The namespace-uri version consistently runs 2-3x faster on a range of test data, and I have no idea why. Here's the prefix version: > '//@gizmo:*/parent::*[ not( self::gizmo:* ) ]' And here's the namespace-uri version: > '//@*[ namespace-uri( ) = "%(gizmo)s" ]/parent::*[ namespace-uri( ) != "%(gizmo)s" ]' % namespaces I'm running these as compiled expressions using etree.XPath( ..., namespaces = namespaces ), if that makes a difference. Any hints? Or a faster XPath to do the same thing for the ambitiously bored? -- John Krukoff <jkrukoff(a)ltgc.com> Land Title Guarantee Company

2 2