Mailman 3 March 2009 - lxml - The Python XML Toolkit

[lxml-dev] lxml has its page on launchpad
by Stefan Behnel April 11, 2023

April 11, 2023

Hi all, I added the lxml project to launchpad, the Ubuntu Bug-Tracker. It also has a FAQ engine and a couple of other goodies. https://launchpad.net/lxml It's easy to sign up for launchpad, BTW, no 90%-footnotes-contract. Have fun, Stefan

9 9

[lxml-dev] Checking whether a node is a comment/element
by Geoffrey Sneddon April 10, 2023

April 10, 2023

Hi, What's the best way to check whether a given node is a comment or an element? For the former, I'm currently using isinstance(node, etree._Comment), which is rather obviously sub-optimal. -- Geoffrey Sneddon <http://gsnedders.com/>

6 6

[lxml-dev] Reparenting a node
by Lawrence Oluyede Jan. 30, 2023

Jan. 30, 2023

I have a doc A and a doc B, I'd like to put a node extracted from A in the document B but I always get a ValueError: ValueError: Element is not a child of this node. I didn't find any "setparent" in the API. How can I do this? -- Lawrence, oluyede.org - neropercaso.it "It is difficult to get a man to understand something when his salary depends on not understanding it" - Upton Sinclair

3 2

[lxml-dev] lxml 2.0.5 released
by Stefan Behnel Jan. 11, 2023

Jan. 11, 2023

Hi all, lxml 2.0.5 is on PyPI. This is a bug-fix-only release of the stable 2.0 series. Have fun, Stefan 2.0.5 (2008-05-01) Bugs fixed * Resolving to a filename in custom resolvers didn't work. * lxml did not honour libxslt's second error state "STOPPED", which let some XSLT errors pass silently. * Memory leak in Schematron with libxml2 >= 2.6.31.

3 4

[lxml-dev] Building LXML Trunk
by Sidnei da Silva Aug. 31, 2022

Aug. 31, 2022

Hi, I've tried to build lxml from trunk today, on Win32. Got the following error: src\lxml\etree.c(880) : error C2059: syntax error : ')' src\lxml\etree.c(881) : error C2059: syntax error : ')' src\lxml\etree.c(882) : error C2059: syntax error : ')' src\lxml\etree.c(883) : error C2059: syntax error : ')' Any clue? Smells like a Pyrex issue? -- Sidnei da Silva Enfold Systems http://enfoldsystems.com Fax +1 832 201 8856 Office +1 713 942 2377 Ext 214

4 4

[lxml-dev] Compilation of lxml on Windows 64-bit
by Hanni Ali July 17, 2009

July 17, 2009

Hi All, Came across this little gem of a Python module which seems able to translate some XML reports for me nicely. I was very pleased to see a Python 2.6 installer for windows, made checking it out much easier than many modules I use. So thanks for that. However although my testing seems to confirm this is the ideal module I do need to deploy to our production environment which is a 64-bit Windows environment running Python 2.6 (I code on 32-bit box for simplicity/resource reasons). Has anyone compiled and used lxml on this platform combination? If not does anyone foresee me having any issues compiling it for this platform? Thanks, Hanni

2 7

[lxml-dev] html5lib tree builder in lxml 2.2
by Geoffrey Sneddon March 29, 2009

March 29, 2009

Hi, Getting around to actually looking at lxml 2.2's html5lib support, I note that it has its own treebuilder: I presume there was some reason (bugs?) that html5lib's own wasn't used. Would it be possible to get a patch for html5lib that would fix these issues (this'll need to be under the MIT license)? -- Geoffrey Sneddon <http://gsnedders.com/>

2 3

Re: [lxml-dev] [Question #65510]: How to set libxml:XML_PARSE_HUGE-option in lxml?
by Stefan Behnel March 27, 2009

March 27, 2009

bol wrote: > we are working with large text corpora (bigger than 10mb). > Lxml is used for splitting this corpora-xml-files and run via sockets (old > non-xml-using) binaries for i.e. pos-tagging or tokenizing. Ok, that gives you a) the bit of structure that you need and b) safe and portable encoding support (which I assume is critical here), so that's fine with me. After all, XML is used for all sorts of things these days... > The option XML_PARSE_HUGE should be as in libxml default off. That's what I was wondering about. It's (sort of) on by default if you use libxml2 2.6.x and 2.7.[012], but it's supposed to be off by default if you use libxml2 2.7.3 and later. That's outside of the control of lxml. So you would get one behaviour on one system and a different behaviour on another system, even with the same version of lxml. However, this is meant as a security measure to prevent traps like the billion laughs attack. Therefore, I do understand that a) most people won't notice and b) having it on by default seems like the right setting. Is there any opposition to keeping the enforced parser restrictions (limited tree depth and text node length) enabled by default in newer libxml2 versions, and to provide a parser switch for disabling them? The alternative would be to disable them by default on all libxml2 versions, and to provide a switch that enables them if libxml2 supports it. But a safe default sounds a lot better. Stefan

2 2

[lxml-dev] Version in trunk (svn:63387) doesn't compile on Fedora 9.x86_64 / Fedora 10.x86_64
by Andreas March 27, 2009

March 27, 2009

Hi, the version in trunk seems broken for Fedora 9/10. $ uname -a Linux rechner1 2.6.27.19-78.2.30.fc9.x86_64 #1 SMP Tue Feb 24 19:44:45 EST 2009 x86_64 x86_64 x86_64 GNU/Linux and $ uname -a Linux rechner2 2.6.27.7-134.fc10.x86_64 #1 SMP Mon Dec 1 22:21:35 EST 2008 x86_64 x86_64 x86_64 GNU/Linux libxml2 2.7.3 Python 2.5.1 ---- snip ---- python setup.py build_ext -i Building lxml version 2.2-63198. Building with Cython 0.11. Using build configuration of libxslt 1.1.24 Building against libxml2/libxslt in the following directory: /usr/lib64 running build_ext building 'lxml.etree' extension creating build creating build/temp.linux-x86_64-2.5 creating build/temp.linux-x86_64-2.5/src creating build/temp.linux-x86_64-2.5/src/lxml gcc -pthread -fno-strict-aliasing -DNDEBUG -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector --param=ssp-buffer-size=4 -m64 -mtune=generic -D_GNU_SOURCE -fPIC -fPIC -I/usr/include/libxml2 -I/usr/include/python2.5 -c src/lxml/lxml.etree.c -o build/temp.linux-x86_64-2.5/src/lxml/lxml.etree.o -w src/lxml/lxml.etree.c:3299: error: expected ‘=’, ‘,’, ‘;’, ‘asm’ or ‘__attribute__’ before ‘__pyx_v_4lxml_5etree_XSLT_DOC_DEFAULT_LOADER’ src/lxml/lxml.etree.c:3600: error: expected declaration specifiers or ‘...’ before ‘xsltLoadType’ src/lxml/lxml.etree.c:3601: error: expected declaration specifiers or ‘...’ before ‘xsltLoadType’ src/lxml/lxml.etree.c:110035: error: expected declaration specifiers or ‘...’ before ‘xsltLoadType’ src/lxml/lxml.etree.c: In function ‘__pyx_f_4lxml_5etree__xslt_store_resolver_exception’: src/lxml/lxml.etree.c:110095: error: ‘__pyx_v_c_type’ undeclared (first use in this function) src/lxml/lxml.etree.c:110095: error: (Each undeclared identifier is reported only once src/lxml/lxml.etree.c:110095: error: for each function it appears in.) src/lxml/lxml.etree.c:110095: error: ‘XSLT_LOAD_DOCUMENT’ undeclared (first use in this function) src/lxml/lxml.etree.c: At top level: src/lxml/lxml.etree.c:110242: error: expected declaration specifiers or ‘...’ before ‘xsltLoadType’ src/lxml/lxml.etree.c: In function ‘__pyx_f_4lxml_5etree__xslt_doc_loader’: src/lxml/lxml.etree.c:110257: error: ‘__pyx_v_c_type’ undeclared (first use in this function) src/lxml/lxml.etree.c:110257: error: ‘XSLT_LOAD_DOCUMENT’ undeclared (first use in this function) src/lxml/lxml.etree.c:110280: error: ‘XSLT_LOAD_STYLESHEET’ undeclared (first use in this function) src/lxml/lxml.etree.c:110385: error: too many arguments to function ‘__pyx_f_4lxml_5etree__xslt_store_resolver_exception’ src/lxml/lxml.etree.c: In function ‘__pyx_pf_4lxml_5etree_4XSLT___call__’: src/lxml/lxml.etree.c:113230: error: ‘xsltTransformContext’ has no member named ‘dict’ src/lxml/lxml.etree.c:113241: error: ‘xsltTransformContext’ has no member named ‘dict’ src/lxml/lxml.etree.c:113253: error: ‘xsltTransformContext’ has no member named ‘dict’ src/lxml/lxml.etree.c:113262: error: ‘xsltTransformContext’ has no member named ‘dict’ src/lxml/lxml.etree.c: In function ‘initetree’: src/lxml/lxml.etree.c:147853: error: ‘__pyx_v_4lxml_5etree_XSLT_DOC_DEFAULT_LOADER’ undeclared (first use in this function) src/lxml/lxml.etree.c:147853: error: ‘xsltDocDefaultLoader’ undeclared (first use in this function) error: command 'gcc' failed with exit status 1 make: *** [inplace] Error 1 ---- snip --- Regards Andreas

2 1

Re: [lxml-dev] [Question #65510]: How to set libxml:XML_PARSE_HUGE-option in lxml?
by Stefan Behnel March 27, 2009

March 27, 2009

Hi, forwarding this to the mailing list where it belongs. bol wrote: > in lixbml2 changelog: > > Daniel Veillard Sun Jan 18 15:06:05 CET > > * include/libxml/parserInternals.h SAX2.c: add a new define > XML_MAX_TEXT_LENGHT limiting the maximum size of a single text node, > the defaultis 10MB and can be removed with the HUGE parsing option Yes, this was changed in libxml2 2.7.x. > So how can I set in lxml this HUGE-option? You currently can't, and I wonder if this should really be an option and what the default should be here. > At the moment I use a from me modified version of libxml2 (with > XML_MAX_TEXT_LENGTH set to 100MB), which solves my problem. But I hope to > find a lxml-way to solve this. Could you say something about your use case? Stefan

1 0

[lxml-dev] test failures, lxml-2.2
by Jorge Barrios March 26, 2009

March 26, 2009

I am hoping that there's an easy fix. I couldn't find anything about it in the mailing list. Anyone know what the problem might be? Jorge root@a0110:/data3/lxml-2.2# uname -a Linux a0110 2.6.18-8.el5PAE #1 SMP Thu Mar 15 20:29:51 EDT 2007 i686 i686 i386 GNU/Linux root@a0110:/data3/lxml-2.2# make test python setup.py build_ext -i Building lxml version 2.2. Building with Cython 0.11. Using build configuration of libxslt 1.1.24 Building against libxml2/libxslt in the following directory: /usr/ local/lib running build_ext python test.py -p -v TESTED VERSION: 2.2.0 Python: (2, 5, 2, 'final', 0) lxml.etree: (2, 2, 0, 0) libxml used: (2, 7, 3) libxml compiled: (2, 7, 3) libxslt used: (1, 1, 24) libxslt compiled: (1, 1, 24) 1010/1010 (100.0%): Doctest: xpathxslt.txt ---------------------------------------------------------------------- Ran 1010 tests in 29.973s OK PYTHONPATH=src: python selftest.py ********************************************************************** File "/var/data3/lxml-2.2/selftest.py", line 575, in selftest.encoding Failed example: serialize(elem) Expected: '<tag key="åöö<>"/>' Got: '<tag key="åöö<>"/>' ********************************************************************** File "/var/data3/lxml-2.2/selftest.py", line 577, in selftest.encoding Failed example: serialize(elem, encoding="utf-8") Expected: '<tag key="\xc3\xa5\xc3\xb6\xc3\xb6<>"/>' Got: '<tag key="åöö<>"/>' ********************************************************************** File "/var/data3/lxml-2.2/selftest.py", line 579, in selftest.encoding Failed example: serialize(elem, encoding="us-ascii") Expected: '<tag key="åöö<>"/>' Got: '<tag key="åöö<>"/>' ********************************************************************** File "/var/data3/lxml-2.2/selftest.py", line 581, in selftest.encoding Failed example: serialize(elem, encoding="iso-8859-1").lower() Expected: '<?xml version=\'1.0\' encoding=\'iso-8859-1\'?>\n<tag key="\xe5\xf6\xf6<>"/>' Got: '<?xml version=\'1.0\' encoding=\'iso-8859-1\'?>\n<tag key="åöö<>"/>' ********************************************************************** 1 items had failures: 4 of 29 in selftest.encoding ***Test Failed*** 4 failures. 181 tests ok. PYTHONPATH=src: python selftest2.py ********************************************************************** File "/var/data3/lxml-2.2/selftest2.py", line 196, in selftest2.encoding Failed example: serialize(elem) Expected: '<tag key="åöö<>"/>' Got: '<tag key="åöö<>"/>' ********************************************************************** File "/var/data3/lxml-2.2/selftest2.py", line 198, in selftest2.encoding Failed example: serialize(elem, "utf-8") Expected: '<tag key="\xc3\xa5\xc3\xb6\xc3\xb6<>"/>' Got: '<tag key="åöö<>"/>' ********************************************************************** File "/var/data3/lxml-2.2/selftest2.py", line 200, in selftest2.encoding Failed example: serialize(elem, "us-ascii") Expected: '<tag key="åöö<>"/>' Got: '<tag key="åöö<>"/>' ********************************************************************** File "/var/data3/lxml-2.2/selftest2.py", line 202, in selftest2.encoding Failed example: serialize(elem, "iso-8859-1").lower() Expected: '<?xml version=\'1.0\' encoding=\'iso-8859-1\'?>\n<tag key="\xe5\xf6\xf6<>"/>' Got: '<?xml version=\'1.0\' encoding=\'iso-8859-1\'?>\n<tag key="åöö<>"/>' ********************************************************************** 1 items had failures: 4 of 29 in selftest2.encoding ***Test Failed*** 4 failures. 102 tests ok. root@a0110:/data3/lxml-2.2#

2 1

[lxml-dev] Spacing and the presence of xml:space="preserve"
by F Wolff March 24, 2009

March 24, 2009

Hallo all We are currently using this expression to obtain a plain text version inside a node: For example: >>> from lxml import etree >>> etree.XPath("string()") >>> string_xpath(etree.fromstring("<a> asdf <b/>fdsa </a>")) ' asdf fdsa ' This works great and returns the string assuming xml:space="preserve", in other words, spacing is taken verbatim. We work on a file format where some of the spacing is very important (XLIFF). We generate such files with xml:space="preserve" in the necessary places. Not everybody generates such files, unfortunately, so we need to also handle the normalised versions. If I rather use the XPath function "normalize-space()", I can get the normalised spacing: 'asdf fdsa' but unfortunately it does this even if xml:space="preserve" is set: >>> etree.XPath("normalize-space()") >>> string_xpath(etree.fromstring('''<a xml:space="preserve"> asdf <b/>fdsa </a>''')) 'asdf fdsa' Unfortunately, I don't see a way to get the correct version (normalised by default, but with white-space preserved if xml:space="preserved" is set). Do I have to handle the cases separately, or is there a way for lxml to help me by just doing the right thing? I could special case on the node, but it would be a bit harder to know if some xml:space directive was given higher up in the tree. Or am I missing something in XPath / lxml? Any help would be appreciated. Friedel Wolff -- Recently on my blog: http://translate.org.za/blogs/friedel/en/content/video-virtaals-functionali…

2 1

[lxml-dev] lxml 2.2 released
by Stefan Behnel March 21, 2009

March 21, 2009

Hi all, I'm proud to announce the release of lxml 2.2 final. This is a major new, stable and mature release that takes over the stable 2.x release series. All previous 2.x releases are now officially out of maintenance. It includes a large number of bug fixes and improvements (see below for a complete changelog) that make lxml 2.2 a lot more robust than the previous 2.1 and older releases. It is therefore generally worth upgrading (and it should not be too hard to do that). You can get it from: http://codespeak.net/lxml/ http://pypi.python.org/pypi/lxml/2.2 This release was built with Cython 0.11 final. Have fun, Stefan 2.2 (2009-03-21) Features added * Support for standalone flag in XML declaration through tree.docinfo.standalone and by passing standalone=True/False on serialisation. Bugs fixed * Crash when parsing an XML Schema with external imports from a filename. 2.2beta4 (2009-02-27) Features added * Support strings and instantiable Element classes as child arguments to the constructor of custom Element classes. * GZip compression support for serialisation to files and file-like objects. Bugs fixed * Deep-copying an ElementTree copied neither its sibling PIs and comments nor its internal/external DTD subsets. * Soupparser failed on broken attributes without values. * Crash in XSLT when overwriting an already defined attribute using xsl:attribute. * Crash bug in exception handling code under Python 3. This was due to a problem in Cython, not lxml itself. * lxml.html.FormElement._name() failed for non top-level forms. * TAG special attribute in constructor of custom Element classes was evaluated incorrectly. Other changes * Official support for Python 3.0.1. * Element.findtext() now returns an empty string instead of None for Elements without text content. 2.2beta3 (2009-02-17) Features added * XSLT.strparam() class method to wrap quoted string parameters that require escaping. Bugs fixed * Memory leak in XPath evaluators. * Crash when parsing indented XML in one thread and merging it with other documents parsed in another thread. * Setting the base attribute in lxml.objectify from a unicode string failed. * Fixes following changes in Python 3.0.1. * Minor fixes for Python 3. Other changes * The global error log (which is copied into the exception log) is now local to a thread, which fixes some race conditions. * More robust error handling on serialisation. 2.2beta2 (2009-01-25) Bugs fixed * Potential memory leak on exception handling. This was due to a problem in Cython, not lxml itself. * iter_links (and related link-rewriting functions) in lxml.html would interpret CSS like url("link") incorrectly (treating the quotation marks as part of the link). * Failing import on systems that have an io module. 2.2beta1 (2008-12-12) Features added * Allow lxml.html.diff.htmldiff to accept Element objects, not just HTML strings. Bugs fixed * Crash when using an XPath evaluator in multiple threads. * Fixed missing whitespace before Link:... in lxml.html.diff. Other changes * Export lxml.html.parse. 2.2alpha1 (2008-11-23) Features added * Support for XSLT result tree fragments in XPath/XSLT extension functions. * QName objects have new properties namespace and localname. * New options for exclusive C14N and C14N without comments. * Instantiating a custom Element classes creates a new Element. Bugs fixed * XSLT didn't inherit the parse options of the input document. * 0-bytes could slip through the API when used inside of Unicode strings. * With lxml.html.clean.autolink, links with balanced parenthesis, that end in a parenthesis, will be linked in their entirety (typical with Wikipedia links).

1 0

[lxml-dev] lxml 2.2beta4 - release candidate for 2.2 final
by Stefan Behnel March 20, 2009

March 20, 2009

Hi all, here is another almost-final version of lxml 2.2. Call it a release candidate, if you prefer. This release was necessary as the changelog was getting way too long, and the (crash-)bugs that were fixed in this release were too important to wait. So, updating is recommended. http://codespeak.net/lxml/dev/ http://pypi.python.org/pypi/lxml/2.2beta4 I'm very happy to announce that this is the first release that fully supports Python 3. The previous releases suffered from an annoying crash bug in Cython that was uncovered by the exception handling changes in Py3. This means that lxml now supports any officially released Python version from 2.3.x through 3.0.1. Note that the PDF documentation was broken in the last betas due to a recent system update on my side. It's back up for beta4. This release was built with revision "1796:cb0f315bb4f5" of the Cython 0.11 development branch. Have fun, Stefan 2.2beta4 (2009-02-27) Features added * Support strings and instantiable Element classes as child arguments to the constructor of custom Element classes. * GZip compression support for serialisation to files and file-like objects. Bugs fixed * Deep-copying an ElementTree copied neither its sibling PIs and comments nor its internal/external DTD subsets. * Soupparser failed on broken attributes without values. * Crash in XSLT when overwriting an already defined attribute using xsl:attribute. * Crash bug in exception handling code under Python 3. This was due to a problem in Cython, not lxml itself. * lxml.html.FormElement._name() failed for non top-level forms. * TAG special attribute in constructor of custom Element classes was evaluated incorrectly. Other changes * Official support for Python 3.0.1. * Element.findtext() now returns an empty string instead of None for Elements without text content.

2 1

[lxml-dev] Possible bug
by Bob Kline March 20, 2009

March 20, 2009

Before I dig into the work of producing a repro case, would the lxml developers be interested in a bug report if I confirm that the XSL/T parser which comes with the lxml package chokes on the serialized version of an XML tree assembled by the lxml's HTML parser when the original HTML document contains a comment which the XML spec doesn't like (because "--" appears inside the comment)? -- Bob Kline http://www.rksystems.com mailto:bkline@rksystems.com

3 8

[lxml-dev] iterparse and namespaces
by Piotr Furman March 17, 2009

March 17, 2009

Hi, I've got small problem with iterparse and namespaces. I have this code: >>> from StringIO import StringIO >>> from lxml import etree >>> print etree.__version__ 2.1.5 >>> xml = """<root xmlns="http://www.example.com"><a>1</a><a>2</a></root>""" >>> a1 = etree.iterparse(StringIO(xml), tag="a") >>> a1.next() Traceback (most recent call last): File "<stdin>", line 1, in ? File "iterparse.pxi", line 515, in lxml.etree.iterparse.__next__ (src/lxml/lxml.etree.c:77014) StopIteration >>> a2 = etree.iterparse(StringIO(xml), tag="{http://www.example.com}a") >>> a2.next() (u'end', <Element {http://www.example.com}a at -488456c4>) Is that possible to get all tags "a" without passing namespace? I mean, a2 works, but could it be possible to make a1 working too? regards Piotr Furman

6 11

[lxml-dev] space in attribute name: xpath expression?
by TP March 17, 2009

March 17, 2009

Hi everybody, It seems not possible to define with fromstring() or ET.XML a tree containing attributes with spaces. But it is possible by adding the attribute containing a space afterwards, see the example below. ################### #!/usr/bin/env python # -*- coding: utf-8 -*- import lxml.etree as ET root = ET.XML("<root><foo attri='bar'>data</foo></root>") foo_elem = root.xpath( "//foo" ) foo_elem[0].set( "tu tu", "22" ) print ET.tostring( root ) ################### We obtain: <root><foo attri="bar" tu tu="22">data</foo></root> It seems a bad idea to have spaces in attributes. I have not found a way to make an xpath request work, for example the two following ones yield an error: print root.xpath( "//*[@tu tu=22]" ) print root.xpath( "//*[@tu\ tu=22]" ) >From another point of view, often we would like to define attribute names as they are, i.e. english expressions with spaces. How do you proceed? Put underscores in the attribute names, and then remove them when displaying in the tree (for example in a graphical widget)? Or define the correspondance between the attribute names and the english names in some part of the XML file (for example, the attribute names could be tags, associated to some text that would contain the english names. Thanks -- python -c "print ''.join([chr(154 - ord(c)) for c in '*9(9&(18%.\ 9&1+,\'Z4(55l4('])" "When a distinguished but elderly scientist states that something is possible, he is almost certainly right. When he states that something is impossible, he is very probably wrong." (first law of AC Clarke)

3 3

Re: [lxml-dev] the subelements of my tree are moving alone
by Stefan Behnel March 14, 2009

March 14, 2009

[and to the list] TP wrote: > I have derived custom classes from ET._ElementTree and ET.ElementBase to > obtain a custom tree suited to my needs. It obviously depends on what you do with it, but I'm not so sure that subclassing the _ElementTree class is a good idea. It's not wrong in general, but maybe writing your own tree wrapper would be better. > It works perfectly, but it seems that the nodes under the root node (the > subelements) move sometimes "alone". The tree structure is kept, but the > address of the elements in memory is changing. As the structure is kept, it > is not a problem for lxml use only: I can walk in the tree, doing what I > need. > > But the problem is that I use this custom tree as the underlying data > structure for a PyQt custom QTreeWidget. In this widget, I use the > method "internalPointer()" of QModelIndex instances (as proposed in the > chapter 16 of book "Rapid GUI Programming with Python > and Qt" by Mark Summerfield (around p.500)). > > The problem is that if the nodes move, the "internalPointer()" of Qt are not > up to date: I obtain segmentation faults. > > Is this normal that nodes of the tree move in memory *alone*? Is this due to > the garbage collector? If yes, how to keep my pointers up to date? Here's some documentation: http://codespeak.net/lxml/dev/element_classes.html#background-on-element-pr… It's not only the "pointer", it's the Element instances that are replaced. It may be that "internalPointer()" (never heard of it, but I assume it's some kind of backpointing mechanism) does not respect Python's reference counting, so that the Element object gets garbage collected. A simple way to work around this is to keep a reference to all Element objects in the tree, initialised as my_tree.element_cache = list(root_element.iter()) But keep in mind that this needs to be maintained on tree changes. Stefan

1 0

Re: [lxml-dev] Fun With Sequencing
by Stefan Behnel March 14, 2009

March 14, 2009

[and to the list] Hi, Tennis Smith wrote: > I am generating xml files (for test purposes) that will be validated > with a schema which uses "xs:sequence". The result is that my docs have > to have exactly the right sequence of elements or it will not validate. Could you say a bit more about your constraints? Why can't you just change the schema, if it's purely for test purposes? And why can't you just look at the schema once and generate the XML documents accordingly? > Its going to be a pain to re-parse the schema file and then use the > munged data to sort my xml docs. So, is there some way in lxml I can > dump the schema's contents and use that to make sure I have everything > in order *before* I try to validate? No, and I doubt that there's a major use case for that. Validation is about figuring out that something is wrong (any usually also what is wrong), not about getting stuff fixed automatically. Stefan

1 0

[lxml-dev] Fun With Sequencing
by Tennis Smith March 11, 2009

March 11, 2009

Hi, I am generating xml files (for test purposes) that will be validated with a schema which uses "xs:sequence". The result is that my docs have to have exactly the right sequence of elements or it will not validate. Its going to be a pain to re-parse the schema file and then use the munged data to sort my xml docs. So, is there some way in lxml I can dump the schema's contents and use that to make sure I have everything in order *before* I try to validate? Failing that, are there any suggestions? Thanks -T

1 0

[lxml-dev] the subelements of my tree are moving alone
by TP March 11, 2009

March 11, 2009

Hi everybody, I have derived custom classes from ET._ElementTree and ET.ElementBase to obtain a custom tree suited to my needs. It works perfectly, but it seems that the nodes under the root node (the subelements) move sometimes "alone". The tree structure is kept, but the address of the elements in memory is changing. As the structure is kept, it is not a problem for lxml use only: I can walk in the tree, doing what I need. But the problem is that I use this custom tree as the underlying data structure for a PyQt custom QTreeWidget. In this widget, I use the method "internalPointer()" of QModelIndex instances (as proposed in the chapter 16 of book "Rapid GUI Programming with Python and Qt" by Mark Summerfield (around p.500)). The problem is that if the nodes move, the "internalPointer()" of Qt are not up to date: I obtain segmentation faults. Is this normal that nodes of the tree move in memory *alone*? Is this due to the garbage collector? If yes, how to keep my pointers up to date? Thanks in advance -- python -c "print ''.join([chr(154 - ord(c)) for c in '*9(9&(18%.\ 9&1+,\'Z4(55l4('])" "When a distinguished but elderly scientist states that something is possible, he is almost certainly right. When he states that something is impossible, he is very probably wrong." (first law of AC Clarke)

2 2

[lxml-dev] Lxml Crash
by Alex Klizhentas March 4, 2009

March 4, 2009

Hi all, sometimes i get exception killing apache process. It happens occasionally (acually it happened once on my production site), so I have no more logs up to the moment, I can only suspect that crash happens when I am trying to replace the node: def replace(self,child,new_child): root = self.getroottree().getroot() index = self.index(child) if root._should_notify(): old_child = deepcopy(child) self.insert(index,new_child) etree.ElementBase.remove(self,child) root._notify(NodeReplaced(old_child,new_child)) return self[index] else: self.insert(index,new_child) etree.ElementBase.remove(self,child) return self[index] crash log is below: *** glibc detected *** /usr/sbin/apache2: free(): invalid pointer: 0x08cd6eca *** ======= Backtrace: ========= /lib/tls/i686/cmov/libc.so.6[0xb7e26a85] /lib/tls/i686/cmov/libc.so.6(cfree+0x90)[0xb7e2a4f0] /usr/lib/libxml2.so.2(xmlFreeNodeList+0x126)[0xa984d1e6] /usr/lib/libxml2.so.2(xmlFreeNode+0x76)[0xa984d656] /usr/lib/python2.5/site-packages/lxml-2.2alpha1-py2.5-linux-i686.egg/lxml/etree.so[0xa9992bf2] /usr/lib/python2.5/site-packages/lxml-2.2alpha1-py2.5-linux-i686.egg/lxml/etree.so[0xa99b529f] I will bring in more logs if crash repeats, but I will appreciate any ideas/thoughts/comments so I can quickly eliminate/workaround/prevent the issue from happening again. -- Regards, Alex

2 2

[lxml-dev] (re-raising) exceptions problem in lxml 2.2beta4
by jholg＠gmx.de March 3, 2009

March 3, 2009

Hi, I just ran into a problem with some code that re-raises exceptions, where accessing sys.exc_info() returns (None, None, None) instead of the expected most recent exception information. This seems to happen if "some operation" is performed on the tree after the exception has been caught and before sys.exc_info() gets invoked. Looks like lxml clears the exception information somewhere on the way. Here's a minimal example where the invocation of iterchildren() triggers the behaviour: $ cat lxml_reraise.py import sys from lxml import etree print "using lxml version", etree.__version__ root = etree.Element('root') try: access = bool(sys.argv[1]) except IndexError: access = False try: raise RuntimeError('Too much foo for bar') except Exception, e: if access: print "children:", list(root.iterchildren()) print sys.exc_info() Run with lxml 2.1.5: $ python2.4 lxml_reraise.py using lxml version 2.1.5 (<class exceptions.RuntimeError at 0x132780>, <exceptions.RuntimeError instance at 0x25f0a8>, <traceback object at 0x25f080>) $ python2.4 lxml_reraise.py 1 using lxml version 2.1.5 children: [] (<class exceptions.RuntimeError at 0x132780>, <exceptions.RuntimeError instance at 0x1d5828>, <traceback object at 0x1d57d8>) Run with lxml 2.2beta4: $ ln -s ~/pydev/tmp/lxml-2.2beta4/build/lib.solaris-2.8-sun4u-2.4/lxml $ python2.4 lxml_reraise.py using lxml version 2.2.beta4 (<class exceptions.RuntimeError at 0x132780>, <exceptions.RuntimeError instance at 0x252260>, <traceback object at 0x252170>) $ python2.4 lxml_reraise.py 1 using lxml version 2.2.beta4 children: [] (None, None, None) Now, I seem to remember some discussion of changes wrt to exceptions for lxml 2.2. Might this be an (unwanted) side effect to these changes? Holger -- Psssst! Schon vom neuen GMX MultiMessenger gehört? Der kann`s mit allen: http://www.gmx.net/de/go/multimessenger01

2 1

[lxml-dev] Segmentation Fault loading graphml.xsd
by Chris Wj March 2, 2009

March 2, 2009

Before posting a new bug I want to confirm this. I am reading in graphml.xsd as a Schema to validate against, which has other xsd files that it references located in same folder. Linux x86_64, Python 2.5, lxml 2.2beta4 lxml.etree: (2, 2, -96, 0) libxml used: (2, 6, 32) libxml compiled: (2, 6, 32) libxslt used: (1, 1, 24) libxslt compiled: (1, 1, 24) Code to reproduce error: In [1]: from lxml import etree In [2]: etree.XMLSchema(file="grap graphml+svg.xsd graphml-attributes.xsd graphml-parseinfo.xsd graphml-structure.xsd graphml.dtd graphml.xsd In [2]: s = etree.XMLSchema(file="graphml.xsd") Segmentation fault Schemas can be obtained here: http://graphml.graphdrawing.org/specification.html Loading the others seg faults too. -Chris

2 1