Mailman 3 August 2005 - lxml - The Python XML Toolkit

[lxml-dev] Python interpreter segfault setting element tail
by Kieran Holland 03 Nov '05

03 Nov '05

Thanks for lxml - very nice. Unfortunately the following fragment causes my python interpreter to crash and burn: ---------------- from lxml import etree root = etree.fromstring('<html><p></p>x</html>') for elem in root: elem.tail = '' ----------------- System: gentoo linux 2004.3 on x86 python 2.4.1 lxml 0.7 (and 0.6) libxml 2.6.19 Kieran

2 2

[lxml-dev] Repeating RelaxNG validation
by Robert Kern 12 Sep '05

12 Sep '05

Hi there, I'm in the process of testing hooks for reporting libxml2's so-called "structured errors" in lxml. Among other things, I want to get a more diagnostic message when a document fails to validate against a RelaxNG schema. I ran across a problem where an invalid document will fail the validation the first time but succeed when the validation is repeated. I haven't been able to isolate a small test case, unfortunately; my attempts at simplification give the expected results. I am on OS X 10.4 using lxml pristine from SVN, libxml2 2.6.20 and libxslt 1.1.14 (note: these are *not* the ones that come with the OS but the problem exists there, too). Attached are the RelaxNG schema, the invalid document, and a failing unit test. I hope they're not too large for the list. I apologize in advance if they are. FYI: the invalid element is /notebook/sheet/ipython-block/para on line 18. Thank you, in advance! -- Robert Kern rkern(a)ucsd.edu "In the fields of hell where the grass grows high Are the graves of dreams allowed to die." -- Richard Harter <notebook> <head> <meta content="/usr/local/bin/ipython" name="cmdline"> </meta> </head> <sheet> <title>Scipy Tutorial</title> <para>There are two (interchangeable) ways to deal with 1-d polynomials in Scipy. The first is to use the <emphasis role="bold">poly1d</emphasis> class in <emphasis role="bold">scipy_base</emphasis>. This class accepts coefficients or polynomial roots to initialize a polynomial. The polynomial object can then be manipulated in algebraic expressions, integrated, differentiated, and evaluated. It even prints like a polynomial: </para> <ipython-block logid="default-log"> <para>This shouldn't be here.</para> <ipython-cell type="input" number="3"> </ipython-cell> <ipython-cell type="input" number="4"> </ipython-cell> <ipython-cell type="input" number="5"> </ipython-cell> <ipython-cell type="stdout" number="5"> </ipython-cell> <ipython-cell type="input" number="6"> </ipython-cell> <ipython-cell type="stdout" number="6"> </ipython-cell> <ipython-cell type="input" number="7"> </ipython-cell> <ipython-cell type="stdout" number="7"> </ipython-cell> <ipython-cell type="input" number="8"> </ipython-cell> <ipython-cell type="output" number="8"> </ipython-cell> <ipython-cell type="input" number="9"> </ipython-cell> <ipython-cell type="output" number="9"> </ipython-cell> </ipython-block> <para>The other way to handle polynomials is an array of coefficients with the first element of the array giving the coefficient of the highest power. There are explicit functions to add, subtract, multiply, divide, integrate, differentiate, and evalute polynomials represented as sequences of coefficients. </para> </sheet> <ipython-log id="default-log"><cell number="3"><input> from scipy import * </input></cell> <cell number="4"><input> p = poly1d([3,4,5]) </input></cell> <cell number="5"><input> print p </input><stdout> 2 3 x + 4 x + 5 </stdout></cell> <cell number="6"><input> print p*p </input><stdout> 4 3 2 9 x + 24 x + 46 x + 40 x + 25 </stdout></cell> <cell number="7"><input> print p.integ(k=6) </input><stdout> 3 2 x + 2 x + 5 x + 6 </stdout></cell> <cell number="8"><input> p.deriv() </input><output>poly1d([6, 4]) </output></cell> <cell number="9"><input> p([4,5]) </input><output>array([ 69, 100]) </output></cell> </ipython-log> </notebook> import unittest from lxml import etree class DuplicateRelaxNGTestCase(unittest.TestCase): def test_once(self): rng = etree.RelaxNG(file='nbk.rng') baddoc = etree.parse('tut-2.3.5-bad.nbk') self.assert_(not rng.validate(baddoc)) # try again self.assert_(not rng.validate(baddoc)) def test_suite(): suite = unittest.TestSuite() suite.addTests([unittest.makeSuite(DuplicateRelaxNGTestCase)]) return suite if __name__ == '__main__': unittest.main()

3 6

[lxml-dev] News on the dump() crash ?
by Steve Howe 09 Sep '05

09 Sep '05

Hello all, Any news on the dump() crash ? ###################################################### from lxml import etree as ElementTree ElementTree.dump(ElementTree.Element('a')) ###################################################### This is crashing Python 2.4.1 on XP. Could anyone please anyone confirm if this crashes interpreters on other platforms too ? -- Best regards, Steve mailto:howe@carcass.dhs.org

4 4

[lxml-dev] [PATCH] etree.fromstring and encoding
by Olivier Grisel 09 Sep '05

09 Sep '05

Hi list, I think ElementTree.fromstring expects a raw string as argument but the current lxml.etree.XML calls .encode("UTF-8") on it which is not nice if it's already an utf8 encoded string. Hence the patch. I'm not sure what should be expected though. If you guys have a clue :) I can write more tests with various encoded strings and unicode object as arg and try to compare ElementTree's behaviour with lxml's to better reproduce it in lxml. Just tell me if it's useful/required. Best, -- Olivier

2 1

[lxml-dev] lxml and XML 1.1
by Ken Beesley 24 Aug '05

24 Aug '05

Does lxml handle XML 1.1 documents? I have an application where I need to use XML 1.1. In particular, I need to parse/process XML documents that include control characters that are valid in XML 1.1 but not in 1.0. Thanks, Ken

2 1

[lxml-dev] XPath id function on newly created elements
by Jamie Norrish 24 Aug '05

24 Aug '05

Hi, I have a document which, after parsing into an ElementTree, I validate against a RELAX NG schema. At this point I can use the XPath id function to find the element with a particular id. However, after adding a new element which has an attribute which the schema defines as holding an ID, that element is not found by the id function. If I validate the document against the schema again, nothing changes (not that I think it should be necessary to do this): the document is valid, and the id function doesn't find the element with the appropriate id. If this is the correct behaviour, how do I get the new ids to be recognised? Jamie -- Artefact Publishing: http://www.artefact.org.nz/ GnuPG Public Key: http://www.artefact.org.nz/people/jamie.html

2 3

[lxml-dev] Adding namespaced elements
by Jamie Norrish 23 Aug '05

23 Aug '05

It appears as though creating a namespaced element using etree.SubElement(d, 'm:bar', nsmap={'m': ns}) does not work correctly, while using the etree.SubElement(d, '{ns}bar') form does. The attached script hopefully illustrates the problem, and also the way that the serialised XML is incorrect in showing an element which was created with no namespace exactly the same as an element in the default namespace (though it appears the parser picks this up correctly when reparsing the output!?). Am I misunderstanding something fundamental? Note that I tried this with both 0.7 and r16184 of the trunk. Jamie -- Artefact Publishing: http://www.artefact.org.nz/ GnuPG Public Key: http://www.artefact.org.nz/people/jamie.html

3 3

Re[2]: [lxml-dev] ElementTree.dump() crashing the interpreter
by Steve Howe 22 Aug '05

22 Aug '05

Hello Martijn, Thursday, August 18, 2005, 12:38:41 PM, you wrote: >> Just wondering: could you reproduce the error ? > I haven't had the chance to try yet. Let's keep the discussion on the > lxml mailing list by the way, that way others can chip in if they feel > like helping. Sorry, but... any news on this subject ? -- Best regards, Steve mailto:howe@carcass.dhs.org

1 0

Re: [lxml-dev] setup on win32
by Martijn Faassen 20 Aug '05

20 Aug '05

[cc-ing this to the list] Steve Howe wrote: > Hello Martijn, > > Thursday, August 18, 2005, 12:47:35 PM, you wrote: > >>Hm, what about leaving the binaries in the original location and writing >>a script that can download them when making the windows installer? I'd >>prefer to keep svn for maintaining the source for now, and by using a >>script we can ensure we get the right versions automatically, so we >>don't place a lot of burden on the person who runs the script to make >>the installer. > >>After all, we're not maintaining those binaries ourselves, and it's not >>unusual for package-build scripts to download stuff from the web. If >>zlatkovic.com tends to go down or something we can always move some of >>the packages onto codespeak's lxml website or something, and then >>download with the script from there. > > I think the zlatkovic.com admins might not like a third-party script to > be downloading from them (I would not like, for sure). Also, you guys > plan to download the .zip package, unpack it, locate the dlls and > install them somewhere the PATH ? I'm not suggesting any end users use this script, just the people who prepare the windows package. > I think the best solution is having a regular lxml package and a > separate one with just the dependency dlls. This way, when the > libxml/libxslt gets updated, just that package needs to be downloaded, > and the same for when lxml gets updated. This way, we can also keep > lxml's download smaller, and who wants to get the libxml/libxslt > packages from zlatkovic.com can do it themselves. The dlls package > should be seen as a *courtesy*. But this makes installation on Windows harder -- it requires people to at the very least download and install two packages, possibly even from multiple locations. I think an installer should make life as easy as possible. A developer who wants to use lxml downloads one package, installs it, and gets going. People who want or need control (such as people who want to bundle lxml with a larger application with an installer of its own, or lxml developers on Windows) can arrange things their own way still, of course. In addition, packaging everything needed for lxml into one package also makes it easier to control which version of libxml2 is used by lxml on Windows (not all combinations might work, after all). I see this as a good thing. While newer versions of lxml might work with older versions of libxml2 and newer versions of libxml2 might work with older versions of lxml this is not always guaranteed. It's better to have a stable combination of lxml and libxml2 (per lxml release) that we support (open-source-ly) on Windows, and for anything else you're on your own. A digression: I've seen some of the problems brought by various combinations of the Python XML core libraries, PyXML and 4suite. While I don't expect libxml2/lxml combinations to be that troublesome (the PyXML situation with Python is special), I'd like to avoid something even remotely similar with lxml -- I've experienced how difficult it made life for developers using PyXML. I think we should reserve the right to say "lxml 0.foo is only supported with libxml2 version bar.baz, anything else you're on your own", when we need to. (on any platform). We're not doing this now, and hopefully we don't have to as libxml2 is stable enough, but I still want to be able to say such a thing in the future if necessary. Something interesting to investigate in all this may be the recent developments in the Python world like setuptools and Python eggs for distribution of Python packages. I haven't had any experience yet with those, but I think there's some dependency management in play. Dependency management might be a good compromise; it makes installation easy while retaining separation between packages. Whether the recent Python package management developments can actually do this I don't know, however -- the libxml2 dlls are not a python package after all. Perhaps someone can investigate this and report to the list. Regards, Martijn

1 0

[lxml-dev] DTD declaration
by Zarick Lau 20 Aug '05

20 Aug '05

Hi lists, I've just started exploring with ElementTree API with lxml. As I need xpath, original etree is not my favour this time. I'm writing a small module to wrapping up a XBEL file. I do it originally with libxml2 and found that it is too difficult to handle. But when trying to port my beta vesion module to lxml base I can't found any way to do the following stuff.. is it I have overlook anything. Or these functionality isnot explosed by lxml? 1. Setting 'DTD' declaration and XML declaration that is something like the following libxml2 o = libxml2.newDoc("1.0") o.createIntSubset("xbel", XBEL_DTD_SYSTEM_ID, XBEL_DTD_PUBLIC_ID) Is there any way to do the same thing with lxml? 2. Writing an indented version of the XML file that is someting like 'formatDump(file, 1)' in libxml2 3. getting 'sibling' and 'parent' Regards, Zarick Lau

2 1