Mailman 3 April 2007 - lxml - The Python XML Toolkit

[lxml-dev] Weird bug
by David Turner April 21, 2007

April 21, 2007

I'm trying to write some code that uses lxml, and I run into a weird memory error. Unfortunately, I can't seem to create a small testcase. So this bug report probably won't be very useful. How to reproduce: Check out the following code: http://codespeak.net/svn/z3/deliverance/branches/parallel python setup.py develop python deliverance/test_wsgi.py This will sometimes run just fine (that is, produce no output). Sometimes, it will give the following error:, which doesn't really seem to matter, since it's "most likely raised during interpreter shutdown" Exception in thread Thread-70 (most likely raised during interpreter shutdown): Traceback (most recent call last): File "/usr/lib64/python2.4/threading.py", line 442, in __bootstrap File "/home/novalis/deliverance/src/deliverance/transcluder/threadpool.py", line 91, in run File "/home/novalis/deliverance/src/deliverance/transcluder/tasklist.py", line 87, in get File "/usr/lib64/python2.4/threading.py", line 197, in wait exceptions.TypeError: 'NoneType' object is not callable Unhandled exception in thread started by Error in sys.excepthook: Original exception was: [nothing is printed here] ------------ And sometimes, there's an error in the actual test: --------- Traceback (most recent call last): File "deliverance/test_wsgi.py", line 361, in ? x[0](*x[1:]) File "deliverance/test_wsgi.py", line 156, in do_aggregate html_string_compare(res.body, res2.body) File "deliverance/test_wsgi.py", line 61, in html_string_compare raise ValueError( ValueError: Comparison failed between actual: ================== <!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN" "http://www.w3.org/TR/html4/loose.dtd"> <html> <head> <meta http-equiv="Content-Type" content="text/html; charset=UTF-8"> <title>I am a title</title> <link rel="rss" href="/rss.xml"> <link rel="rss" href="/rss.xml"> </head> <body> <div id="navbar"><div id="nav">Additional Nav Info</div></div> Some text <div id="content"> <p>Paragraph one</p> <p>Paragraph two</p> <div id="external_content"> external body text <br><br> </div> </div> </body> </html> expected: ================== <html> <head> <meta http-equiv="Content-Type" content="text/html; charset=UTF-8"> <title>I am a title</title> <link rel="rss" href="/rss.xml"> </head> <body> <div id="navbar"><div id="nav">Additional Nav Info</div></div> Some text <div id="content"> <p>Paragraph one</p> <p>Paragraph two</p> <div id="external_content"> external body text <br><br> </div> </div> </body> </html> Report: children length differs, 4 != 3 children 1 do not match: head ------------ Running valgrind shows a couple of memory errors. The first is in xmlFreeNode, when it attempts to get the dict from a doc that has been freed. The node in question is created at line 327 of tasklist.py in transcluder -- but the error comes later, during garbage collection. If anyone has any ideas, I'm all ears.

2 3

[lxml-dev] Misc questions
by David Soulayrol April 20, 2007

April 20, 2007

Hello, I'm trying my first script with lxml, and here are some (more or less) blockers I have: * How do I generate the XML header in the output ? From documentation, I thought write() would do, but I can't get it. * Is there a way to manage DTDs ? I think I read that something is ready in CVS. Is it true ? Note that it could be the moment for me to learn XML schemas or Relax NG (which one should I choose ? :) ) * For readability only, is there a way to specify the id that is created for each namespace used ? * At last, I discovered by chance the pretty_print argument of the write method. But I did not read anything about this in documentation, nor in the pydocs. Is there another interesting source of documentation to get complete function signatures (apart from the source) ? Thanks, -- David.

2 1

[lxml-dev] XSLT parameter ignored?
by Jeroen van Holst April 20, 2007

April 20, 2007

Hello, I'm trying to pass a parameter to an XSLT object as follows: xslt = etree.parse(stylesheet) style = etree.XSLT(xslt) params = {'profile.lang': 'en'} result = style(doc, params) The stylesheet is applied, but the parameter is ignored. This works in libxslt, so what am I missing? TIA, Jeroen -- -- Jeroen van Holst <jeroen(a)xos.nl> -- X/OS Experts in Open Systems BV | Phone: +31 20 6938364 -- Amsterdam, The Netherlands | Fax: +31 20 6948204

2 2

[lxml-dev] greetings, and another bug...
by Jim Rees April 17, 2007

April 17, 2007

Itamar told me I'd get best results by joining this list. First off lxml.etree is great. I'm relatively new to both Python and XML, and this is the only way to code XML stuff. I have found a few bugs, the first set of which Itamar may have already forwarded along. Today I found another. A valid XSD construct fails to validate. (Not the document to be validated, but the schema doc itself). If the minInclusive/maxInclusive facets are removed, the problem goes away. xmllint running against the same libxml2 shlibs has no problem with this. This has been consistent across 1.1.2, 1.2.1, and 1.3.beta. import lxml.etree as ET import sys trivial_schema = """<?xml version="1.0" encoding="UTF-8"?> <xsd:schema xmlns:xsd="http://www.w3.org/2001/XMLSchema"> <xsd:simpleType name="typePercentage"> <xsd:restriction base="xsd:decimal"> <xsd:minInclusive value="0"/> <xsd:maxInclusive value="100"/> </xsd:restriction> </xsd:simpleType> <xsd:element name="Percentage" type="typePercentage"/> </xsd:schema> """ schematree = ET.XML(trivial_schema) validator = ET.XMLSchema(schematree) trivial_document = """<?xml version="1.0" encoding="UTF-8"?> <Percentage>99.99999999999999999999</Percentage> """ doctree = ET.XML(trivial_document) validator.assertValid(doctree) print "Okay."

2 3

[lxml-dev] Parsing Information
by Stephan Richter April 15, 2007

April 15, 2007

Hi all, I have recently reimplemented RML (Reportlab's XML format to generate PDFs) using lxml. All works well. Now, I would like to give my users some more information when an error occurs. For a pure XML parsing error, everything is fine (though I found the failure points hard to interpret at times). But what if the XML parses correctly, but while working with the element tree an error occurs? In this case I would like to tell the user not only the error message, but also the line/column and filename of point of failure. Ideally I would have the filename, start row and start column of each element available as part of the etree Element. I have tried to find this information or hooks for it.unsuccessfully. Could someone help me out here? Regards, Stephan -- Stephan Richter CBU Physics & Chemistry (B.S.) / Tufts Physics (Ph.D. student) Web2k - Web Software Design, Development and Training

5 12

[lxml-dev] el.attrib.pop()
by Ian Bicking April 15, 2007

April 15, 2007

Should the .attrib object have a pop method? -- Ian Bicking | ianb(a)colorstudy.com | http://blog.ianbicking.org | Write code, do good | http://topp.openplans.org/careers

2 1

[lxml-dev] LXML-based doctest output checker
by Ian Bicking April 15, 2007

April 15, 2007

I have a rough but probably useful output checker for doctest that uses lxml to parse (and HTML). It's a bit like formencode.doctest_xml_compare (which uses ElementTree), but I think the output is nicer and of course the lxml aspect. It's pretty rough and highly untested, and the way it injects its output comparison into doctest is kind of lame. I suppose there's a way you could subclass the parser to use this output checker, but I didn't look into it too closely. A quick grep of doctest.py doesn't make it look easy. For now it's here: http://svn.pythonpaste.org/Paste/WSGIFilter/trunk/wsgifilter/lxmldoctest.py -- but really we have a number of little routines around lxml that we should probably break off somewhere, as they aren't directly related to WSGIFilter. Mostly they are HTML-related things, which probably don't belong in lxml directly. Though I dunno, lxml.html? It's a grab-bag though, so I'm not really proposing that at this time. -- Ian Bicking | ianb(a)colorstudy.com | http://blog.ianbicking.org | Write code, do good | http://topp.openplans.org/careers

1 0

[lxml-dev] [objectify] patch/changes proposal: xsiannotate, deannotate
by jholg＠gmx.de April 11, 2007

April 11, 2007

Hi all, I suggest 1. adding two functions to lxml.objectify: def xsiannotate(element_or_tree, ignore_old=True): """Recursively annotates the elements of an XML tree with 'xsi:type' attributes. If the 'ignore_old' keyword argument is True (the default), current 'xsi:type' attributes will be ignored and replaced. Otherwise, they will be checked and only replaced if they no longer fit the current text value. """ [...] Note: Will simply take the first schema type in PyType.xmlSchemaTypes list. def deannotate(element_or_tree, pytype=True, xsi=True): """Recursively de-annotate the elements of an XML tree by removing 'pytype' and/or 'type' attributes. If the 'pytype' keyword argument is True (the default), 'pytype' attributes will be removed. If the 'xsi' keyword argument is True (the default), 'xsi:type' attributes will be removed. """ [...] 2. Patching annotate() so that it allows for leaving pytype="str" as is if ignore_old=False. Currently it will start type-guessing/xsi-type lookup as PyType(str,...) uses no type_check function. 3. Modifying the objectify.Element() factory to default nsmap to nsmap = { "py": PYTYPE_NAMESPACE, "xsi": XML_SCHEMA_INSTANCE_NS } if it is None. This keeps namespace-information in non-root nodes nice and clean with the cool new 1.3 lookup-if-ns-is-defined-up-in-the-tree functionality. 4. Patch DataElement so that it allows s.o. using an _xsitype argument that is not registered (or even plain wrong). Currently, this raises a KeyError, whereas using an unknown pytype defaults to StringElement. 5. Restructure pytype<-->XML Schema type mapping a bit, as e.g XML Schema type integer fits better to a Python long than a Python int regarding value space. a) I propose the following for non-fractional: pytype = PyType('int', int, IntElement) pytype.xmlSchemaTypes = ("int", "short", "byte", "unsignedShort", "unsignedByte",) pytype.register() pytype = PyType('long', long, LongElement) pytype.xmlSchemaTypes = ("integer", "nonPositiveInteger", "negativeInteger", "long", "nonNegativeInteger", "unsignedLong", "unsignedInt", "positiveInteger",) pytype.register() (Anything that fits in 32bit becomes a Python int, everything else a Python long. Maybe slightly arbitrary, but ok for 32bit-machines :-) This does not have big implications in practice, it's more or less for consistency. One thing remains: xsiannotate()-ing an IntElement >=2**31 will still xsi:type that as "int", which is not really valid regarding schema types. This could be addressed by using a more elaborate type_check for PyType("int",...) but I'm unsure about performance drawback and if it's worth the effort. b) Add all (non-list) XML Schema datatypes that restrict "string" to PyType('str', ...) As StringElement is the default these end up in StringElement anyway today. Adding them can result in faster lookup as no type-guessing will be invoked, and just for completeness... It also does not hurt s.o. who defines some custom class that handles a special schema datatype as this will override the objectify default. S.th. along the lines of pytype = PyType('str', None, StringElement) pytype.xmlSchemaTypes = ("string", "normalizedString", "token", "language", "Name", "NCName", "ID", "IDREF", "ENTITY", "NMTOKEN", ) What do you say? I've attached the patch/doc/tests for the proposed behaviour, for discussion, based on trunk versions of 2007/04/03. Holger -- "Feel free" - 10 GB Mailbox, 100 FreeSMS/Monat ... Jetzt GMX TopMail testen: http://www.gmx.net/de/go/topmail

2 2

[lxml-dev] ObjectPath for "current node"
by Christian Zagrodnick April 10, 2007

April 10, 2007

Hoi, in the object paths can be relative, like '.foo.bar'. Shouldn't it be possible then to create the object path of '.' referencing the current node? There would be a more general way to use path.find and path.setattr then. At least for me anyway :) -- Christian Zagrodnick gocept gmbh & co. kg · forsterstrasse 29 · 06112 halle/saale www.gocept.com · fon. +49 345 12298894 · fax. +49 345 12298891

2 2

Re: [lxml-dev] Document is not valid XML Schema
by jholg＠gmx.de April 7, 2007

April 7, 2007

Hi, one thing I've noticed - they seem to have different schema notions for each platform & version, and some seem to use Schematron (I'm not familiar with that): Complete Schema - has all documentation embedded and the Schematron mark-up. Minimal Schema - includes the raw xml schema only. Maybe you used a "Complete Schema" and should rather use a "Minimal schema" for lxml, which has W3C XML Schema and RelaxNG support? FWIW, Holger -- "Feel free" - 10 GB Mailbox, 100 FreeSMS/Monat ... Jetzt GMX TopMail testen: http://www.gmx.net/de/go/topmail

2 1