Mailman 3 July 2007 - lxml - The Python XML Toolkit

[lxml-dev] lxml has its page on launchpad
by Stefan Behnel 11 Apr '23

11 Apr '23

Hi all, I added the lxml project to launchpad, the Ubuntu Bug-Tracker. It also has a FAQ engine and a couple of other goodies. https://launchpad.net/lxml It's easy to sign up for launchpad, BTW, no 90%-footnotes-contract. Have fun, Stefan

9 9

[lxml-dev] Building LXML Trunk
by Sidnei da Silva 31 Aug '22

31 Aug '22

Hi, I've tried to build lxml from trunk today, on Win32. Got the following error: src\lxml\etree.c(880) : error C2059: syntax error : ')' src\lxml\etree.c(881) : error C2059: syntax error : ')' src\lxml\etree.c(882) : error C2059: syntax error : ')' src\lxml\etree.c(883) : error C2059: syntax error : ')' Any clue? Smells like a Pyrex issue? -- Sidnei da Silva Enfold Systems http://enfoldsystems.com Fax +1 832 201 8856 Office +1 713 942 2377 Ext 214

4 4

[lxml-dev] Pickling objectified trees
by Christian Zagrodnick 09 Jun '08

09 Jun '08

Hi, the other day I had to pickle objectified trees. I just thought to share my findings. Pickling is about serialization. IMHO the natural serialization of an objectified tree is its XML representation. So the following basically does that: -------------------------- import copy_reg import lxml.etree import lxml.objectify def treeFactory(state): """Un-Pickle factory.""" return lxml.objectify.fromstring(state) copy_reg.constructor(treeFactory) def reduceObjectifiedElement(object): """Reduce function for lxml.objectify trees. See http://docs.python.org/lib/pickle-protocol.html for details. """ return (treeFactory, (lxml.etree.tostring(object), )) copy_reg.pickle(lxml.objectify.ObjectifiedElement, reduceObjectifiedElement, treeFactory) ----------------------------------------- You might consider just registering the reduce function in lxml itself. Shouldn't hurt, should it. -- Christian Zagrodnick gocept gmbh & co. kg · forsterstrasse 29 · 06112 halle/saale www.gocept.com · fon. +49 345 12298894 · fax. +49 345 12298891

2 4

[lxml-dev] lxml namespaces
by Maxim Sloyko 20 Feb '08

20 Feb '08

Hi All! I have a little problem with XML namespaces. In my application I have two XML processors, that process the same document, one after the other. The first one looks for nodes in 'ns1' namespace, and substitutes them, according to some algorithm. After this processor is finished, it is guaranteed that there are no more 'ns1' nodes left in the tree. 'ns1' namespace dclaration is still there, in the root node (well, I put it there manually). Now, when this namespace is no longer needed, I want to get rid of it, because it confuses some other processors (namely, my browser) So, the question is, how do I do that? del tree.getroot().nsmap['ns1'] does not seem to do the trick :( Thanks in advance and Happy Holidays! -- Maxim Sloyko

3 5

[lxml-dev] lxml 1.3.3 released
by Stefan Behnel 01 Aug '07

01 Aug '07

Hi all, I just released lxml 1.3.3 to cheeseshop. This is mainly a bug-fix release for the stable 1.3 series. Changelog follows. Have fun, Stefan 1.3.3 (2007-07-26) ================== Features added -------------- * ElementTree compatible parser ETCompatXMLParser strips processing instructions and comments while parsing XML * Parsers now support stripping PIs (keyword argument 'remove_pis') * etree.fromstring() now supports parsing both HTML and XML, depending on the parser you pass. * Support base_url keyword argument in HTML() and XML() Bugs fixed ---------- * Parsing from Python Unicode strings failed on some platforms * Element() did not raise an exception on tag names containing ':' * Element.getiterator(tag) did not accept Comment and ProcessingInstruction as tags. It also accepts Element now.

4 7

[lxml-dev] lxml 2.0 will require Cython instead of Pyrex
by Stefan Behnel 30 Jul '07

30 Jul '07

Hi all, the new Cython project has started its work on the original Pyrex source code. It already has several advantages over Pyrex, including support for list comprehension and type based optimisation. http://www.cython.org/ Cython includes the patches lxml originally needed in Pyrex. lxml 2.0 will therefore stop shipping with a modified Pyrex and simply require Cython if people want to build from non-release sources. Note that we will continue to ship releases with the generated .c files, so no Cython is required by people who just want to build from release sources. Have fun, Stefan

1 0

[lxml-dev] reloading a network resource fails
by jholg＠gmx.de 27 Jul '07

27 Jul '07

Hi, parsing a network resource multiple times fails in current trunk: *** 2.0.dev-45372 *** >>> etree.parse("http://adevp01:8080/validSummary-1.2.xml") <etree._ElementTree object at 0x2e4850> >>> etree.parse("http://adevp01:8080/validSummary-1.2.xml") Traceback (most recent call last): File "<stdin>", line 1, in ? File "etree.pyx", line 2065, in etree.parse File "parser.pxi", line 1004, in etree._parseDocument File "parser.pxi", line 1008, in etree._parseDocumentFromURL File "parser.pxi", line 925, in etree._parseDocFromFile File "parser.pxi", line 585, in etree._BaseParser._parseDocFromFile File "parser.pxi", line 682, in etree._handleParseResult File "parser.pxi", line 630, in etree._raiseParseError etree.XMLSyntaxError: Attempt to load network entity http://adevp01:8080/validSummary-1.2.xml >>> etree.parse("http://adevp01:8080/validSummary-1.2.xml") Traceback (most recent call last): File "<stdin>", line 1, in ? File "etree.pyx", line 2065, in etree.parse File "parser.pxi", line 1004, in etree._parseDocument File "parser.pxi", line 1008, in etree._parseDocumentFromURL File "parser.pxi", line 925, in etree._parseDocFromFile File "parser.pxi", line 585, in etree._BaseParser._parseDocFromFile File "parser.pxi", line 682, in etree._handleParseResult File "parser.pxi", line 630, in etree._raiseParseError etree.XMLSyntaxError: Attempt to load network entity http://adevp01:8080/validSummary-1.2.xml >>> etree.parse("http://adevp01:8080/validSummary-1.2.xml") Traceback (most recent call last): File "<stdin>", line 1, in ? File "etree.pyx", line 2065, in etree.parse File "parser.pxi", line 1004, in etree._parseDocument File "parser.pxi", line 1008, in etree._parseDocumentFromURL File "parser.pxi", line 925, in etree._parseDocFromFile File "parser.pxi", line 585, in etree._BaseParser._parseDocFromFile File "parser.pxi", line 682, in etree._handleParseResult File "parser.pxi", line 630, in etree._raiseParseError etree.XMLSyntaxError: Attempt to load network entity http://adevp01:8080/validSummary-1.2.xml >>> I rather often use local files, so I ran into this by accident. 1.3 does not have the same problem (same libxml2 version). I also reported this one on the new launchpad bugtracker. Can this be set up to let one choose the version, or should we introduce a convention on naming the summary line, e.g. [trunk], [1.3]? Holger -- Der GMX SmartSurfer hilft bis zu 70% Ihrer Onlinekosten zu sparen! Ideal für Modem und ISDN: http://www.gmx.net/de/go/smartsurfer

2 1

[lxml-dev] [objectify] Typed E-factory for objectify, PT DataElement()-wrapper
by jholg＠gmx.de 26 Jul '07

26 Jul '07

Hi, attached patch (against trunk) * adds a typed E-factory (called T-factory) * inserts NoneType into the E-factory/T-factory typemap * adds the PT() (="PyTyped(<arg>)) convenience function that is a thin wrapper uses the argument value's type to set the pytype * provides unittests for E-factory, T-factory and PT() * fixes DataElement() to care for some previously-unhandled corner cases concerning None and/or _pytype "none" Despite of what I previously said ;-) I now think it would be better to rename "none" to "NoneType", to use the same name as the Python builtin original. While it is a longer name I seriously doubt you need to actually use it explicitly very often. By convention, the PyType name should match the Python builtin type name; then both the T-factory and the PT() function can work smoothly (the only thing special-cased is the Python type name "unicode" with gets substituted by "str"). Therefore, the patch also changes "none" to "NoneType" in objectify and the objectify tests/doctests. I'd really like to see the PT() function go into the 1.3 series, too. Please take a look, I can come up with some documentation if you like it. Holger -- Der GMX SmartSurfer hilft bis zu 70% Ihrer Onlinekosten zu sparen! Ideal für Modem und ISDN: http://www.gmx.net/de/go/smartsurfer

2 5

[lxml-dev] Strange behaviour with namespaces
by Albert Brandl 23 Jul '07

23 Jul '07

Hi! I'd like to serialize a part of an XML document for later retrieval. Since the elements are defined using namespaces, I created an ElementTree instance for the element to be serialized and called its write_c14n method. >>> from import etree >>> import cStringIO >>> XML=""" ... <n1:a xmlns:a="http://a.org" xmlns:b="http://b.org" xmlns:c="http://c.org"> ... <n2:b> ... <n3:c/> ... </n2:b> ... </n1:a>""" >>> e = etree.fromstring(XML) >>> et = etree.ElementTree(e[0]) >>> sb = cStringIO.StringIO() >>> et.write_c14n(sb) >>> sb.getvalue() '<b xmlns:a="http://a.org" xmlns:b="http://b.org" xmlns:c="http://c.org">\n <c></c>\n </b>' The xmlns information is transported correctly, but the information about the namespaces for the elements is lost. I assume this is a bug. Is there another way to extract a part of the document in textual form such that namespace information is preserved? Using tostring does not work, since this method throws away the xmlns attributes altogether. Thanks & best regards, Albert Brandl

2 2

[lxml-dev] lxml & parsing: return of a classes
by Ian Bicking 22 Jul '07

22 Jul '07

So I was thinking a little about how we could allow easy customization of the URL getter, since we can't attach it to the tree or any element. And then generally how any customization could be done, for instance if you want a new method on all elements. This isn't that easy currently. You'd have to subclass a bunch of classes and rewrite a bunch of functions. But I think if we move all parsing to a single class it would help a great deal. The idea is something like: class Parser(object): _etree_parser_class = etree.HTMLParser def __init__(self): self._etree_parser = self._etree_parser_class() self._etree_parser.setElementClassLookup(self) def __call__(self, filename, **kw): return etree.parse(filename, self._etree_parser, **kw) def fromstring(...): ... And so forth. Then either expose this via: parse = Parser() Or perhaps: _parser = Parser() parse = _parser fromstring = _parser.fromstring And so forth. If you want to adjust something, you don't have to reimplement all the forms of parsers, since they all would just use self, and are mostly defined in terms of each other. We could support subclassing with something like this: class Parser(object): _element_classes = {} _element_mixins = {} def __init__(self): self._element_classes = self._element_classes.copy() mixers = {} for name, value in _element_mixins: if name == '*': for n in self._element_classes.keys(): mixers.setdefault(n, []).append(value) else: mixers.setdefault(name, []).append(value) for name, mixins in mixers: cur = self._element_classes.get(name, HtmlElement) bases = mixins + [cur] new_class = type(cur.__name__, tuple(bases), {}) self._element_classes[name] = new_class class MyMixin(object): extra methods class FormMixin(object): other methods for the form element class ParserMixedIn(Parser): _element_mixins = {'*': MyMixin, 'form': FormMixin} And then it would be really easy to create local extensions for all HTML elements, or particular elements. I'm not sure exactly how to attach the URL getting method to the Parser object in this model, because I'm not sure how to give elements a reference back to it. We could do it with class variables, but then the parser would *have* to subclass every element everytime it was instantiated, so it could make new classes with a reference back to itself. But maybe there's a better way. Do the elements already have a reference back to that etree.HTMLParser() instance, and could we attach this to that instance? Or perhaps extend HTMLParser directly instead of having this other parser class? -- Ian Bicking : ianb(a)colorstudy.com : http://blog.ianbicking.org : Write code, do good : http://topp.openplans.org/careers

2 4