Mailman 3 February 2008 - lxml - The Python XML Toolkit

Re: [lxml-dev] c14n, pretty printing and diffing
by Olivier Collioud 12 Feb '08

12 Feb '08

Thanks Stephan. I prefer visual diffing : the ones provided by Eclipse, TkDiff or WinMerge. I did not fin any doc or usage example of lxml.usedoctest, could you please give some pointer ? Let me share my simple (because I do not use any namespace, PI, comment...) solution based on iterparse: depth = 0 sourceTree = ElementTree.iterparse(open(inputFile, 'r'), events=("start", "end")) for event, elem in sourceTree: if event == "start": i = "\n" + depth*" " depth += 1 outputFile.write('%s<%s' % (i,elem.tag)) if len(elem.items()): attrs = elem.items() attrs.sort() outputFile.write(' ') outputFile.write(' '.join(['%s="%s"' % (a[0],a[1]) for a in attrs if a[0] != 'size'])) if elem.text and elem.text.strip(): outputFile.write('>%s' % elem.text.strip('\n').encode('utf-8')) elif len(elem): outputFile.write('>') if event == "end": if (elem.text and elem.text.strip()) or len(elem): outputFile.write('%s</%s>' % (i,elem.tag)) else: outputFile.write('/>') if elem.tail and elem.tail.strip(): outputFile.write(elem.tail.strip('\n').encode('utf-8')) depth -= 1 elem.clear() Olivier. >>> Stefan Behnel <stefan_ml(a)behnel.de> 11/02/08 7:56 pm >>> Hi, Olivier Collioud wrote: > I would like to use my favourite text diffing tool to compare XML > files. Which is not lxml.html.diff, I assume? (I'm not sure how HTML specific that is, BTW). Also, for doctests, there is lxml.usedoctest that you can import (the lxml web pages use it for doctests). > Is their a way to produce a pretty printed canonical version of my XML > files using lxml ? Not using the c14n interface (libxml2 doesn't support it). Serialising by hand is not too hard, though. You can look at ElementTree._write() for an example: http://svn.effbot.org/public/elementtree/elementtree/ElementTree.py Stefan _______________________________________________ lxml-dev mailing list lxml-dev(a)codespeak.net http://codespeak.net/mailman/listinfo/lxml-dev ------ World Intellectual Property Organization Disclaimer: This electronic message may contain privileged, confidential and copyright protected information. If you have received this e-mail by mistake, please immediately notify the sender and delete this e-mail and all its attachments. Please ensure all e-mail attachments are scanned for viruses prior to opening or using.

1 0

[lxml-dev] Bug: type annotation namespace-prefix goes missing
by Christian Zagrodnick 11 Feb '08

11 Feb '08

Hi there, given the following little script: import lxml.etree import lxml.objectify xml = lxml.objectify.XML( '<a xmlns:py="http://codespeak.net/lxml/objectify/pytype">' '<title py:pytype="str">tit</title><text>foo</text><text>bar</text></a>') foo = xml['b'][0] bar = xml['b'][1] foo['text'] = 'FOO!' print lxml.etree.tostring(xml, pretty_print=True) foo = xml.xpath('b[@name="b1"]')[0] bar = xml.xpath('b[@name="b2"]')[0] xml['b'] = [bar, foo] print lxml.etree.tostring(xml, pretty_print=True) The first print statement prints, where everything is fine: <a xmlns:py="http://codespeak.net/lxml/objectify/pytype"> <title py:pytype="str">tit</title> <text py:pytype="str">FOO!</text> <text>bar</text> </a> The second prints: <a xmlns:py="http://codespeak.net/lxml/objectify/pytype"> <text>bar</text> <title py:pytype="str">tit</title> <text pytype="str">FOO!</text> </a> Note the last <text node where it says pytype instead of py:pytype. I think is has something todo with the xpath, but I cannot really figure out what. That's with lxml 2.0 and python 2.4 -- Christian Zagrodnick gocept gmbh & co. kg · forsterstrasse 29 · 06112 halle/saale www.gocept.com · fon. +49 345 12298894 · fax. +49 345 12298891

3 16

[lxml-dev] zif.sedna
by Jim Washington 07 Feb '08

07 Feb '08

Hi, all The Zif Collective has released to the cheese shop a python adapter to Sedna, an open source "native" XML database. Sedna is available at http://modis.ispras.ru/sedna/ in binary and source form under an Apache 2.0 license. Sedna stores XML in documents and collections, and can support multiple "databases" of related documents and collections. Sedna has many of the features of relational databases, but instead of SQL it uses XQuery. Sedna extends XQuery to support updates, inserts, deletes, etc. Sedna supports ACID transactions. zif.sedna provides communication with a Sedna server in python. It also provides a dbapi-like interface (connections and cursors). XQuery can be cool. Getting your results in object form can be as simple as parsing in lxml.objectify. Enjoy! -Jim Washington

1 0

[lxml-dev] requesting lxml testimonials?
by Martijn Faassen 05 Feb '08

05 Feb '08

Hi there, With Stefan, I really like reading lxml testimonials. Though lxml is now in many ways far more his than mine, it's still also my baby and I'm a happy user and quite proud. My testimonial won't count for much though. :) It looked like Stefan was interested in collecting testimonials to link to from the lxml pages. Perhaps we can expand this and get some quotes to put on the website. I think that would be a good idea - lxml should be considered by anyone who needs a good XML library with Python, and a few happy quotes by users might make them feel more confident about it. What do you think about making this a little project, Stefan? Anyone else wants to give a testimonial, together with permission for its use? Regards, Martijn

7 6

[lxml-dev] build & performance issues with 2.0beta2
by jholg＠gmx.de 04 Feb '08

04 Feb '08

Hi there, sorry to interrupt the testimonial celebrations ;-) but I'm having some problems with 2.0beta2: First of all, it does not build any more using gcc 2.95.2, yes I know, might old compiler...then again, Cython produces C code, not funky C++ stuff (2.0alpha-r47832 still built without problems). This is the error I get: $ /apps/pydev/hjoukl/bin/python2.4 setup.py build Building with Cython 0.9.6.11b. Building lxml version 2.0.beta2-51091. running build [...] cythoning src/lxml/lxml.etree.pyx to src/lxml/lxml.etree.c building 'lxml.etree' extension creating build/temp.solaris-2.8-sun4u-2.4 creating build/temp.solaris-2.8-sun4u-2.4/src creating build/temp.solaris-2.8-sun4u-2.4/src/lxml gcc -fno-strict-aliasing -DNDEBUG -g -O3 -Wall -Wstrict-prototypes -fPIC -I/apps/prod//include -I/apps/prod//include/libxml2 -I/apps/prod/include/libxml2 -I/apps/prod/include -I/apps/pydev/hjoukl/include/python2.4 -c src/lxml/lxml.etree.c -o build/temp.solaris-2.8-sun4u-2.4/src/lxml/lxml.etree.o -w src/lxml/lxml.etree.c: In function `__pyx_PyInt_AsLongLong': src/lxml/lxml.etree.c:110165: parse error before `long' src/lxml/lxml.etree.c:110167: `val' undeclared (first use in this function) src/lxml/lxml.etree.c:110167: (Each undeclared identifier is reported only once src/lxml/lxml.etree.c:110167: for each function it appears in.) src/lxml/lxml.etree.c: In function `__pyx_PyInt_AsUnsignedLongLong': src/lxml/lxml.etree.c:110185: parse error before `long' src/lxml/lxml.etree.c:110187: `val' undeclared (first use in this function) error: command 'gcc' failed with exit status 1 1 pytaf@adevp02 .../lxml-2.0beta2 $ Now, when I switch to use gcc 3.4.4 I can build successfully, but: 0 lb54320@adevp02 .../lxml-2.0beta2 $ LD_LIBRARY_PATH=/apps/prod/gcc/3.4.4/lib python2.4 test.py -p -v '' '!test_schematron_invalid*' /data/pydev/DOWNLOADS/LXML/lxml/versions/SVN_CHECKOUTS/TAGS/lxml-2.0beta2/src/lxml/html/__init__.py:22: UserWarning: This version of libxml2 has a known XPath bug. Use it at your own risk. _rel_links_xpath = etree.XPath("descendant-or-self::a[@rel]") TESTED VERSION: 2.0.beta2-51091 Python: (2, 4, 4, 'final', 0) lxml.etree: (2, 0, -98, 51091) libxml used: (2, 6, 27) libxml compiled: (2, 6, 27) libxslt used: (1, 1, 20) libxslt compiled: (1, 1, 20) 855/855 (100.0%): Doctest: xpathxslt.txt ====================================================================== FAIL: Doctest: validation.txt ---------------------------------------------------------------------- Traceback (most recent call last): File "/apps/prod//lib/python2.4/unittest.py", line 260, in run testMethod() File "/apps/prod//lib/python2.4/doctest.py", line 2157, in runTest raise self.failureException(self.format_failure(new.getvalue())) AssertionError: Failed doctest test for validation.txt File "/data/pydev/DOWNLOADS/LXML/lxml/versions/SVN_CHECKOUTS/TAGS/lxml-2.0beta2/src/lxml/tests/../../../doc/validation.txt", line 0 ---------------------------------------------------------------------- File "/data/pydev/DOWNLOADS/LXML/lxml/versions/SVN_CHECKOUTS/TAGS/lxml-2.0beta2/src/lxml/tests/../../../doc/validation.txt", line 113, in validation.txt Failed example: dtd = etree.DTD(external_id = docbook) # requires catalog support Exception raised: Traceback (most recent call last): File "/apps/prod//lib/python2.4/doctest.py", line 1248, in __run compileflags, 1) in test.globs File "<doctest validation.txt[16]>", line 1, in ? dtd = etree.DTD(external_id = docbook) # requires catalog support File "dtd.pxi", line 50, in lxml.etree.DTD.__init__ DTDParseError: failed to load external entity "-//OASIS//DTD DocBook XML V4.2//EN" ---------------------------------------------------------------------- File "/data/pydev/DOWNLOADS/LXML/lxml/versions/SVN_CHECKOUTS/TAGS/lxml-2.0beta2/src/lxml/tests/../../../doc/validation.txt", line 116, in validation.txt Failed example: dtd.assertValid(root) # doctest: +ELLIPSIS Expected: Traceback (most recent call last): DocumentInvalid: Element article content does not follow the DTD, ... Got: Traceback (most recent call last): File "/apps/prod//lib/python2.4/doctest.py", line 1248, in __run compileflags, 1) in test.globs File "<doctest validation.txt[18]>", line 1, in ? dtd.assertValid(root) # doctest: +ELLIPSIS File "lxml.etree.pyx", line 2375, in lxml.etree._Validator.assertValid DocumentInvalid: No declaration for element article, line 1 ====================================================================== FAIL: Doctest: validation.txt ---------------------------------------------------------------------- Traceback (most recent call last): File "/apps/prod//lib/python2.4/unittest.py", line 260, in run testMethod() File "/apps/prod//lib/python2.4/doctest.py", line 2157, in runTest raise self.failureException(self.format_failure(new.getvalue())) AssertionError: Failed doctest test for validation.txt File "/data/pydev/DOWNLOADS/LXML/lxml/versions/SVN_CHECKOUTS/TAGS/lxml-2.0beta2/src/lxml/tests/../../../doc/validation.txt", line 0 ---------------------------------------------------------------------- File "/data/pydev/DOWNLOADS/LXML/lxml/versions/SVN_CHECKOUTS/TAGS/lxml-2.0beta2/src/lxml/tests/../../../doc/validation.txt", line 113, in validation.txt Failed example: dtd = etree.DTD(external_id = docbook) # requires catalog support Exception raised: Traceback (most recent call last): File "/apps/prod//lib/python2.4/doctest.py", line 1248, in __run compileflags, 1) in test.globs File "<doctest validation.txt[16]>", line 1, in ? dtd = etree.DTD(external_id = docbook) # requires catalog support File "dtd.pxi", line 50, in lxml.etree.DTD.__init__ DTDParseError: failed to load external entity "-//OASIS//DTD DocBook XML V4.2//EN" ---------------------------------------------------------------------- File "/data/pydev/DOWNLOADS/LXML/lxml/versions/SVN_CHECKOUTS/TAGS/lxml-2.0beta2/src/lxml/tests/../../../doc/validation.txt", line 116, in validation.txt Failed example: dtd.assertValid(root) # doctest: +ELLIPSIS Expected: Traceback (most recent call last): DocumentInvalid: Element article content does not follow the DTD, ... Got: Traceback (most recent call last): File "/apps/prod//lib/python2.4/doctest.py", line 1248, in __run compileflags, 1) in test.globs File "<doctest validation.txt[18]>", line 1, in ? dtd.assertValid(root) # doctest: +ELLIPSIS File "lxml.etree.pyx", line 2375, in lxml.etree._Validator.assertValid DocumentInvalid: No declaration for element article, line 1 ====================================================================== FAIL: Doctest: validation.txt ---------------------------------------------------------------------- Traceback (most recent call last): File "/apps/prod//lib/python2.4/unittest.py", line 260, in run testMethod() File "/apps/prod//lib/python2.4/doctest.py", line 2157, in runTest raise self.failureException(self.format_failure(new.getvalue())) AssertionError: Failed doctest test for validation.txt File "/data/pydev/DOWNLOADS/LXML/lxml/versions/SVN_CHECKOUTS/TAGS/lxml-2.0beta2/src/lxml/tests/../../../doc/validation.txt", line 0 ---------------------------------------------------------------------- File "/data/pydev/DOWNLOADS/LXML/lxml/versions/SVN_CHECKOUTS/TAGS/lxml-2.0beta2/src/lxml/tests/../../../doc/validation.txt", line 113, in validation.txt Failed example: dtd = etree.DTD(external_id = docbook) # requires catalog support Exception raised: Traceback (most recent call last): File "/apps/prod//lib/python2.4/doctest.py", line 1248, in __run compileflags, 1) in test.globs File "<doctest validation.txt[16]>", line 1, in ? dtd = etree.DTD(external_id = docbook) # requires catalog support File "dtd.pxi", line 50, in lxml.etree.DTD.__init__ DTDParseError: failed to load external entity "-//OASIS//DTD DocBook XML V4.2//EN" ---------------------------------------------------------------------- File "/data/pydev/DOWNLOADS/LXML/lxml/versions/SVN_CHECKOUTS/TAGS/lxml-2.0beta2/src/lxml/tests/../../../doc/validation.txt", line 116, in validation.txt Failed example: dtd.assertValid(root) # doctest: +ELLIPSIS Expected: Traceback (most recent call last): DocumentInvalid: Element article content does not follow the DTD, ... Got: Traceback (most recent call last): File "/apps/prod//lib/python2.4/doctest.py", line 1248, in __run compileflags, 1) in test.globs File "<doctest validation.txt[18]>", line 1, in ? dtd.assertValid(root) # doctest: +ELLIPSIS File "lxml.etree.pyx", line 2375, in lxml.etree._Validator.assertValid DocumentInvalid: No declaration for element article, line 1 ====================================================================== FAIL: Doctest: validation.txt ---------------------------------------------------------------------- Traceback (most recent call last): File "/apps/prod//lib/python2.4/unittest.py", line 260, in run testMethod() File "/apps/prod//lib/python2.4/doctest.py", line 2157, in runTest raise self.failureException(self.format_failure(new.getvalue())) AssertionError: Failed doctest test for validation.txt File "/data/pydev/DOWNLOADS/LXML/lxml/versions/SVN_CHECKOUTS/TAGS/lxml-2.0beta2/src/lxml/tests/../../../doc/validation.txt", line 0 ---------------------------------------------------------------------- File "/data/pydev/DOWNLOADS/LXML/lxml/versions/SVN_CHECKOUTS/TAGS/lxml-2.0beta2/src/lxml/tests/../../../doc/validation.txt", line 113, in validation.txt Failed example: dtd = etree.DTD(external_id = docbook) # requires catalog support Exception raised: Traceback (most recent call last): File "/apps/prod//lib/python2.4/doctest.py", line 1248, in __run compileflags, 1) in test.globs File "<doctest validation.txt[16]>", line 1, in ? dtd = etree.DTD(external_id = docbook) # requires catalog support File "dtd.pxi", line 50, in lxml.etree.DTD.__init__ DTDParseError: failed to load external entity "-//OASIS//DTD DocBook XML V4.2//EN" ---------------------------------------------------------------------- File "/data/pydev/DOWNLOADS/LXML/lxml/versions/SVN_CHECKOUTS/TAGS/lxml-2.0beta2/src/lxml/tests/../../../doc/validation.txt", line 116, in validation.txt Failed example: dtd.assertValid(root) # doctest: +ELLIPSIS Expected: Traceback (most recent call last): DocumentInvalid: Element article content does not follow the DTD, ... Got: Traceback (most recent call last): File "/apps/prod//lib/python2.4/doctest.py", line 1248, in __run compileflags, 1) in test.globs File "<doctest validation.txt[18]>", line 1, in ? dtd.assertValid(root) # doctest: +ELLIPSIS File "lxml.etree.pyx", line 2375, in lxml.etree._Validator.assertValid DocumentInvalid: No declaration for element article, line 1 ---------------------------------------------------------------------- Ran 855 tests in 37.860s FAILED (failures=4) 1 lb54320@adevp02 .../lxml-2.0beta2 $ Compared to 2.0alpha (I rebuilt that also with gcc 3.4.4): 2 pytaf@adevp02 .../current $ LD_LIBRARY_PATH=/apps/prod/gcc/3.4.4/lib python2.4 test.py -p -v '' '!test_schematron_invalid*' /data/pydev/DOWNLOADS/LXML/lxml/versions/lxml-2.0alpha-r47832/src/lxml/html/__init__.py:22: UserWarning: This version of libxml2 has a known XPath bug. Use it at your own risk. _rel_links_xpath = etree.XPath("descendant-or-self::a[@rel]") TESTED VERSION: 2.0.alpha4 Python: (2, 4, 4, 'final', 0) lxml.etree: (2, 0, -196, 0) libxml used: (2, 6, 27) libxml compiled: (2, 6, 27) libxslt used: (1, 1, 20) libxslt compiled: (1, 1, 20) 824/824 (100.0%): Doctest: xpathxslt.txt ---------------------------------------------------------------------- Ran 824 tests in 2.698s OK So basically performance drops by factor >10 for me, on a Sparc Solaris 8 box, python2.4, gcc 3.4.4. I haven't yet looked into the failing tests. Remarks: I currently disable the schematron tests because some of them dump core with my setup. Holger -- GMX FreeMail: 1 GB Postfach, 5 E-Mail-Adressen, 10 Free SMS. Alle Infos und kostenlose Anmeldung: http://www.gmx.net/de/go/freemail

2 10

Re: [lxml-dev] Build problems under MacOS-X
by Stefan Behnel 03 Feb '08

03 Feb '08

Hi, js wrote: > seg fault occured to me, too. > Repeating etree.parse causes that. > > Here's the crash.log > Command: python2.5 > Path: /opt/local/bin/python2.5 > Parent: zsh [12091] > > Version: ??? (???) > > PID: 12274 > Thread: 0 > > Exception: EXC_BAD_ACCESS (0x0001) > Codes: KERN_INVALID_ADDRESS (0x0001) at 0x6f632e6f > > Thread 0 Crashed: > 0 libxml2.2.dylib 0x01371fc0 xmlDictFree + 45 > 1 libxml2.2.dylib 0x0137200e xmlDictFree + 123 > 2 etree.so 0x010634ad > __pyx_f_4lxml_5etree_24_ParserDictionaryContext_initThreadDictRef + 63 > (lxml.etree.c:45919) > .... Have you set DYLD... to the directory where the new lib versions are installed when running this? (compile time is *not* enough) If you did, could you try passing the "--without-threading" option to setup.py and rebuild with that to see if the problem persists? Stefan

1 0

[lxml-dev] lxml 2.0 is born!
by Stefan Behnel 01 Feb '08

01 Feb '08

Hi everyone, I'm very happy to announce the official release of lxml 2.0! http://codespeak.net/lxml/ http://pypi.python.org/pypi/lxml/2.0 This release marks the end of a development effort of more than 6 months, starting with the release of the last stable series lxml 1.3. The major differences are explained on this page: http://codespeak.net/lxml/lxml2.html lxml 2.0 is not a revolution, it is a gradual move towards a cleaner API with more things working together as expected. But it nevertheless comes with a lot of new tools and features, that makes your XML life easier - and even more your HTML life. There are also a couple of minor things that were deprecated, which will be removed for lxml 2.1. See the above link for details. The new release has already adopted a lot of changes from the upcoming ElementTree 1.3 library, and implements a much broader set of compatible features, such as the TreeBuilder interface for parser targets. I appended the complete changelog, but lets start with the most important things: * * * * * * * * * * * * * * * ( ) (*) (*) * | | |~| |~| * | | | | | | ,,.......,, | | ,.a@@@@| |@@@@@@@@@@@@@@@@| |@@@@a. .,a@@@@@@@@@| |@@@@@@@@@@@@@@@@| |@@@@@@@@a,. ,a@@@@@@@@@@@@| |@@@@@@.@@@@@@@@@| |@@@@@@@@@@@@a, a@@@@@@@@@@@@@@@@@@@@@' . `@@@@@@@@@@@@@@@@@@@@@@@@a ;`@@@@@@@@@@@@@@@@@@' . `@@@@@@@@@@@@@@@@@@@@@'; ;@@@`@@@@@@@@@@@@@' . `@@@@@@@@@@@@@@@@'@@@; ;@@@;,.aaaaaaaaaa . aaaaa,,aaaaaaa,;@@@; ;;@;;;;@@@@@@@@;@ @.@ ;@@@;;;@@@@@@;;;;@@; ;;;;;;;@@@@;@@;;@ @@ . @@ ;;@;;;;@@;@@@;;;;;;; ;;;;;;;;@@;;;;;;; @@ . @@ ;;;;;;;;;;;@@;;;;@;; ;;;;;;;;;;;;;;;;;@@ . @@;;;;;;;;;;;;;;;;@@@; ,%%%;;;;;;;;@;;;;;;;; . ;;;;;;;;;;;;;;;;@@;;%%%, .%%%%%%;;;;;;;@@;;;;;;;; ,%%%, ;;;;;;;;;;;;;;;;;;;;%%%%%%, .%%%%%%%;;;;;;;@@;;;;;;;; ,%%%%%%%, ;;;;;;;;;;;;;;;;;;;;%%%%%%%, %%%%%%%%`;;;;;;;;;;;;;;;; %%%%%%%%%%% ;;;;;;;;;;;;;;;;;;;'%%%%%%%% %%%%%%%%%%%%`;;;;;;;;;;;;,%%%%%%%%%%%%%,;;;;;;;;;;;;;;;'%%%%%%%%%%%% `%%%%%%%%%%%%%%%%%,,,,,,,%%%%%%%%%%%%%%%,,,,,,,%%%%%%%%%%%%%%%%%%%%' `%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%' `%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%' """"""""""""""`,,,,,,,,,'""""""""""""""""" `%%%%%%%' `%%%%%' %%% generously donated by Susie Oviatt %%%%% .,%%%%%%%,. ,%%%%%%%%%%%%%%%%%%%, Happy birthday, lxml - and may the force be with you! :) Have fun, Stefan ** ChangeLog: 2.0 (2008-02-01) ================ Features added -------------- * Passing the ``unicode`` type as ``encoding`` to ``tostring()`` will serialise to unicode. The ``tounicode()`` function is now officially deprecated. * ``XMLSchema()`` and ``RelaxNG()`` can parse from StringIO. * ``makeparser()`` function in ``lxml.objectify`` to create a new parser with the usual objectify setup. Bugs fixed ---------- Other changes ------------- 2.0beta2 (2008-01-26) ===================== Features added -------------- * Plain ASCII XPath string results are no longer forced into unicode objects as in 2.0beta1, but are returned as plain strings as before. * All XPath string results are 'smart' objects that have a ``getparent()`` method to retrieve their parent Element. * ``with_tail`` option in serialiser functions. * More accurate exception messages in validator creation. Bugs fixed ---------- * Missing import in ``lxml.html.clean``. * Some Python 2.4-isms prevented lxml from building/running under Python 2.3. Other changes ------------- * Exceptions carry only the part of the error log that is related to the operation that caused the error. * ``XMLSchema()`` and ``RelaxNG()`` now enforce passing the source file/filename through the ``file`` keyword argument. * The test suite now skips most doctests under Python 2.3. * ``make clean`` no longer removes the .c files (use ``make realclean`` instead) 2.0beta1 (2008-01-11) ===================== Features added -------------- * Parse-time XML schema validation (``schema`` parser keyword). * XPath string results of the ``text()`` function and attribute selection make their Element container accessible through a ``getparent()`` method. As a side-effect, they are now always unicode objects (even ASCII strings). * ``XSLT`` objects are usable in any thread - at the cost of a deep copy if they were not created in that thread. * Invalid entity names and character references will be rejected by the ``Entity()`` factory. * ``entity.text`` returns the textual representation of the entity, e.g. ``&``. Bugs fixed ---------- * XPath on ElementTrees could crash when selecting the virtual root node of the ElementTree. * Compilation ``--without-threading`` was buggy in alpha5/6. Other changes ------------- * Minor performance tweaks for Element instantiation and subelement creation 2.0alpha6 (2007-12-19) ====================== Features added -------------- * New properties ``position`` and ``code`` on ParseError exception (as in ET 1.3) Bugs fixed ---------- * Memory leak in the ``parse()`` function. * Minor bugs in XSLT error message formatting. * Result document memory leak in target parser. Other changes ------------- * Various places in the XPath, XSLT and iteration APIs now require keyword-only arguments. * The argument order in ``element.itersiblings()`` was changed to match the order used in all other iteration methods. The second argument ('preceding') is now a keyword-only argument. * The ``getiterator()`` method on Elements and ElementTrees was reverted to return an iterator as it did in lxml 1.x. The ET API specification allows it to return either a sequence or an iterator, and it traditionally returned a sequence in ET and an iterator in lxml. However, it is now deprecated in favour of the ``iter()`` method, which should be used in new code wherever possible. * The 'pretty printed' serialisation of ElementTree objects now inserts newlines at the root level between processing instructions, comments and the root tag. * A 'pretty printed' serialisation is now terminated with a newline. * Second argument to ``lxml.etree.Extension()`` helper is no longer required, third argument is now a keyword-only argument ``ns``. * ``lxml.html.tostring`` takes an ``encoding`` argument. 2.0alpha5 (2007-11-24) ====================== Features added -------------- * Rich comparison of ``element.attrib`` proxies. * ElementTree compatible TreeBuilder class. * Use default prefixes for some common XML namespaces. * ``lxml.html.clean.Cleaner`` now allows for a ``host_whitelist``, and two overridable methods: ``allow_embedded_url(el, url)`` and the more general ``allow_element(el)``. * Extended slicing of Elements as in ``element[1:-1:2]``, both in etree and in objectify * Resolvers can now provide a ``base_url`` keyword argument when resolving a document as string data. * When using ``lxml.doctestcompare`` you can give the doctest option ``NOPARSE_MARKUP`` (like ``# doctest: +NOPARSE_MARKUP``) to suppress the special checking for one test. Bugs fixed ---------- * Target parser failed to report comments. * In the ``lxml.html`` ``iter_links`` method, links in ``<object>`` tags weren't recognized. (Note: plugin-specific link parameters still aren't recognized.) Also, the ``<embed>`` tag, though not standard, is now included in ``lxml.html.defs.special_inline_tags``. * Using custom resolvers on XSLT stylesheets parsed from a string could request ill-formed URLs. * With ``lxml.doctestcompare`` if you do ``<tag xmlns="...">`` in your output, it will then be namespace-neutral (before the ellipsis was treated as a real namespace). Other changes ------------- * The module source files were renamed to "lxml.*.pyx", such as "lxml.etree.pyx". This was changed for consistency with the way Pyrex commonly handles package imports. The main effect is that classes now know about their fully qualified class name, including the package name of their module. * Keyword-only arguments in some API functions, especially in the parsers and serialisers. 2.0alpha4 (2007-10-07) ====================== Features added -------------- Bugs fixed ---------- * AttributeError in feed parser on parse errors Other changes ------------- * Tag name validation in lxml.etree (and lxml.html) now distinguishes between HTML tags and XML tags based on the parser that was used to parse or create them. HTML tags no longer reject any non-ASCII characters in tag names but only spaces and the special characters ``<>&/"'``. 2.0alpha3 (2007-09-26) ====================== Features added -------------- * Separate ``feed_error_log`` property for the feed parser interface. The normal parser interface and ``iterparse`` continue to use ``error_log``. * The normal parsers and the feed parser interface are now separated and can be used concurrently on the same parser instance. * ``fromstringlist()`` and ``tostringlist()`` functions as in ElementTree 1.3 * ``iterparse()`` accepts an ``html`` boolean keyword argument for parsing with the HTML parser (note that this interface may be subject to change) * Parsers accept an ``encoding`` keyword argument that overrides the encoding of the parsed documents. * New C-API function ``hasChild()`` to test for children * ``annotate()`` function in objectify can annotate with Python types and XSI types in one step. Accompanied by ``xsiannotate()`` and ``pyannotate()``. Bugs fixed ---------- * XML feed parser setup problem * Type annotation for unicode strings in ``DataElement()`` Other changes ------------- * lxml.etree now emits a warning if you use XPath with libxml2 2.6.27 (which can crash on certain XPath errors) * Type annotation in objectify now preserves the already annotated type by default to prevent loosing type information that is already there. 2.0alpha2 (2007-09-15) ====================== Features added -------------- * ``ET.write()``, ``tostring()`` and ``tounicode()`` now accept a keyword argument ``method`` that can be one of 'xml' (or None), 'html' or 'text' to serialise as XML, HTML or plain text content. * ``iterfind()`` method on Elements returns an iterator equivalent to ``findall()`` * ``itertext()`` method on Elements * Setting a QName object as value of the .text property or as an attribute will resolve its prefix in the respective context * ElementTree-like parser target interface as described in http://effbot.org/elementtree/elementtree-xmlparser.htm * ElementTree-like feed parser interface on XMLParser and HTMLParser (``feed()`` and ``close()`` methods) Bugs fixed ---------- * lxml failed to serialise namespace declarations of elements other than the root node of a tree * Race condition in XSLT where the resolver context leaked between concurrent XSLT calls Other changes ------------- * ``element.getiterator()`` returns a list, use ``element.iter()`` to retrieve an iterator (ElementTree 1.3 compatible behaviour) 2.0alpha1 (2007-09-02) ====================== Features added -------------- * Reimplemented ``objectify.E`` for better performance and improved integration with objectify. Provides extended type support based on registered PyTypes. * XSLT objects now support deep copying * New ``makeSubElement()`` C-API function that allows creating a new subelement straight with text, tail and attributes. * XPath extension functions can now access the current context node (``context.context_node``) and use a context dictionary (``context.eval_context``) from the context provided in their first parameter * HTML tag soup parser based on BeautifulSoup in ``lxml.html.ElementSoup`` * New module ``lxml.doctestcompare`` by Ian Bicking for writing simplified doctests based on XML/HTML output. Use by importing ``lxml.usedoctest`` or ``lxml.html.usedoctest`` from within a doctest. * New module ``lxml.cssselect`` by Ian Bicking for selecting Elements with CSS selectors. * New package ``lxml.html`` written by Ian Bicking for advanced HTML treatment. * Namespace class setup is now local to the ``ElementNamespaceClassLookup`` instance and no longer global. * Schematron validation (incomplete in libxml2) * Additional ``stringify`` argument to ``objectify.PyType()`` takes a conversion function to strings to support setting text values from arbitrary types. * Entity support through an ``Entity`` factory and element classes. XML parsers now have a ``resolve_entities`` keyword argument that can be set to False to keep entities in the document. * ``column`` field on error log entries to accompany the ``line`` field * Error specific messages in XPath parsing and evaluation NOTE: for evaluation errors, you will now get an XPathEvalError instead of an XPathSyntaxError. To catch both, you can except on ``XPathError`` * The regular expression functions in XPath now support passing a node-set instead of a string * Extended type annotation in objectify: new ``xsiannotate()`` function * EXSLT RegExp support in standard XPath (not only XSLT) Bugs fixed ---------- * lxml.etree did not check tag/attribute names * The XML parser did not report undefined entities as error * The text in exceptions raised by XML parsers, validators and XPath evaluators now reports the first error that occurred instead of the last * Passing '' as XPath namespace prefix did not raise an error * Thread safety in XPath evaluators Other changes ------------- * objectify.PyType for None is now called "NoneType" * ``el.getiterator()`` renamed to ``el.iter()``, following ElementTree 1.3 - original name is still available as alias * In the public C-API, ``findOrBuildNodeNs()`` was replaced by the more generic ``findOrBuildNodeNsPrefix`` * Major refactoring in XPath/XSLT extension function code * Network access in parsers disabled by default

1 0

[lxml-dev] Compile error with trunk
by Ian Bicking 01 Feb '08

01 Feb '08

I tried rebuilding the trunk, installing cython and deleting the .c files I had around. I got this: ~/src/lxml$ python setup.py develop Building with Cython 0.9.6.11. Building lxml version 2.0.beta2-51162. running develop running egg_info writing src/lxml.egg-info/PKG-INFO writing top-level names to src/lxml.egg-info/top_level.txt writing dependency_links to src/lxml.egg-info/dependency_links.txt reading manifest template 'MANIFEST.in' warning: no files found matching 'lxml.objectify.c' under directory 'src/lxml' warning: no files found matching 'lxml.pyclasslookup.c' under directory 'src/lxml' warning: no files found matching '*.html' under directory 'doc' warning: no previously-included files found matching 'src/lxml/etree.pxi' writing manifest file 'src/lxml.egg-info/SOURCES.txt' running build_ext building 'lxml.etree' extension gcc -pthread -fno-strict-aliasing -DNDEBUG -g -O2 -Wall -Wstrict-prototypes -fPIC -I/usr/include/libxml2 -I/usr/include/python2.5 -c src/lxml/lxml.etree.c -o build/temp.linux-i686-2.5/src/lxml/lxml.etree.o -w src/lxml/lxml.etree.c: In function ‘__pyx_tp_clear_4lxml_5etree__BaseParser’: src/lxml/lxml.etree.c:91711: error: invalid lvalue in assignment src/lxml/lxml.etree.c:91714: error: invalid lvalue in assignment src/lxml/lxml.etree.c:91717: error: invalid lvalue in assignment src/lxml/lxml.etree.c:91720: error: invalid lvalue in assignment src/lxml/lxml.etree.c:91723: error: invalid lvalue in assignment src/lxml/lxml.etree.c: In function ‘__pyx_tp_clear_4lxml_5etree__Document’: src/lxml/lxml.etree.c:91930: error: invalid lvalue in assignment src/lxml/lxml.etree.c: In function ‘__pyx_tp_clear_4lxml_5etree_DocInfo’: src/lxml/lxml.etree.c:92086: error: invalid lvalue in assignment src/lxml/lxml.etree.c: In function ‘__pyx_tp_clear_4lxml_5etree__Element’: src/lxml/lxml.etree.c:92310: error: invalid lvalue in assignment src/lxml/lxml.etree.c: In function ‘__pyx_tp_clear_4lxml_5etree__ElementTree’: src/lxml/lxml.etree.c:93275: error: invalid lvalue in assignment src/lxml/lxml.etree.c:93278: error: invalid lvalue in assignment src/lxml/lxml.etree.c: In function ‘__pyx_tp_clear_4lxml_5etree__Attrib’: src/lxml/lxml.etree.c:93467: error: invalid lvalue in assignment src/lxml/lxml.etree.c: In function ‘__pyx_tp_clear_4lxml_5etree__AttribIterator’: src/lxml/lxml.etree.c:93653: error: invalid lvalue in assignment src/lxml/lxml.etree.c: In function ‘__pyx_tp_clear_4lxml_5etree__ElementIterator’: src/lxml/lxml.etree.c:93973: error: invalid lvalue in assignment src/lxml/lxml.etree.c: In function ‘__pyx_tp_clear_4lxml_5etree_ElementDepthFirstIterator’: src/lxml/lxml.etree.c:94539: error: invalid lvalue in assignment src/lxml/lxml.etree.c:94542: error: invalid lvalue in assignment src/lxml/lxml.etree.c: In function ‘__pyx_tp_clear_4lxml_5etree_ElementTextIterator’: src/lxml/lxml.etree.c:94708: error: invalid lvalue in assignment src/lxml/lxml.etree.c: In function ‘__pyx_tp_clear_4lxml_5etree__BaseErrorLog’: src/lxml/lxml.etree.c:95110: error: invalid lvalue in assignment src/lxml/lxml.etree.c: In function ‘__pyx_tp_clear_4lxml_5etree_FallbackElementClassLookup’: src/lxml/lxml.etree.c:96712: error: invalid lvalue in assignment src/lxml/lxml.etree.c: In function ‘__pyx_tp_clear_4lxml_5etree__ResolverRegistry’: src/lxml/lxml.etree.c:98662: error: invalid lvalue in assignment src/lxml/lxml.etree.c: In function ‘__pyx_tp_clear_4lxml_5etree__ResolverContext’: src/lxml/lxml.etree.c:98832: error: invalid lvalue in assignment src/lxml/lxml.etree.c:98835: error: invalid lvalue in assignment src/lxml/lxml.etree.c: In function ‘__pyx_tp_clear_4lxml_5etree__ParserDictionaryContext’: src/lxml/lxml.etree.c:99002: error: invalid lvalue in assignment src/lxml/lxml.etree.c: In function ‘__pyx_tp_clear_4lxml_5etree__FileReaderContext’: src/lxml/lxml.etree.c:99192: error: invalid lvalue in assignment src/lxml/lxml.etree.c: In function ‘__pyx_tp_clear_4lxml_5etree__ParserContext’: src/lxml/lxml.etree.c:99366: error: invalid lvalue in assignment src/lxml/lxml.etree.c:99369: error: invalid lvalue in assignment src/lxml/lxml.etree.c: In function ‘__pyx_tp_clear_4lxml_5etree__SaxParserContext’: src/lxml/lxml.etree.c:99529: error: invalid lvalue in assignment src/lxml/lxml.etree.c: In function ‘__pyx_tp_clear_4lxml_5etree__ParserSchemaValidationContext’: src/lxml/lxml.etree.c:99856: error: invalid lvalue in assignment src/lxml/lxml.etree.c: In function ‘__pyx_tp_clear_4lxml_5etree__Validator’: src/lxml/lxml.etree.c:100012: error: invalid lvalue in assignment src/lxml/lxml.etree.c: In function ‘__pyx_tp_clear_4lxml_5etree_TreeBuilder’: src/lxml/lxml.etree.c:100641: error: invalid lvalue in assignment src/lxml/lxml.etree.c:100656: error: invalid lvalue in assignment src/lxml/lxml.etree.c: In function ‘__pyx_tp_clear_4lxml_5etree__FilelikeWriter’: src/lxml/lxml.etree.c:101436: error: invalid lvalue in assignment src/lxml/lxml.etree.c:101439: error: invalid lvalue in assignment src/lxml/lxml.etree.c: In function ‘__pyx_tp_clear_4lxml_5etree__IterparseContext’: src/lxml/lxml.etree.c:101634: error: invalid lvalue in assignment src/lxml/lxml.etree.c:101637: error: invalid lvalue in assignment src/lxml/lxml.etree.c: In function ‘__pyx_tp_clear_4lxml_5etree__IDDict’: src/lxml/lxml.etree.c:102200: error: invalid lvalue in assignment src/lxml/lxml.etree.c: In function ‘__pyx_tp_clear_4lxml_5etree_XInclude’: src/lxml/lxml.etree.c:102381: error: invalid lvalue in assignment src/lxml/lxml.etree.c: In function ‘__pyx_tp_clear_4lxml_5etree__BaseContext’: src/lxml/lxml.etree.c:102750: error: invalid lvalue in assignment src/lxml/lxml.etree.c:102771: error: invalid lvalue in assignment src/lxml/lxml.etree.c:102774: error: invalid lvalue in assignment src/lxml/lxml.etree.c: In function ‘__pyx_tp_clear_4lxml_5etree__ElementUnicodeResult’: src/lxml/lxml.etree.c:102965: error: invalid lvalue in assignment src/lxml/lxml.etree.c: In function ‘__pyx_tp_clear_4lxml_5etree__XPathEvaluatorBase’: src/lxml/lxml.etree.c:103314: error: invalid lvalue in assignment src/lxml/lxml.etree.c:103317: error: invalid lvalue in assignment src/lxml/lxml.etree.c: In function ‘__pyx_tp_clear_4lxml_5etree_XPathElementEvaluator’: src/lxml/lxml.etree.c:103487: error: invalid lvalue in assignment src/lxml/lxml.etree.c: In function ‘__pyx_tp_clear_4lxml_5etree__XSLTResolverContext’: src/lxml/lxml.etree.c:104093: error: invalid lvalue in assignment src/lxml/lxml.etree.c: In function ‘__pyx_tp_clear_4lxml_5etree_XSLT’: src/lxml/lxml.etree.c:104554: error: invalid lvalue in assignment src/lxml/lxml.etree.c:104557: error: invalid lvalue in assignment src/lxml/lxml.etree.c:104560: error: invalid lvalue in assignment src/lxml/lxml.etree.c:104563: error: invalid lvalue in assignment src/lxml/lxml.etree.c: In function ‘__pyx_tp_clear_4lxml_5etree__XSLTResultTree’: src/lxml/lxml.etree.c:104742: error: invalid lvalue in assignment src/lxml/lxml.etree.c:104745: error: invalid lvalue in assignment error: command 'gcc' failed with exit status 1

2 1