Mailman 3 June 2006 - lxml - The Python XML Toolkit

[lxml-dev] make html doesn't work due to rest2html beeing renamed to rst2html
by Andreas Pakulat 08 Jun '06

08 Jun '06

Hi, I just executed make html and got: andreas@morpheus:~/compiling/python/xml/lxml>make html mkdir -p doc/html python doc/mkhtml.py doc/html . `cat version.txt` sh: rest2html: command not found :-( I guess docutils renamed rest2html to rst2html some time ago, at least my docutils package version 0.4 uses rst2html. Of course I have no problem renaming rest2html in mkhtml.py myself, others might though. Andreas -- You'll feel devilish tonight. Toss dynamite caps under a flamenco dancer's heel.

2 1

[lxml-dev] Improved docstrings for etree.pyx
by Andreas Pakulat 08 Jun '06

08 Jun '06

Hi, attached is a patch that adds docstrings to those methods of Element and Elementtree that didn't have any. I also changed some docstrings to use """ instead of just " to make it consistent with the rest. Most of the strings are "copied" from the official element tree api. I did not include any reference to the parameters, because help() doesn't show the parameteres here anyway. The diff is against revision 28530 of trunk. Andreas -- Afternoon very favorable for romance. Try a single person for a change.

2 1

[lxml-dev] using pyflakes
by Olivier Grisel 08 Jun '06

08 Jun '06

Hi Stefan and others, I have just checked in a Makefile target to run pyflakes on the lxml source tree: $ sudo easy_install pyflakes $ make lint pyflakes . ./selftest2.py:7: 'sys' imported but unused ./selftest2.py:7: 'StringIO' imported but unused ./setup.py:1: redefinition of unused 'os' from line 1 ./setup.py:10: redefinition of unused 'setup' from line 5 ./setup.py:11: redefinition of unused 'Extension' from line 6 ./bench.py:2: 'from itertools import *' used; unable to detect undefined names ./bench.py:632: redefinition of unused 'time' from line 1 ./selftest.py:12: 'sys' imported but unused ./selftest.py:12: 'StringIO' imported but unused ./update-error-constants.py:3: redefinition of unused 'os' from line 3 ./src/doctest.py:98: 'types' imported but unused ./src/lxml/tests/common_imports.py:11: redefinition of unused 'ElementTree' from line 9 ./src/lxml/tests/test_htmlparser.py:10: 'fileInTestDir' imported but unused ./src/lxml/tests/test_etree.py:14: 'SillyFileLike' imported but unused ./src/lxml/tests/test_errors.py:2: 'doctest' imported but unused ./src/lxml/tests/test_unicode.py:2: 'doctest' imported but unused ./src/lxml/tests/test_unicode.py:4: 'SillyFileLike' imported but unused ./src/lxml/tests/test_io.py:10: 'fileInTestDir' imported but unused ./src/lxml/tests/test_elementtree.py:11: 'doctest' imported but unused ./src/lxml/tests/test_elementtree.py:14: 'HelperTestCase' imported but unused pylint and pychecker might be useful too but pyflakes is simple and very fast. pylint is much more verbose and pychecker requires to import the modules to do its work thus needs inplace compilation first. -- Olivier

2 3

Re: [lxml-dev] Bus Error in external XPath function
by Stefan Behnel 08 Jun '06

08 Jun '06

Hi Petr, Petr van Blokland wrote: > On Jun 8, 2006, at 12:27 PM, Stefan Behnel wrote: >> I committed a (rather large) patch to the trunk, not yet to the >> branch. It >> should fix the problem, please test it. I still have to come up with a >> suitable test case for the test suite. > > That seems to work much better. Thanks. I am going to test is further > in the real server. But at first glance it works. Great, finally. > Just interested: what kind of problem was it? Garbage collection? Sorry, I normally write a bit more about these things, but I was just in a hurry and wanted to get a mail out to have you test it. So here's a longer explanation. It was quite a bit of work to get that fixed. I had replaced the extension function call registry stuff with a generic run-time lookup function in 1.0. However, libxslt seems to have problems with that when the call is coming from an included stylesheet. So the current test case I have is: * a stylesheet that builds a variable value from an extension function call * a second stylesheet that includes and uses the first one * the lxml code that parses the second stylesheet and applies it to a tree The result is that the stylesheet is broken after the first run and subsequent runs no longer work as expected. I have no idea why that happens, no idea at all. I assume that it is a bug in libxslt, but I can't tell for sure. All I know is that it does *not* break when the extension function is registered with libxslt when it is called. So, basically what I did is, I rewrote the code to use pre-registered extension functions as before. So now, there still is a bit of cleanup to be done, also in the XPath code. But the bug should be fixed. Stefan

1 0

Re: [lxml-dev] Bus Error in external XPath function
by Stefan Behnel 08 Jun '06

08 Jun '06

Hi Petr, Petr van Blokland wrote: > > On Jun 7, 2006, at 6:02 PM, Stefan Behnel wrote: > >> Hi Petr, >> >> Petr van Blokland wrote: >>> The error remains, different for both versions though. >>> De different behaviour is reproducable. >> >> It is no difference what the error messages end with as long as they >> state the >> "unregistering unknown proxy" bit. The rest is simply trying to access >> invalid >> (= already freed) memory regions, which simply crashes in most cases. >> >> Still, thanks for testing. The sooner we have this resolved, the >> sooner I can >> release 1.0.1. >> >> Stefan >> > > If you change the > <xsl:variable name="xpyth_path" select="xpyth:get('path')"/> > in > <xsl:variable name="xpyth_path" select="'xxxxxxxx'"/> > > in the template.xsl, then the template works alright. I committed a (rather large) patch to the trunk, not yet to the branch. It should fix the problem, please test it. I still have to come up with a suitable test case for the test suite. Stefan

1 0

[lxml-dev] Losing namespace information after deepcopy()
by Johnny deBris 08 Jun '06

08 Jun '06

Hello! This is new since 1.0: after deepcopying certain structures, namespace information gets lost. Attached is a small test case, if you run it you'll see that the 't' namespace is not displayed in a string representation of a copy of the node with the t:foo attribute. I'm really clueless what could cause this... Hope the snippet helps. Cheers, Guido P.S. Thanks for the file issue fixes, works like a charm now.

2 3

[lxml-dev] parser bug in lxml 1.0
by Martijn Faassen 07 Jun '06

07 Jun '06

Hi there, After a hint from Guido Wesdorp, I tried the following with lxml 1.0: utf.xml: <?xml version="1.0"?> <foo> This is some UTF-8 content: ë </foo> and this script (tryparse.py): from lxml import etree f = open('utf.xml', 'r') etree.parse(f) f.close() running it gives the following traceback: Traceback (most recent call last): File "tryparse.py", line 4, in ? etree.parse(f) File "etree.pyx", line 1468, in etree.parse File "parser.pxi", line 671, in etree._parseDocument File "parser.pxi", line 697, in etree._parseFilelikeDocument File "parser.pxi", line 622, in etree._parseDocFromFilelike File "parser.pxi", line 379, in etree._BaseParser._parseDocFromFilelike File "parser.pxi", line 418, in etree._handleParseResult File "etree.pyx", line 151, in etree._ExceptionContext._raise_if_stored File "parser.pxi", line 159, in etree.copyToBuffer File "apihelpers.pxi", line 319, in etree._utf8 AssertionError: All strings must be Unicode or ASCII This is of course wrong. lxml should definitely be able to parse UTF-8 encoded XML files. This did work in previous versions of lxml too. It also looks like it is going into an in-memory string parser. I recall in earlier versions of lxml this wasn't necessary - the file object was inspected and the filename was extracted, passing it into libxml2 directly. Regards, Martijn

3 9

[lxml-dev] documentation improvement
by Andreas Pakulat 06 Jun '06

06 Jun '06

Hi, I recently overlooked the prefix attribute for lxml Element's and I wonder if there are any plans to provide an api document that is similar to the one of ElementTree but includes all "extensions"? I already tried to use the docstrings from the source, but there are quite some things missing (I didn't get any _* classes on the first try). I know pretty much the whole documentation is already "there", but it's distributed over many text files. One needs to read all and remember everything, especially for the extensions because there's no way to easily see the extensions of a given class. I'm willing to help here and maybe for starters I could "collect" all current documentation in a howto-like latex file (using Python's howto document class). From this one can easily generate various formats (pdf, html, ps and ascii) using Pythons mkhowto tool. Andreas -- Courage is your greatest present need.

4 11

[lxml-dev] Bus Error in external XPath function
by Petr van Blokland 06 Jun '06

06 Jun '06

Hi, I have build lxml XSLT transformation into a webserver, based on Twisted Matrix. I runs all right, unless I used a call to a Python function in an XSL template with XPath. The generation of a page prints a remark to the output "transforming page". The following error is very consistent, and predictable for the amount of pages after which is goes wrong. --> python start.py Starting server as user/group 501/501 (petr) ====== transforming page ====== transforming page ====== transforming page ====== transforming page Exception exceptions.AssertionError: 'Tried to unregister unknown proxy' in Bus error --> So after the rendering of 5 identical pages (reloads from a browser) the XPyth call generates a Bus error (as said, this does not happen with XSL without this call). I guess this is too little information, so I'll try to make a small test program that can reproduce it. Regards, Petr ---------------------------------------------- Petr van Blokland buro(a)petr.com | www.petr.com | +31 15 219 10 40 ----------------------------------------------

2 8

[lxml-dev] Threading redux
by Stefan Behnel 05 Jun '06

05 Jun '06

Hi all, there is now a branch where I implemented thread concurrency for the parsers. http://codespeak.net/svn/lxml/branch/threading/ This branch is not very well tested. I attached the only test program I have so far. There are likely some bizarre race conditions that I overlooked. Note that parsers are not currently locked, so lxml will crash if multiple threads use /the same/ parser concurrently. I'll see how to do that efficiently. If you have an interest in getting this integrated, please build the branch on your side and do some testing. There are other places where we can try to add concurrency, but that is still to come. Feel free to post a list of personal preferences including little test programs (like the one attached) that show concurrent usage. Stefan

1 1