Mailman 3 February 2006 - lxml - The Python XML Toolkit

[lxml-dev] Call for Contribution: lxml 0.9
by Stefan Behnel 09 Mar '06

09 Mar '06

Hello, everyone! Martijn and I realised that there has not been a release of lxml for quite a while - despite a lot of important changes to the code base. We therefore agreed to prepare the release of lxml 0.9 for *Monday, March 20*. The remaining time will be used to get the trunk ready for a release. It will include the current feature set of the trunk plus the support for extension functions that is currently implemented in the scoder2 branch. The latter still needs a bit of discussion and merging. Any comments on this are appreciated. For you, this announcement means the following. - If you want any features included, make sure you send requests and patches to the list within the next days or try to fix a date when a patch will be available. Remember that we may have to discuss patches before applying them. - If you feel that you especially like or dislike anything about the new features and the way they are made available through APIs, feel free to tell us over the list. - If you feel that anything from past discussions has been left out that should be worth remembering for a new release, speak up now. - We know that there are still a few places where documentation would help users to understand new features. A good way to contribute would therefore be to look through the documentation in the "doc" directory and see if it is understandable or if there is anything you might want to add. - In the same line, usage examples in the form of doctests would be nice to have for more API functions. They also belong into the files in the "doc" directory, which are (or must be) called from the related test cases in src/lxml/tests. Happy coding, Stefan

5 14

[lxml-dev] Request for help on testing new libxml2 feature
by Andreas Pakulat 06 Mar '06

06 Mar '06

Hi, regarding the remove-redundant-namespaces issue there are news: kbuchcik implemented the xmlDOMWrapReconcileNamespaces in tree.c of libxml2 so it should remove redundant NS decl. However neither do I have any experience with libxml2 nor do I have the time to dig into it so that I can build a test program for this. Thus I ask you guys here, who surely are libxml2 experts, if you could help me out here. Either some "hack" for lxml that allows me to test this or a small programm that takes an xml file and applies this function to it's DOM tree (and outputs the result) would be really great. Thanks, Andreas -- You now have Asian Flu.

4 12

[lxml-dev] lxml build problem
by Allan Curtis 27 Feb '06

27 Feb '06

Hi, I am trying to build lxml from the latest svn build on a Suse10.0 based computer with GCC4.02 and I seem to be getting gcc related errors when compiling etree. python setup.py build_ext -i running build_ext building 'lxml.etree' extension gcc -pthread -fno-strict-aliasing -DNDEBUG -O2 -fmessage-length=0 -Wall -D_FORTI FY_SOURCE=2 -g -fPIC -I/usr/include/python2.4 -c src/lxml/etree.c -o build/temp. linux-x86_64-2.4/src/lxml/etree.o -w -I/usr/include/libxml2 src/lxml/etree.c: In function ‘__pyx_f_5etree_12_ElementTree_parse’: src/lxml/etree.c:921: error: invalid lvalue in assignment src/lxml/etree.c:932: error: invalid lvalue in assignment src/lxml/etree.c: In function ‘__pyx_f_5etree__newElementTree’: src/lxml/etree.c:1858: error: invalid lvalue in assignment src/lxml/etree.c:1884: error: invalid lvalue in assignment src/lxml/etree.c: In function ‘__pyx_f_5etree__elementFactory’: src/lxml/etree.c:4135: error: invalid lvalue in assignment src/lxml/etree.c: In function ‘__pyx_f_5etree__commentFactory’: src/lxml/etree.c:4522: error: invalid lvalue in assignment src/lxml/etree.c: In function ‘__pyx_f_5etree__attribFactory’: src/lxml/etree.c:5591: error: invalid lvalue in assignment src/lxml/etree.c: In function ‘__pyx_f_5etree__xsltResultTreeFactory’: src/lxml/etree.c:8377: error: invalid lvalue in assignment src/lxml/etree.c: In function '__pyx_f_5etree_22XPathDocumentEvaluator___init__’ : src/lxml/etree.c:8473: error: invalid lvalue in assignment src/lxml/etree.c:8478: error: invalid lvalue in assignment src/lxml/etree.c: In function ‘__pyx_f_5etree_21XPathElementEvaluator___init__’: src/lxml/etree.c:9256: error: invalid lvalue in assignment src/lxml/etree.c: In function ‘__pyx_f_5etree_changeDocumentBelowHelper’: src/lxml/etree.c:13105: error: invalid lvalue in assignment error: command 'gcc' failed with exit status 1 make: *** [inplace] Error 1 Is anyone able to assist with resolving this? Cheers, -- Allan

3 5

[lxml-dev] Use of XPathEvaluator object in extension functions?
by Stefan Behnel 25 Feb '06

25 Feb '06

Hi, I just looked through the documentation a bit and started rewriting xpath.txt for the new Namespace API. I never actually thought much about it, but the docs made me stumble over the fact that extension functions take the XPathEvaluator object as first parameter. I don't really know what that is good for. Has anyone used this feature so far? What purpose does it serve? When I rewrote the implementation, I introduced a 'ContextBuilder', which is now passed instead. I'm wondering if that can fulfil the original intention... Can anyone enlighten me? Stefan

1 0

Re: [lxml-dev] xml:space and xml:lang problem
by Stefan Behnel 22 Feb '06

22 Feb '06

Scott Haeger wrote: > It seems that lxml is not serializing properly. The following code will > demonstrate: > > from lxml import etree > import sys > > # parse xml from file > intree = etree.parse("test.xml") > > # create root for new tree > outroot = etree.Element("root") > > # go through original and append to new tree > doc = intree.getiterator() > for el in doc: > newel = el > outroot.append(newel) > > # create new tree and output to screen > outtree = etree.ElementTree(outroot) > outtree.write(sys.stdout) > > > Test.xml is the following: > > <?xml version="1.0"?> > <svg > xmlns:xml=" http://www.w3.org/1998/XML"> > <a id="first" xml:space="default"></a> > </svg> > > Although the demo does not show the random characters (garbage) as seen > in my full application, it does demostrate a failure to serialize > properly. Copying elements between trees will most likely not change the result. I cut that down to the following: ---------- .>>> from lxml import etree .>>> intree = etree.XML("""<?xml version="1.0"?> ... <svg xmlns:xml=" http://www.w3.org/1998/XML"> ... <a id="first" xml:space="default"></a> ... </svg> ... """) .>>> etree.tostring(intree) '<svg>\n<a id="first" xml:space="default"/>\n</svg>' ---------- So this definitely misses the XML namespace declaration. BUT, according to the spec, that is not a problem. """ The prefix xml is by definition bound to the namespace name http://www.w3.org/XML/1998/namespace. """ Source: http://www.w3.org/TR/REC-xml-names/ Is that what you meant when you said it demonstrates a failure? > The problem occurs with and without the namespace declaration. Because, according to the spec, both are the same. > Also, removing the xml:space attribute corrects the problem. Probably, since it only references explicitly declared namespaces in that case. Still, could you try to come up with an example that shows your unreadable characters on serialization? Thanks, Stefan

3 4

[lxml-dev] xml:space and xml:lang problem
by Scott Haeger 21 Feb '06

21 Feb '06

It is my understanding that xml has two special attributes under the xml namespace, namely xml:space and xml:lang. My problem is that I am getting garbage for the xml namespace such as xmlns:="(XXX)://www.w3.org/XML/1998/namespace", where (XXX) is complete garbage. The problem occurs both when the namespace is declared and when it is not declared. Scott

2 1

[lxml-dev] is valgrind still used?
by Victor Ng 17 Feb '06

17 Feb '06

Hi all, It's been a long time since I've been on this list, but I'm hoping to start hacking again. Whoo! New job. More open source hacking time. On to the questions! Is valgrind still being used anymore? I just ran : > valgrind --tool=memcheck --suppressions=valgrind-python.supp python test.py and got: ==13242== ==13242== ERROR SUMMARY: 138 errors from 16 contexts (suppressed: 10026 from 6) ==13242== malloc/free: in use at exit: 2848232 bytes in 2981 blocks. ==13242== malloc/free: 53700 allocs, 50719 frees, 21000875 bytes allocated. ==13242== For counts of detected errors, rerun with: -v ==13242== searching for pointers to 2981 not-freed blocks. ==13242== checked 3490012 bytes. ==13242== ==13242== LEAK SUMMARY: ==13242== definitely lost: 4759 bytes in 168 blocks. ==13242== possibly lost: 0 bytes in 0 blocks. ==13242== still reachable: 2843473 bytes in 2813 blocks. ==13242== suppressed: 0 bytes in 0 blocks. ==13242== Use --leak-check=full to see details of leaked memory. victorng@mbdev1:~/dev/lxml$ Is this expected? Is there a Trac instance or some other bug tracker that I can look at to see what the open bugs are? Thanks! Hope to be less than useless soon. :) vic

2 1

Re: [lxml-dev] fix for another memory leak
by Stefan Behnel 16 Feb '06

16 Feb '06

Narayan Desai schrieb: > I updated to the current svn, and it looks like this chunk didn't get > committed. Is this intentional? Oh, thanks for checking! :) It was by accident. It looked so much like the patch I had committed before the last one that I just thought you had copied it. I'll check it in as soon as I get to it. Thank you for taking care, Stefan > Index: src/lxml/etree.pyx > =================================================================== > --- src/lxml/etree.pyx (revision 23382) > +++ src/lxml/etree.pyx (working copy) > @@ -775,16 +775,18 @@ > > def __getitem__(self, key): > cdef xmlNs* c_ns > - cdef char* result > + cdef char* cresult > ns, tag = _getNsTag(key) > if ns is None: > - result = tree.xmlGetNoNsProp(self._c_node, tag) > + cresult = tree.xmlGetNoNsProp(self._c_node, tag) > else: > - result = tree.xmlGetNsProp(self._c_node, tag, ns) > - if result is NULL: > + cresult = tree.xmlGetNsProp(self._c_node, tag, ns) > + if cresult is NULL: > # XXX free namespace that is not in use..? > raise KeyError, key > - return funicode(result) > + result = funicode(cresult) > + tree.xmlFree(cresult) > + return result > > def __len__(self): > cdef int c

1 0

[lxml-dev] fix for another memory leak
by Narayan Desai 14 Feb '06

14 Feb '06

In addition to the memory leak in the Element.get method, I have recently been seeing memory leaks in Element.Attrib.has_key. This results in the following trace from valgrind: ==12133== 8 bytes in 4 blocks are definitely lost in loss record 2 of 48 ==12133== at 0x401B422: malloc (vg_replace_malloc.c:149) ==12133== by 0x453D8B5: xmlStrndup (in /usr/lib/libxml2.so.2.6.23) ==12133== by 0x453D943: xmlStrdup (in /usr/lib/libxml2.so.2.6.23) ==12133== by 0x453E0D1: xmlStrcat (in /usr/lib/libxml2.so.2.6.23) ==12133== by 0x44E98A1: xmlNodeListGetString (in /usr/lib/libxml2.so.2.6.23) ==12133== by 0x44EF6D2: xmlGetNoNsProp (in /usr/lib/libxml2.so.2.6.23) ==12133== by 0x4462ED0: __pyx_f_5etree_7_Attrib_has_key (etree.c:5389) ==12133== by 0x80B6BE3: (within /usr/bin/python2.3) ==12133== by 0x80B8356: PyEval_EvalCodeEx (in /usr/bin/python2.3) ==12133== by 0x80B85D4: PyEval_EvalCode (in /usr/bin/python2.3) ==12133== by 0x80D8EFF: PyRun_InteractiveOneFlags (in /usr/bin/python2.3) ==12133== by 0x80D9018: PyRun_InteractiveLoopFlags (in /usr/bin/python2.3) The following patch appears to fix things for me. It is basically copied from the Element.get fix. -nld Index: src/lxml/etree.pyx =================================================================== --- src/lxml/etree.pyx (revision 23162) +++ src/lxml/etree.pyx (working copy) @@ -855,6 +855,7 @@ result = tree.xmlGetNoNsProp(self._c_node, tag) else: result = tree.xmlGetNsProp(self._c_node, tag, ns) + tree.xmlFree(result) return result is not NULL def __contains__(self, key):

2 3

[lxml-dev] Use lxml under windows
by Andreas Pakulat 14 Feb '06

14 Feb '06

Hi, I'd like to use lxml under windows, but I can't compile the C extension: C:\lxml-scoder2>python setup.py build -c mingw32 running build running build_py running build_ext building 'lxml.etree' extension creating build\temp.win32-2.4 creating build\temp.win32-2.4\Release creating build\temp.win32-2.4\Release\src creating build\temp.win32-2.4\Release\src\lxml c:\Programme\CodeBlocks\bin\gcc.exe -mno-cygwin -mdll -O -Wall -IC:\libxml\inclu de -IC:\Python24\include -IC:\Python24\PC -c src/lxml/etree.c -o build\temp.win3 2-2.4\Release\src\lxml\etree.o -w writing build\temp.win32-2.4\Release\src\lxml\etree.def c:\Programme\CodeBlocks\bin\gcc.exe -mno-cygwin -shared -s build\temp.win32-2.4\ Release\src\lxml\etree.o build\temp.win32-2.4\Release\src\lxml\etree.def -LC:\li bxml\lib -LC:\Python24\libs -LC:\Python24\PCBuild -llibexslt -lzdll -llibxml2 -l libxslt -liconv -lzlib -lpython24 -lmsvcr71 -o build\lib.win32-2.4\lxml\etree.py d build\temp.win32-2.4\Release\src\lxml\etree.o:etree.c:(.text+0x38b3): undefined reference to `_imp__xmlFree' build\temp.win32-2.4\Release\src\lxml\etree.o:etree.c:(.text+0x193b0): undefined reference to `_imp__xmlFree' build\temp.win32-2.4\Release\src\lxml\etree.o:etree.c:(.text+0x202d6): undefined reference to `_imp__xmlFree' build\temp.win32-2.4\Release\src\lxml\etree.o:etree.c:(.text+0x2044a): undefined reference to `_imp__xmlFree' collect2: ld returned 1 exit status error: command 'gcc' failed with exit status 1 I already altered the setup.py to include include_dir=['c:\libxml\include'] library_dirs=['c:\libxml\lib'] libaries=['libxml','libxslt','iconv','zlib'] libxml2 is from the prebuilt-windows-binaries. Now the question is: Are those prebuilt-binaries incorrectly built and thus don't include the necessary symbols or is lxml at fault here? Would maybe building libxml2 myself solve this? Andreas -- Today is what happened to yesterday.

1 0