Mailman 3 September 2006 - lxml - The Python XML Toolkit

[lxml-dev] Failure to Compile on Windows
by Sidnei da Silva 25 Oct '06

25 Oct '06

I've got pretty far with setting up the dependencies and all but after that the compilation failed with this error. Anyone have a clue? Looks like it's missing some .h that 'nano http' depends on. C:\Arquivos de programas\Microsoft Visual C++ Toolkit 2003\bin\link.exe /DLL /no logo /INCREMENTAL:NO /LIBPATH:c:\Python24\libs /LIBPATH:c:\Python24\PCBuild /EXPORT:initobjectify build\temp.win32-2.4\Release\src/lxml/objectify.obj /OUT:build\lib.win32-2.4\lxml\objectify.pyd /IMPLIB:build\temp.win32-2.4\Release\src/lxml\objectify.lib ..\libxml2-2.6.26.win32\lib\libxml2_a.lib ..\libxslt-1.1.17.win32\lib\libxslt_a.lib ..\libxslt-1.1.17.win32\lib\libexslt_a.lib ..\zlib-1.2.3.win32\lib\zlib.lib ..\iconv-1.9.2.win32\lib\iconv_a.lib Creating library build\temp.win32-2.4\Release\src/lxml\objectify.lib and object build\temp.win32-2.4\Release\src/lxml\objectify.exp libxml2_a.lib(nanohttp.obj) : error LNK2019: unresolved external symbol __imp__WSAGetLastError@0 referenced in function _socket_errno libxml2_a.lib(nanohttp.obj) : error LNK2019: unresolved external symbol __imp__WSACleanup@0 referenced in function _xmlNanoHTTPCleanup libxml2_a.lib(nanoftp.obj) : error LNK2001: unresolved external symbol __imp__WSACleanup@0 libxml2_a.lib(nanohttp.obj) : error LNK2019: unresolved external symbol __imp__closesocket@4 referenced in function _xmlNanoHTTPFreeCtxt libxml2_a.lib(nanoftp.obj) : error LNK2019: unresolved external symbol __imp__closesocket@4 referenced in function _xmlNanoFTPNewCtxt libxml2_a.lib(nanohttp.obj) : error LNK2019: unresolved external symbol __imp__select@20 referenced in function _xmlNanoHTTPSend libxml2_a.lib(nanoftp.obj) : error LNK2001: unresolved external symbol __imp__select@20 libxml2_a.lib(nanohttp.obj) : error LNK2019: unresolved external symbol __imp__send@16 referenced in function _xmlNanoHTTPSend libxml2_a.lib(nanoftp.obj) : error LNK2001: unresolved external symbol __imp__send@16 libxml2_a.lib(nanohttp.obj) : error LNK2019: unresolved external symbol __imp__recv@16 referenced in function _xmlNanoHTTPRecv libxml2_a.lib(nanoftp.obj) : error LNK2001: unresolved external symbol __imp__recv@16 libxml2_a.lib(nanohttp.obj) : error LNK2019: unresolved external symbol __imp__getsockopt@20 referenced in function _xmlNanoHTTPConnectAttempt libxml2_a.lib(nanohttp.obj) : error LNK2019: unresolved external symbol ___WSAFDIsSet@8 referenced in function _xmlNanoHTTPConnectAttempt libxml2_a.lib(nanohttp.obj) : error LNK2019: unresolved external symbol __imp__connect@12 referenced in function _xmlNanoHTTPConnectAttempt libxml2_a.lib(nanoftp.obj) : error LNK2001: unresolved external symbol __imp__connect@12 libxml2_a.lib(nanohttp.obj) : error LNK2019: unresolved external symbol __imp__ioctlsocket@12 referenced in function _xmlNanoHTTPConnectAttempt libxml2_a.lib(nanohttp.obj) : error LNK2019: unresolved external symbol __imp__socket@12 referenced in function _xmlNanoHTTPConnectAttempt libxml2_a.lib(nanoftp.obj) : error LNK2001: unresolved external symbol __imp__socket@12 libxml2_a.lib(nanohttp.obj) : error LNK2019: unresolved external symbol __imp__htons@4 referenced in function _xmlNanoHTTPConnectHost libxml2_a.lib(nanoftp.obj) : error LNK2001: unresolved external symbol __imp__htons@4 libxml2_a.lib(nanohttp.obj) : error LNK2019: unresolved external symbol __imp__gethostbyname@4 referenced in function _xmlNanoHTTPConnectHost libxml2_a.lib(nanoftp.obj) : error LNK2001: unresolved external symbol __imp__gethostbyname@4 libxml2_a.lib(nanohttp.obj) : error LNK2019: unresolved external symbol __imp__WSAStartup@8 referenced in function _xmlNanoHTTPInit libxml2_a.lib(nanoftp.obj) : error LNK2001: unresolved external symbol __imp__WSAStartup@8 libxml2_a.lib(nanoftp.obj) : error LNK2019: unresolved external symbol __imp__listen@8 referenced in function _xmlNanoFTPGetConnection libxml2_a.lib(nanoftp.obj) : error LNK2019: unresolved external symbol __imp__bind@12 referenced in function _xmlNanoFTPGetConnection libxml2_a.lib(nanoftp.obj) : error LNK2019: unresolved external symbol __imp__getsockname@12 referenced in function _xmlNanoFTPGetConnection build\lib.win32-2.4\lxml\objectify.pyd : fatal error LNK1120: 17 unresolved externals error: command '"C:\Arquivos de programas\Microsoft Visual C++ Toolkit 2003\bin\link.exe"' failed with exit status 1120 -- Sidnei da Silva Enfold Systems http://enfoldsystems.com Fax +1 832 201 8856 Office +1 713 942 2377 Ext 214

3 6

[lxml-dev] Building Problems
by Achim Kern 13 Oct '06

13 Oct '06

Hello everybody, during googeling on how to write easier xml datastores with python I just found our project. Especialy the objectify modules impressed me. So to test things I wanted to install it. Unfortunatly I can not use the provided debian package as there is only one for version 1.03 not including the objectify extension. So I downloaded the source of 1.1.1 from codespeak.net extracted it and that's what it did. # tar -xzf lxml-1.1.1.tgz # ce lxml-1.1.1 # make clean test. python setup.py build_ext -i Building lxml version 1.1 running build_ext python test.py -p -v Traceback (most recent call last): File "test.py", line 591, in ? exitcode = main(sys.argv) File "test.py", line 554, in main test_cases = get_test_cases(test_files, cfg, tracer=tracer) File "test.py", line 254, in get_test_cases module = import_module(file, cfg, tracer=tracer) File "test.py", line 197, in import_module mod = __import__(modname) File "/tmp/lxml-1.1/src/lxml/tests/test_objectify.py", line 16, in ? from lxml import objectify ImportError: /tmp/lxml-1.1/src/lxml/objectify.so: undefined symbol: previousElement make: *** [test_inplace] Error 1 It builds just ok but when it comes to test it crashes. There seams to be something missing. I have installed libxml2 in version 2.6.26 and libxslt in version 1.1.17. I investigated the problem a little further with ldd and it gives me: # ldd -d src/lxml/objectify.so undefined symbol: PyTraceBack_Type (src/lxml/objectify.so) undefined symbol: _Py_NoneStruct (src/lxml/objectify.so) undefined symbol: PyString_Type (src/lxml/objectify.so) undefined symbol: previousElement (src/lxml/objectify.so) undefined symbol: PyExc_SystemError (src/lxml/objectify.so) undefined symbol: PyExc_ValueError (src/lxml/objectify.so) undefined symbol: PyExc_TypeError (src/lxml/objectify.so) undefined symbol: PyBaseString_Type (src/lxml/objectify.so) undefined symbol: PyExc_IndexError (src/lxml/objectify.so) undefined symbol: PyTuple_Type (src/lxml/objectify.so) undefined symbol: PyExc_AttributeError (src/lxml/objectify.so) undefined symbol: PyClass_Type (src/lxml/objectify.so) undefined symbol: nextElement (src/lxml/objectify.so) undefined symbol: PyList_Type (src/lxml/objectify.so) undefined symbol: PyExc_NotImplementedError (src/lxml/objectify.so) undefined symbol: PyBool_Type (src/lxml/objectify.so) undefined symbol: PyInstance_Type (src/lxml/objectify.so) undefined symbol: PyType_Type (src/lxml/objectify.so) undefined symbol: PyObject_GC_Del (src/lxml/objectify.so) undefined symbol: PyExc_NameError (src/lxml/objectify.so) linux-gate.so.1 => (0xffffe000) libexslt.so.0 => /usr/lib/libexslt.so.0 (0xa7eb2000) libxslt.so.1 => /usr/lib/libxslt.so.1 (0xa7e80000) libxml2.so.2 => /usr/lib/libxml2.so.2 (0xa7d67000) libpthread.so.0 => /lib/tls/libpthread.so.0 (0xa7d55000) libc.so.6 => /lib/tls/libc.so.6 (0xa7c23000) libgcrypt.so.11 => /usr/lib/libgcrypt.so.11 (0xa7bd2000) libgpg-error.so.0 => /usr/lib/libgpg-error.so.0 (0xa7bcd000) libm.so.6 => /lib/tls/libm.so.6 (0xa7ba8000) libdl.so.2 => /lib/tls/libdl.so.2 (0xa7ba4000) libz.so.1 => /usr/lib/libz.so.1 (0xa7b90000) /lib/ld-linux.so.2 (0x75555000) libnsl.so.1 => /lib/tls/libnsl.so.1 (0xa7b7a000) Can anybody give me a hint where previousElement is defined? Or did I do something wrong? Thanks in advance. Achim Kern PS: if responding please put me into the cc, as I am currently not able to receive mails from mailinglists. Thank you.

4 13

[lxml-dev] a buildout for lxml
by Martijn Faassen 05 Oct '06

05 Oct '06

Hi there, I've just created what's called a "buildout" for lxml. Buildout is a new system for constructing Python applications and its constituent parts. More information can be found here: http://cheeseshop.python.org/pypi/zc.buildout/ Now before you read all that (it's fairly technical), let me explain what problem the lxml buildout tries to solve. Yesterday I tried using lxml's XMLSchema validator against an XML file and a schema. It gave a validation error, but to the best of my knowledge, the XML file should've been valid. Thinking it might perhaps be a bug that was fixed in more recent versions of libxml2, I set out to build and install libxml2 and libxslt locally, compiled lxml against it, and confirmed it: the document does validate against the schema with the most recent release of libxml2. Now downloading, building and installing libxml2 and libxslt and linking lxml against it is quite a bit of work. Buildout to the rescue! Now it's just this: $ svn co https://infrae.com/svn/buildout/lxml-recent/trunk lxml-recent $ cd lxml-recent $ python bootstrap/bootstrap.py $ bin/buildout and then wait. It'll download, configure and compile libxml2, libxslt, will downoad the lxml source and compile it against the libraries. In the end you end up with a script that invokes the Python interpreter with the right paths, so you can import the new lxml: $ bin/lxmlpython Though you can also manipulate the PYTHONPATH yourself in your scripts to put the lxml development egg on it, and then you can use the regular Python you used to run bootstrap.py with. To see how this works, you can take a look at buildout.cfg. This specifies how things are put together. Note that 'zc.recipe.cmmi' is a 'buildout recipe'. Recipes are components responsible for deployment of particular bits of an application. If zc.recipe.cmmi were released as an egg onto the cheeseshop, buildout would go out and download and install the recipe from there. Since it's not on the cheeseshop yet, I've installed it as a development-egg and included it as an svn:external, but this can be fixed by wrapping that into an egg. Many buildout recipes exist: this one does 'configure; make; make install', hence 'cmmi'. There are also recipes that build and install Zope 3, recipes to build tricky Python modules that don't work as an egg yet, and so on. New recipes can be developed. At Infrae's we've created recipes that can deploy OpenOffice in server mode, so that it can be used through UNO for document conversion purposes. This expanding pool of recipes will allow us to put together extremely complicated applications in a reproducable fashion. The idea of buildout is to have control - to know exactly what is installed and not to have to install everything into site-packages. buildout uses setuptools and easy_install internally, so the whole egg story just works; it expands on that. I think this way of building lxml has more benefits than just my single case. It'll allow one to deploy lxml against a known-good version of libxml2 and libxslt, at least on a linux system. It should also help testing and developing lxml - it becomes a lot more easy to test and develop against different versions of libxml2. I've placed this buildout in infrae.com SVN for now (checkout command above), but if people think this is interesting, I can integrate this into the lxml trunk. This can be done without harming the current way of working, but it'll allow someone to test against any version of libxml2 and libxslt a lot more easily than one can do now. Comments? Regards, Martijn

3 6

[lxml-dev] lxml replace() deletes tail
by Chris Abraham 02 Oct '06

02 Oct '06

Hi, We have a question about the etree.replace() function. We found that it doesn't preserve the tail text from the replaced node when inserting a new node. Perhaps this is the intended behavior, but, to us, it was unexpected. In the example below, notice how the "tail" text is deleted when the <p> is replaced: >>> tree = etree.HTML("<html><body>text before<p>textin</p>tail</body></html>") >>> newel = etree.HTML("<html><body><div>new</div></body></html>") >>> tree[0].replace(tree[0][0], newel[0][0]) >>> etree.tostring(tree) '<html><body>text before<div>new</div></body></html>' Any ideas or comments? Thanks. Luke Tucker & Chris Abraham

2 1

[lxml-dev] Stylesheet Processing Instruction
by Sidnei da Silva 28 Sep '06

28 Sep '06

Hi there, Someone asked me if lxml would handle a 'Stylesheet Processing Instruction', which seems to be the way to embed the stylesheet into the XML to be transformed. ie, if you use the said instruction and open the XML in the browser (IE and Firefox?) the browser automatically applies the transform. Since the 'xsltproc' command also seems to do this, from it's man page, I expected lxml to do as well, but didn't actually try. So, can anyone confirm/deny if it's supported? -- Sidnei da Silva Enfold Systems http://enfoldsystems.com Fax +1 832 201 8856 Office +1 713 942 2377 Ext 214

3 27

[lxml-dev] problems accessing style tag text
by Luke Tucker 27 Sep '06

27 Sep '06

Hey, I was trying to get at the contents of a style tag parsed with etree.HTML and it doesn't seem to be accessible as the text attribute of the element. This appears to be related to the insertion of the CDATA marker around the text. Maybe there is something obvious that I'm missing. I tried this against a 1.1 release and a recent checkout of the trunk. I appreciate any advice you all can offer. The following demonstrates what I'm seeing: >>> from lxml import etree >>> text = '<html><head><style>foo</style></head></html>' >>> html = etree.HTML(text) >>> print html[0][0] <Element style at 2b97a66f9140> >>> print html[0][0].text None >>> etree.tostring(html[0][0]) '<style><![CDATA[foo]]></style>' >>> len(html[0][0]) 0 Thanks, - Luke

3 3

[lxml-dev] lxml 1.1.1 win32 builds for 2.4 and 2.5
by Ashish Kulkarni 27 Sep '06

27 Sep '06

Hello, The builds are up at the usual location: http://puggy.symonds.net/~ashish/downloads/ Please note that the requirement for linking in wsock32.lib is still present in both 2.4 and 2.5. It should at least be mentioned as a possible requirement in the documentation. Regards, ashish

5 9

[lxml-dev] strange problem with lxml-1.1.1 and py2.5
by Narayan Desai 26 Sep '06

26 Sep '06

I am running into a strange problem when using python2.5 and lxml-1.1.1. calling findall causing a non-sensical exception to be raised. I know that this code is fine; it works perfectly with python2.4. The code in my app calls findall on an Element. Traceback (most recent call last): File "/usr/lib/python2.5/site-packages/Bcfg2/Server/Core.py", line 175, in Service self.handles[event.requestID].HandleEvent(event) File "/usr/lib/python2.5/site-packages/Bcfg2/Server/Plugin.py", line 301, in HandleEvent DirectoryBacked.HandleEvent(self, event) File "/usr/lib/python2.5/site-packages/Bcfg2/Server/Plugin.py", line 119, in HandleEvent self.AddEntry(event.filename) File "/usr/lib/python2.5/site-packages/Bcfg2/Server/Plugin.py", line 109, in AddEntry self.entries[name].HandleEvent() File "/usr/lib/python2.5/site-packages/Bcfg2/Server/Plugin.py", line 267, in HandleEvent self.pnode = self.__node__(xdata, self.items) File "/usr/lib/python2.5/site-packages/Bcfg2/Server/Plugins/Pkgmgr.py", line 22, in __init__ Bcfg2.Server.Plugin.INode.__init__(self, data, pdict, parent) File "/usr/lib/python2.5/site-packages/Bcfg2/Server/Plugin.py", line 218, in __init__ self.children.append(mytype(item, idict, self)) File "/usr/lib/python2.5/site-packages/Bcfg2/Server/Plugins/Pkgmgr.py", line 25, in __init__ for pkg in data.findall('./Package'): File "etree.pyx", line 1164, in etree._Element.findall File "/usr/lib/python2.5/site-packages/lxml/_elementpath.py", line 193, in findall return _compile(path).findall(element) File "/usr/lib/python2.5/site-packages/lxml/_elementpath.py", line 155, in findall for node in node: File "etree.pyx", line 1446, in etree._ElementIterator.__next__ TypeError: exceptions must be strings, classes, or instances, not type I have not tried svn, though I could if it would be useful. thanks.. -nld

2 2

[lxml-dev] xinclude bug?
by Greg Steffensen 25 Sep '06

25 Sep '06

I'm working on a project that will use lxml's xinclude functionality to insert the contents of python files into an xml document and have noticed a possible bug. When you xinclude with the parse attribute set to "text", the text frequently (though not always) gets loaded into multiple adjacent text nodes, so that if you access the text attribute of a containing element, you only get part of the actual text. You can verify this by calling element.xpath('text()') on the container... you get back a list with multiple elements. Is this how things are supposed to work? Also, escaping seems to occur in strings accessed from the "text" attribute of xincluded content, but not in strings retrieved via xpath, as described above. Is there a reliable way to reverse the escaping process, so that the original contents of the xincluded file can be retrieved? I assume that xml.sax.saxutils.unescape() would work, but don't know for sure. I've pasted some example code below to demonstrate the seemingly broken "text" attribute use and the different escaping styles. Thanks, Greg doc.xml ----------------------------- <?xml version="1.0" encoding="UTF-8"?> <doc xmlns:xi="http://www.w3.org/2001/XInclude"> <xi:include href="doc.py" parse="text"/> </doc> doc.py --------------------------- #!/usr/bin/python s1 = '3 < 4' s2 = "hello;" test.py -------------------------- from lxml import etree tree = lxml.parse('doc.xml') tree.xinclude () root = tree.getroot() print repr(root.text) print '----' print root.xpath('text()')

2 2

Re: [lxml-dev] xinclude bug?
by Greg Steffensen 25 Sep '06

25 Sep '06

In retrospect my example was poorly chosen, but fortunately, you got the gist. The escaping inconsistency I thought I'd noticed now seems to only occur when using etree.tostring, which replaces "<" and ">" in text nodes with their entities. That seems like correct behavior after all. Thanks, Greg

1 0