Mailman 3 December 2005 - lxml - The Python XML Toolkit

[lxml-dev] install problem on Windows XP
by Werner F. Bruhin 02 Feb '06

02 Feb '06

I am running into problems trying to: \python24\python setup.py install I changed the following in setup.py: #include_dirs = guess_dirs('--cflags', '-I') #library_dirs = guess_dirs('--libs', '-L') # if you want to configure include and library dir manually, you can do # so here, for instance: include_dirs = [r'C:\Python24\Lib\site-packages\libxslt-1.1.14.win32\include',] # '/home/faassen/tmp/local_new/include'] library_dirs = [r'C:\Python24\Lib\site-packages\libxml2-2.6.20.win32\lib'] But I am getting an error that I need .net SDK (error: The .NET Framework SDK needs to be installed before building extensions f or Python.). Did I not change the setup.py correctly or do I really need the .net SDK. Any chance that there is a binary build for Windows XP is available? See you Werner

3 2

[lxml-dev] lxml branch
by Godefroid Chapelle 24 Jan '06

24 Jan '06

Hi Martijn, list I have checked code in a branch that allow to use a different parser than the default globally shared. This allows to setup different options than the default. All previous tests (321 ?) do pass. I'd be pleased to get feedback about the code before working on the supplementary tests needed. I am not totally sure I went the right way even if it works for me... Cheers -- Godefroid Chapelle (aka __gotcha)- BubbleNet http://bubblenet.be

4 17

[lxml-dev] Doctest for Namespace class
by Stefan Behnel 22 Jan '06

22 Jan '06

Hi! The Namespace class now has a doctest in scoder2/doc/namespace_extensions.txt. I also gave a quick example on how to use XPath extension functions, but I still somewhat feel that the prefixes should not have to be specified on evaluation. My current example looks like this: .>>> from lxml.etree import Namespace .>>> def tag_of(context, elem): ... return elem[0].tag .>>> namespace = Namespace('myfunctions') .>>> namespace['tagname'] = tag_of .>>> element = XML('<test><honk honking="true"/></test>') .>>> element.xpath('f:tagname(//honk)', {'f' : 'myfunctions'}) 'honk' What bothers me is the {'f' : 'myfunctions'} in there. The user shouldn't have to care about that. What about supporting this: .>>> element.xpath('{myfunctions}tagname(//honk)') My XPath class already supports that for elements, but it is not currently done for extension functions (and not in the two original evaluators either, for backward compatibility). Personally, I'd prefer removing the current extension support completely, since it makes the code rather complicated. Providing two different APIs to the user and then merging the functions from both of them internally is not exactly elegant. Stefan

4 7

[lxml-dev] pb installation lxml
by Eric Ferrante 30 Dec '05

30 Dec '05

hello, I can't install lxml. I get gcc 4.0 error. I have installed the patches for Pyrex, but I still get 6 errors. My system is Fedora (last version). Coul'd you help me please ? If you need more details, tell me. Eric Ferrante ---------------------------------------------------------------- This message was sent using IMP, the Internet Messaging Program.

1 0

[lxml-dev] Encapsulating text in CDATA
by Matias 28 Dec '05

28 Dec '05

Hi Everyone, How can I use CDATA blocks to encapsulate text? I try to add the CDATA identifiers manually, but it doesn't work. Is there a way to do this? I didn' t find anything anything about this on the lxml library documentation. I've just found that the libxml has a function to do this xmlNewCDataBlock, but i didn't find a wrapper for the ElementTree api. Thank you! Matias Gonzalez

3 2

[lxml-dev] Re: ElementTree in the standard library!
by Dennis Benzinger 25 Dec '05

25 Dec '05

Philipp von Weitershausen wrote: > Stefan Behnel wrote: >> +1 for this suggestion, though. I don't think there are any modules in stdlib >> that use upper-case names. > > Unfortunately there are: > > Tkinter, UserDict, UserList, UserString, cPickle, StringIO, cStringIO, > SocketServer, {Base|Simple|CGI}HTTPServer, Cookie, > {Simple|Doc}XMLRPCServer, MimeWriter, HTMLParser, and probably some more. > > That list may seem long, but compared to the actual number of modules > inside the stdlib, it's quite small. Still, it would be nicer if the > modules-are-lower-case rule was followed better and were perhaps even > immortalized in a style PEP. > [...] This rule is in PEP 8 <http://www.python.org/peps/pep-0008.html>. Bye, Dennis

2 1

[lxml-dev] Re: scoder2 Namespace Extensions (Was Re: SimpleXMLWriter vs. lxml performance)
by Stefan Behnel 23 Dec '05

23 Dec '05

Paul Everitt wrote: > Stefan Behnel wrote: > BTW, anybody interested in having an lxml channel on Freenode? Or, is > there already one? There is one, it's called #lxml >> I already implemented something called an XPathModel, subclassing >> ElementBase, that is basically a meta-model based on XPath. You write >> XPath expressions into the class body and they are transformed into >> attributes and access/modifier methods at class creation time. I also >> have an autoconstructor, that creates children at first attribute access. >> So, all you'd have to do for the above, is to make honkstyles a property >> and create the necessary children on access. > > I'm going to read and re-read that a couple of times. :^) > > Any chance I can get a look at what you did? It's part of a project called 'slow' that will be on Berlios 'soon'. I just didn't have the time to upload the code yet. > I'm working with Berkeley DB XML (BDB XML) and Python on some stuff. Over > the years, I've been interested in a pattern where I can do executable > prototypes in a very productive way. Namely, I sit in a nice XML tool > (like oXygen), whip up some mock XML, some mock XSLT, and iterate until I > have what I like. I would like to then drop that into a "system" that > pulls in data from a database, but keeps the productivity of the mockup > stuff. > > As such, I'd really be interested in not having a hard requirement on lxml, > nor even extension functions, for the part that pulls in sample data. > Instead, I'd like to simply have <model> or <db> in an XML file, with some > dummy data underneath, and the <model> element is dynamically generated > when evaluated in my system. > > If this publishing pipeline requires lxml or extensions, I can't do the > rapid prototyping with other tools. > > I originally hoped to tackle this with custom resolvers. But those aren't > available yet. > > Next, I planned to have the input document generated from Python in the > pipeline, vs. read statically from disk during prototyping. This, though, > would mean that changing your mind about the input document contents would > mean changing Python code. (Not too bad, but not very declarative either.) > It would also mean grabbing all the nodes that might possibly be needed, > instead of only getting the ones you traverse. > > Finally, I hoped to have XSLT extension functions where you pass in the > XQuery and get back node sets. > > Along comes your element binding work, and gives me yet another way to look > at it. :^) > > At the end of the day, my central goal is to increase productivity by > allowing XML studios to help. My second goal is to find clear, but > extensible, ways to pull in foreign data on-demand. See my other mail on a way how I think this could be done. Stefan

2 1

[lxml-dev] Ebbeding image data in XML
by Werner F. Bruhin 21 Dec '05

21 Dec '05

I restart a thread here on this subject as in the meantime one of the SMPT servers of free.fr got onto a blacklist which means I can't post to XML-SIG anymore. Alexandre Fayolle wrote: >On Tue, Dec 20, 2005 at 07:33:29PM +0100, Werner F. Bruhin wrote: > > >>I would like to embed an image in an XML file. >> >>I was thinking of using PIL.tostring() and put it into the imageasstring >>element. >> >><imagedata> >> <imagemode>RGB</imagemode> >> <imagesize>(540, 982)</imagesize> >> <imageasstring>HERE GOES IMAGE DATA</imageasstring> >></imagedata> >> >>Currently I use lxml.etree and get encoding errors when I write the >>image into the element. >> >>I would like the file to XML Schema complient, so what "xs:" type should >>one use for the imageasstring element? >> >> > >You'll need to encode the imagestring data to base64, and use the >appropriate type. > > Yeap, that works. Also the resulting file is huge (e.g. a 88kb JPEG results in a XML of over 2MB). I brought this down by doing a PIL.image.thumbnail, but are there some other ways to reduce the XML file size? I also noticed that "</imageasstring>" is "overlapped" by the last line of the image data. This happens if I use (using lxml 0.8): def addImageElement(doc, name, value): new = et.SubElement(doc, name) new.text = value return new If I do this: def addImageElement(doc, name, value): new = et.SubElement(doc, name) new.text = value + '\n' return new Then it shows fine in Altova XMLSpy, but I wonder if this is the right thing to do. Another question I have is using the above solution (PIL and base64 encoding) is the data (within "imageasstring") usable on other developement platforms, e.g. Delphi, Java. Could they read this data and create an image file from it and vic versa put data into the XML with whatever tools they have and I could read it and create an image file. Thanks Werner P.S. is there some rough road map for future releases?

2 4

Re: [lxml-dev] Re: SimpleXMLWriter vs. lxml performance
by Stefan Behnel 21 Dec '05

21 Dec '05

Paul Everitt wrote: > On Dec 21, 2005, at 1:10 PM, Stefan Behnel wrote: > >> If you call the xslt processor from lxml (at least in the current >> scoder2 >> branch), it will use extension functions just as in XPath itself. It >> uses the >> same infrastructure behind the scenes. > > Hmmm, wait, this might be all that I need. > > In XSLT, I don't need the dynamically-pulled-in-nodes in the original > tree. I just need to grab them and apply rules, generating new stuff > for the output tree. > > The only mild issue, under-the-hood: will the XSLT processor care if it > suddenly starts dealing with new nodes, presumably in their own > ElementTree/document? > > It shouldn't, as XSLT has the document() function which can also do this. > > Would it also be possible, in XSLT, to have arguments? For example, at > the top of an XSLT: > > <xsl:variable "model" select="f:query('some xquery statement here')"/> > > And finally...do you know if EXSLT extensions are available in lxml? Only if they are available in libxslt by default, but I'm not sure. Anyway, I don't know if it's a good idea to use XSLT here, since there may be too many 'ifs' and 'maybes'. But using element classes, you could do something like this (untested): --------------------------- class MyDataFiller(ElementBase): def _init(self): child = self[0] if child.tag == 'sqlquery': query = child.text del self[0] # remove sqlquery element to prevent running this twice # run SQL query, generate result child nodes for it Namespace('ns')['mydata'] = MyDataFiller xml = XML(""" <myroot xmlns='ns'> <mydata> <sqlquery>SELECT from ...</sqlquery> </mydata> </myroot> """) data = xml[0] # will call _init() for data_child in data: # do something with data --------------------------- I said there was no way to have a constructor, but there actually is an _init() method that can be overridden and that will be called after instance initialization (which is a bit after instance creation, so this is different from __init__). But you have to assure that it is either harmless to call it multiple times or that it does something that is reflected in the underlying XML to assure it is only executed once (like I do above). Stefan

2 2

[lxml-dev] SimpleXMLWriter vs. lxml performance
by Paul Everitt 21 Dec '05

21 Dec '05

Hi. On my blog I mentioned that I was building up a tree from the portal catalog and that ElementTree's SimpleXMLWriter was 2.5 times faster than doing it via lxml. Martijn asked me to post here the code I used, as it shouldn't have been such a large difference. He was right -- sorry!!!. In the original comparison, for the lxml part I turned the pathindex string into subnodes for paths but didn't do this for the XMLWriter part. In a 1:1 comparison, lxml is actually a bit faster. For those interested, my test is below, with the results in the docstring. I'll post a correction in my blog post. (Note: I'm using the scoder2 branch.) --Paul """ A simple speed comparison of SimpleXMLWriter versus lxml for DOM node creation. Results for 50 entries: 0.0206758022308 ...XMLWriter average time 0.0190546989441 ...Etree average time Results for 5000 entries: 1.80105571747 ...XMLWriter average time 0.999091792107 ...Etree average time """ from time import time from lxml.etree import Element import cStringIO from elementtree.SimpleXMLWriter import XMLWriter entries = 5000 entry = { "id": "1092309103910930", "creator": "automaticforthepeople", "title": "An entry in the portal_catalog", "description": "A longer textual description would go here", "created": "12/25/2005 00:00:00 GMT+1", "is_folderish": "1", "portal_type": "ATFolder", "review_state": "published", "path": "/Members/automaticforthepeople/junk", } def makeXMLWriter(): f = cStringIO.StringIO() tree = XMLWriter(f) root = tree.start("catalog") for i in range(entries): tree.start("entry", id=entry['id'], creator=entry['creator'], title=entry['title'], description=entry['description'], created=entry['created'], is_folderish=entry['is_folderish'], portal_type=entry['portal_type'], review_state=entry['review_state'], ) tree.end() tree.close(root) def makeEtree(): root = Element("catalog") for i in range(entries): item = Element("entry") item.set("id", entry["id"]) item.set("creator", entry["creator"]) item.set("title", entry["title"]) item.set("description", entry["description"]) item.set("created", entry["created"]) item.set("is_folderish", entry["is_folderish"]) item.set("portal_type", entry["portal_type"]) item.set("review_state", entry["review_state"]) root.append(item) def main(): repeat = 10 # Time first start1 = time() for i in range(repeat): makeXMLWriter() print (time() - start1)/repeat, "...XMLWriter average time" # Time second start2 = time() for i in range(repeat): makeEtree() print (time() - start2)/repeat, "...Etree average time" if __name__ == "__main__": main()

3 7

lxml - The Python XML Toolkit December 2005