Mailman 3 June 2010 - lxml - The Python XML Toolkit

[lxml-dev] Need to parse python dictionaries into xml
by abhijeet thatte June 17, 2010

June 17, 2010

Hello, I am a novice Python user. I am using Python to parse some hardware specifications and create xml files from them. I generate dict of really huge sizes. (I am parsing some 10,000 register definitions.) So, it looks like : {elem1,elem2, elem3,dict1,{elem4,elem5, dict2 {elem6, elem7, dict3{.....}}}}. Is it possible to parse such dictionaries into xml without any new tags other than tags used in dictionaries. Thanks, Abhijeet

2 1

[lxml-dev] Problem with schema validation
by Isaac Wagner June 16, 2010

June 16, 2010

I'm using LXML version 2.2.6, libxml 2.7.7, and libxslt 1.1.26. I've gone through the example of validating a XML document against a schema, but I get an error every time. Here is the relevant portion of my Python code: from lxml import etree schemaDoc = etree.parse(schemaFileName) schema = etree.XMLSchema(schemaDoc) xmlDoc = etree.parse(xmlFileName) if schema.validate(xmlDoc): print "Valid" else: print schema.error_log My XML document fails with this error: Test.xsd:3:0:ERROR:SCHEMASV:SCHEMAV_CVC_ELT_1: Element '{http://www.w3.org/2001/XMLSchema}schema': No matching global declaration available for the validation root. I've run xmllint against the schema and input XML and it works fine: $ xmllint --schema Test.xsd --noout Test.xml Test.xml validates I've also loaded my schema and XML into Eclipse and run the XML and XSD validators and those work. So, I believe the problem is not in libxml nor in my documents. Can someone please shed some light on this problem?

2 1

[lxml-dev] Resolver not getting proper SYSTEM id / parsing XHTML with entities
by fantasai June 14, 2010

June 14, 2010

Hello, I'm trying to process around 10,000 XHTML files with lxml: specifically parsing them, extracting some metadata, then serializing to XHTML and HTML with html5lib. My problem is that lxml chokes on   et al. Now, I could turn on dtd_validation, and maybe (maybe) that would work, but it would also make the W3C sysadmins hate me because lxml does not cache the DTD: http://www.hoboes.com/Mimsy/hacks/caching-dtds-using-lxml-and-etree/ There is also no option to turn off entity checking and just parse their names into the tree, which would also solve my problem: https://bugs.launchpad.net/lxml/+bug/267825 So the best idea I could come up with was to borrow Jerry Stratton's code for caching the DTDs. So I copied it: class CachingDTDResolver(etree.Resolver): """ For caching DTDs. Copied from http://www.hoboes.com/Mimsy/hacks/caching-dtds-using-lxml-and-etree/ """ def resolve(self, URL, id, context): print "args %s %s %s %s" % (self, URL, id, context) ... from Utils import CachingDTDResolver __parser = etree.XMLParser(no_network=False, dtd_validation=True, remove_comments=False, strip_cdata=False, resolve_entities=False) __parser.resolvers.add(CachingDTDResolver()) And tried to parse with it: self.tree = etree.parse(self.sourcepath, parser=self.__parser) The following file (sourcepath = path/to/file.xht) <!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd"> <html xmlns="http://www.w3.org/1999/xhtml"> <head> ... Which has the correct DOCTYPE according to W3C: http://www.w3.org/TR/xhtml1/#strict And got the following debugging output: args <CSSTestLib.Utils.CachingDTDResolver object at 0x923fe8c> path/to/file.xht None <lxml.etree._ParserContext object at 0x99f4994> I double-checked the Resolver interface: resolve(...) resolve(self, system_url, public_id, context) Override this method to resolve an external source by ``system_url`` and ``public_id``. The third argument is an opaque context object. And I double-checked that lxml found the correct system ID without the custom resolver: print self.tree.docinfo.system_url "http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd" And I double-checked that the file validated at W3C: http://validator.w3.org/ And could not understand why a) I didn't get a proper system_url b) I didn't get a proper public_id c) I got the relative filepath??? End result: Entity-parsing problem not solved! :( Can anyone tell me what I'm doing wrong? Surely parsing XHTML with lxml is a solved problem! ~fantasai

2 1

[lxml-dev] Pretty print indent level
by Isaac Wagner June 14, 2010

June 14, 2010

Here's a little snippet of what I'm doing: from lxml import etree doc = etree.parse(file) text = etree.tostring(doc, pretty_print=True) ... So, that works great and outputs a nicely formatted XML document. However, the indent level of pretty_print is 2 spaces whereas I'd like 4 spaces. Is there any way to control the number of spaces per indent level?

2 1

[lxml-dev] Generating lxml objects from expat
by Arc Riley June 13, 2010

June 13, 2010

We have a C application that uses expat to parse a XML stream (XMPP). The stream only terminates when the connection closes and we need to process (and respond) to it while its open, so a DOM parser would not work. Inside the <stream/> (root element) are "stanzas", elements at depth=1 which are generally short and easy to parse, so its desirable to pass each stanza to a DOM tree for further processing in Python. We can do this with cElementTree due to the exposed expat handlers, but I have not yet seen anything in lxml's C api which would suggest that its capable of this. Is it possible to construct and populate a lxml.etree externally using startElement, endElement, etc calls in a manner which is compatible with libxslt? Its also a bit frustrating that lxml's C headers are not installed to the system (and thus unpackaged on both Gentoo and Debian/Ubuntu).

2 1

[lxml-dev] Force element content to be string
by Sebastian Rahlf June 9, 2010

June 9, 2010

Hi all! I have a bit of XML which I want to parse via lxml.objectify. from lxml import objectify node = objectify.fromstring(''' <Item> <ASIN>0747532745</ASIN> <ItemAttributes> <Author>Joanne K. Rowling</Author> <Manufacturer>Bloomsbury Publishing</Manufacturer> <ProductGroup>Book</ProductGroup> <Title>Harry Potter and the Philosopher's Stone</Title> </ItemAttributes> </Item> ''') I have the following problem: node.ASIN is evaluated to integer value 747532745 but should be a string ('0747532745'). There is no way for me to influence the incoming XML, so any py:pytype magic or adding a schema is out of the question. Is there a way to ensure that ASIN elements are always evaluated to a string? Cheers, Seb. -- Sebastian Rahlf basti(a)redtoad.de

2 3

[lxml-dev] lxml for Python 3.1
by Edgardo C. June 4, 2010

June 4, 2010

Hello everyone, I've installed Ubuntu 10.04LTS in my pc and python-lxml 2.2.4-1 is already in. I installed it, but once done it finished in the Python 2.6 already installed version. But, I'm developing for 3.1 and I would like to have lxml working for it. My question is, does the last version of lxml compile for Python 3.1? What do I need for that? I would really appreciate any help because I need it very soon. Thanks in advance, Ed.

3 2

[lxml-dev] Unable to build lxml 2.2.6 on WinXP
by Russ Johnson June 2, 2010

June 2, 2010

I'm attempting to built lxml 2.2.6 with MinGW for Python 2.7b2 on Windows XP Pro SP 3 using static linking, but the build fails with the following error. Any suggestions? C:\lxml-2.2.6>python setup.py install --static Building lxml version 2.2.6. NOTE: Trying to build without Cython, pre-generated 'src/lxml/lxml.etree.c' needs to be available. ERROR: 'xslt-config' is not recognized as an internal or external command, operable program or batch file. ** make sure the development packages of libxml2 and libxslt are installed ** Using build configuration of libxslt Building against libxml2/libxslt in one of the following directories: ..\iconv-1.9.2.win32\lib ..\libxml2-2.7.6.win32\lib ..\libxslt-1.1.26.win32\lib ..\zlib-1.2.3.win32\lib running install install_dir C:\Python27\Lib\site-packages\ running bdist_egg running egg_info writing src\lxml.egg-info\PKG-INFO writing top-level names to src\lxml.egg-info\top_level.txt writing dependency_links to src\lxml.egg-info\dependency_links.txt reading manifest file 'src\lxml.egg-info\SOURCES.txt' reading manifest template 'MANIFEST.in' warning: no files found matching 'lxml.etree.c' under directory 'src\lxml' warning: no files found matching 'lxml.objectify.c' under directory 'src\lxml' warning: no files found matching 'lxml.etree.h' under directory 'src\lxml' warning: no files found matching 'lxml.etree_api.h' under directory 'src\lxml' warning: no files found matching 'etree_defs.h' under directory 'src\lxml' warning: no files found matching 'pubkey.asc' under directory 'doc' warning: no files found matching 'tagpython*.png' under directory 'doc' writing manifest file 'src\lxml.egg-info\SOURCES.txt' installing library code to build\bdist.win32\egg running install_lib running build_py running build_ext building 'lxml.etree' extension C:\MinGW\bin\gcc.exe -mno-cygwin -mdll -O -Wall -I..\iconv-1.9.2.win32\include - I..\libxml2-2.7.6.win32\include -I..\libxslt-1.1.26.win32\include -I..\zlib-1.2. 3.win32\include -IC:\Python27\include -IC:\Python27\PC -c src/lxml/lxml.etree.c -o build\temp.win32-2.7\Release\src\lxml\lxml.etree.o -w writing build\temp.win32-2.7\Release\src\lxml\etree.def C:\MinGW\bin\gcc.exe -mno-cygwin -shared -s build\temp.win32-2.7\Release\src\lxm l\lxml.etree.o build\temp.win32-2.7\Release\src\lxml\etree.def -L..\iconv-1.9.2. win32\lib -L..\libxml2-2.7.6.win32\lib -L..\libxslt-1.1.26.win32\lib -L..\zlib-1 .2.3.win32\lib -LC:\Python27\libs -LC:\Python27\PCbuild -llibxslt_a -llibexslt_a -llibxml2_a -liconv_a -lzlib -lWS2_32 -lpython27 -lmsvcr90 -o build\lib.win32-2 .7\lxml\etree.pyd Warning: .drectve `/DEFAULTLIB:"uuid.lib" /DEFAULTLIB:"uuid.lib" /DEFAULTLIB:"MS VCRT" /DEFAULTLIB:"OLDNAMES" ' unrecognized Warning: .drectve `/DEFAULTLIB:"uuid.lib" /DEFAULTLIB:"uuid.lib" /DEFAULTLIB:"MS VCRT" /DEFAULTLIB:"OLDNAMES" ' unrecognized Warning: .drectve `/DEFAULTLIB:"uuid.lib" /DEFAULTLIB:"uuid.lib" /DEFAULTLIB:"MS VCRT" /DEFAULTLIB:"OLDNAMES" ' unrecognized Warning: .drectve `/DEFAULTLIB:"uuid.lib" /DEFAULTLIB:"uuid.lib" /DEFAULTLIB:"ws 2_32.lib" /DEFAULTLIB:"MSVCRT" /DEFAULTLIB:"OLDNAMES" ' unrecognized -- snip -- collect2: ld returned 1 exit status error: command 'gcc' failed with exit status 1

3 4