Mailman 3 February 2015 - lxml - The Python XML Toolkit

Avoiding validation errors when offline
by Charlie Clark 25 Feb '15

25 Feb '15

Hi, while travelling yesterday I got a shock when I ran my tests and they failed with a validation error: xmlschema.pxi:87: in lxml.etree.XMLSchema.__init__ (src/lxml/lxml.etree.c:174765) ??? E lxml.etree.XMLSchemaParseError: Element '{http://www.w3.org/2001/XMLSchema}element', attribute 'ref': The QName value '{http://purl.org/dc/terms/}created' does not resolve to a(n) element declaration., line 19 My machine had been playing up a bit so I initially thought this might have come from a corrupt file but this was, fortunately (or perhaps unfortunately), not the case. The error arises, I think, because a schema (see below) is parsed with an online resource. In the particular case I can probably remove the validation test as it's effectively redundant but I wondered what is the best way to deal with this. Should the test be skipped if I can somehow workout that there is no network connection? Is there a way of caching the online schema? Can lxml be allowed to fail gracefully if it can't load a remote resource? Charlie <xs:import namespace="http://purl.org/dc/elements/1.1/" schemaLocation="http://dublincore.org/schemas/xmls/qdc/2003/04/02/dc.xsd"/> <xs:import namespace="http://purl.org/dc/terms/" schemaLocation="http://dublincore.org/schemas/xmls/qdc/2003/04/02/dcterms.xsd"/> <xs:import id="xml" namespace="http://www.w3.org/XML/1998/namespace"/> <xs:element name="coreProperties" type="CT_CoreProperties"/> <xs:complexType name="CT_CoreProperties"> <xs:all> <xs:element name="category" minOccurs="0" maxOccurs="1" type="xs:string"/> <xs:element name="contentStatus" minOccurs="0" maxOccurs="1" type="xs:string"/> <xs:element ref="dcterms:created" minOccurs="0" maxOccurs="1"/> <xs:element ref="dc:creator" minOccurs="0" maxOccurs="1"/> <xs:element ref="dc:description" minOccurs="0" maxOccurs="1"/> <xs:element ref="dc:identifier" minOccurs="0" maxOccurs="1"/> <xs:element name="keywords" minOccurs="0" maxOccurs="1" type="CT_Keywords"/> <xs:element ref="dc:language" minOccurs="0" maxOccurs="1"/> <xs:element name="lastModifiedBy" minOccurs="0" maxOccurs="1" type="xs:string"/> <xs:element name="lastPrinted" minOccurs="0" maxOccurs="1" type="xs:dateTime"/> <xs:element ref="dcterms:modified" minOccurs="0" maxOccurs="1"/> <xs:element name="revision" minOccurs="0" maxOccurs="1" type="xs:string"/> <xs:element ref="dc:subject" minOccurs="0" maxOccurs="1"/> <xs:element ref="dc:title" minOccurs="0" maxOccurs="1"/> <xs:element name="version" minOccurs="0" maxOccurs="1" type="xs:string"/> </xs:all> </xs:complexType> -- Charlie Clark Managing Director Clark Consulting & Research German Office Kronenstr. 27a Düsseldorf D- 40217 Tel: +49-211-600-3657 Mobile: +49-178-782-6226

1 0

Is there a way to disable error logging?
by Александр Чекунков 24 Feb '15

24 Feb '15

Hi, is there any way to disable error logging in lxml completely. I'm not sure but I think I have some memory issue related to lxml error logging, at least when memory usage of my application getting huge heapy shows the following: ipdb> hp Partition of a set of 648522 objects. Total size = 314884560 bytes. Index Count % Size % Cumulative % Kind (class / dict of class) 0 6422 1 179466752 57 179466752 57 unicode 1 188069 29 62420432 20 241887184 77 str 2 119316 18 13363392 4 255250576 81 lxml.etree._LogEntry 3 97582 15 8826128 3 264076704 84 tuple 4 11013 2 7514616 2 271591320 86 dict (no owner) 5 24769 4 4259720 1 275851040 88 list 6 1400 0 3499328 1 279350368 89 dict of module 7 3487 1 3148856 1 282499224 90 type 8 3487 1 3043048 1 285542272 91 dict of type 9 9803 2 2744840 1 288287112 92 dict of pycparser.plyparser.Coord ipdb> log_entries = hp[2] ipdb> it = iter(log_entries.nodes) ipdb> it.next() http://www.soap.com/p/tweezerman-petite-tweeze-set-with-pink-case-1176168:1…: htmlParseEntityRef: expecting ';' ipdb> it.next() http://www.soap.com/p/tweezerman-petite-tweeze-set-with-pink-case-1176168:1…: htmlParseEntityRef: expecting ';' ipdb> it.next() http://www.soap.com/p/tweezerman-petite-tweeze-set-with-pink-case-1176168:1…: htmlParseEntityRef: expecting ';' ipdb> it.next() http://www.soap.com/p/tweezerman-petite-tweeze-set-with-pink-case-1176168:1…: htmlParseEntityRef: expecting ';' ipdb> it.next() http://www.soap.com/p/tweezerman-petite-tweeze-set-with-pink-case-1176168:1…: htmlParseEntityRef: expecting ';' ipdb> it.next() http://www.soap.com/p/tweezerman-petite-tweeze-set-with-pink-case-1176168:1…: htmlParseEntityRef: expecting ';' ipdb> it.next() http://www.soap.com/p/tweezerman-petite-tweeze-set-with-pink-case-1176168:1…: htmlParseEntityRef: expecting ';' ipdb> it.next() http://www.soap.com/p/tweezerman-petite-tweeze-set-with-pink-case-1176168:1…: htmlParseEntityRef: no name ipdb> it.next() http://www.soap.com/p/tweezerman-petite-tweeze-set-with-pink-case-1176168:1…: htmlParseEntityRef: expecting ';' heapy shows ~300 MB memory usage and total memory usage of a process was 690 MB, so I assume it can be some unreleased memory outside of python interpreter (C code?). -- Best regards, Alexander Chekunkov

3 3

egg package
by Tcll 17 Feb '15

17 Feb '15

hello, I'm using a portable python installation and do not have the means to build the source. may I request an egg distribution for python 2.7 x86 thanks :)

2 1

How to mix lxml.objectify.ElementMaker with Sub/Data/Element
by Werner 11 Feb '15

11 Feb '15

Hi, I currently use amara 2 for the little bit of XML stuff I do, would like to switch to something which I can use both with Py2 and Py3. I am getting pretty far with lxml.objectify but can't figure out the following. This works: |myE = objectify.ElementMaker(namespace="http://www.vinoxml.org/XMLschema", nsmap={None :"http://www.vinoxml.org/XMLschema"}) docO = myE.query(myE.querycreator()) docO.querycreator.name = objectify.DataElement(u"The Wine Cellar Book - version %s" % 15.1)| But when I generate the XML in question I have lots of optional elements and so would like to do something along these lines: | docO = myE.query() qc = objectify.SubElement(docO, 'querycreator') docO.querycreator.name = objectify.DataElement(u"The Wine Cellar Book - version %s" % 15.1) But I get an error 'no such child: querycreator' when trying to add 'name'. What am I doing wrong here? Appreciate any tips. Werner P.S. I asked the question on stackoverflow (http://stackoverflow.com/questions/28408065) too, I'will update it there. |

2 4

lxml 3.4.2 released
by Stefan Behnel 07 Feb '15

07 Feb '15

Hi all, I just released a bug fix version of lxml, 3.4.2, which contains a fix for a crash when creating a XMLSchema instance from a subtree of an XML document. Note that the 3.4 release series no longer supports Python versions before Py2.6 and Py3.2. Support for very old versions of libxml2 and libxslt (<=2008) was also removed. The documentation is here: http://lxml.de/ Download: http://lxml.de/files/lxml-3.4.2.tgz Signature: http://lxml.de/files/lxml-3.4.2.tgz.asc Changelog: http://lxml.de/3.4/changes-3.4.2.html Github: https://github.com/lxml/lxml/commit/c1dfce0462b3bf3930938f5d08b8f0188803efcf This release was built using Cython 0.21.1, but should also build fine with 0.20.x. If you are interested in commercial support or customisations for the lxml package, please contact me directly. Have fun, Stefan 3.4.2 (2015-02-07) ================== Bugs fixed ---------- * LP#1415907: Crash when creating an XMLSchema from a non-root element of an XML document. * LP#1369362: HTML cleaning failed when hitting processing instructions with pseudo-attributes. * ``CDATA()`` wrapped content was rejected for tail text. * CDATA sections were not serialised as tail text of the top-level element.

1 0

Re: [lxml] etree.ETCompatXMLParser(strip_cdata=False) continues to remove cdata
by George David 07 Feb '15

07 Feb '15

Oh I see. How would I determine if the text I am looking at was surrounded by CDATA tags? I'm pulling the data out of lxml into separate python classes and then at a later point I recreate the XML, most likely with some of the data changed. George On Fri Feb 06 2015 at 2:11:31 AM Stefan Behnel <stefan_ml(a)behnel.de> wrote: > George David schrieb am 06.02.2015 um 03:48: > > I am trying to preserver CDATA tags when parsing XML. But even if I set > > strip_cdata=False, the parser still removes the CDATA tags, see code > below: > > > > from lxml import etree > > from StringIO import StringIO > > > > xml=''' > > <cdata:cdata xmlns:cdata="urn:cdata"> > > <cdata:script><![CDATA[ > > var x, text; > > > > // Get the value of the input field with id="numb" > > x = document.getElementById("numb one").value; > > > > // If x is Not a Number or less than one or greater than 10 > > if (isNaN(x) || x < 1 || x > 10) { > > text = "Input not valid"; > > } else { > > text = "Input OK"; > > } > > document.getElementById("demo").innerHTML = text; > > ]]></cdata:script> > > </cdata:cdata> > > ''' > > > > parser = etree.ETCompatXMLParser(strip_cdata=False) > > > > doc = etree.parse(StringIO(xml), parser) > > > > rootNode = doc.getroot() > > for child in rootNode: > > print "####" + child.tag + "####" > > print child.text > > > > > > The output is this: > > > > ####{urn:cdata}script#### > > > > var x, text; > > > > // Get the value of the input field with id="numb" > > x = document.getElementById("numb one").value; > > > > // If x is Not a Number or less than one or greater than 10 > > if (isNaN(x) || x < 1 || x > 10) { > > text = "Input not valid"; > > } else { > > text = "Input OK"; > > } > > document.getElementById("demo").innerHTML = text; > > > > > > Am I doing something wrong? > > Yes. "CDATA" is a part of the XML structure, not of the text. So you won't > see it in the text content that lxml returns. When you serialise the > document, you'll get your CDATA back at the place where it sticks in the > tree. > > Stefan > > _________________________________________________________________ > Mailing list for the lxml Python XML toolkit - http://lxml.de/ > lxml(a)lxml.de > https://mailman-mail5.webfaction.com/listinfo/lxml >

2 1

etree.ETCompatXMLParser(strip_cdata=False) continues to remove cdata
by George David 06 Feb '15

06 Feb '15

I am trying to preserver CDATA tags when parsing XML. But even if I set strip_cdata=False, the parser still removes the CDATA tags, see code below: from lxml import etree from StringIO import StringIO xml=''' <cdata:cdata xmlns:cdata="urn:cdata"> <cdata:script><![CDATA[ var x, text; // Get the value of the input field with id="numb" x = document.getElementById("numb one").value; // If x is Not a Number or less than one or greater than 10 if (isNaN(x) || x < 1 || x > 10) { text = "Input not valid"; } else { text = "Input OK"; } document.getElementById("demo").innerHTML = text; ]]></cdata:script> </cdata:cdata> ''' parser = etree.ETCompatXMLParser(strip_cdata=False) doc = etree.parse(StringIO(xml), parser) rootNode = doc.getroot() for child in rootNode: print "####" + child.tag + "####" print child.text The output is this: ####{urn:cdata}script#### var x, text; // Get the value of the input field with id="numb" x = document.getElementById("numb one").value; // If x is Not a Number or less than one or greater than 10 if (isNaN(x) || x < 1 || x > 10) { text = "Input not valid"; } else { text = "Input OK"; } document.getElementById("demo").innerHTML = text; Am I doing something wrong? Thanks, George

2 1

CFI Support
by Jens Tröger 06 Feb '15

06 Feb '15

Hello, Does lxml support Canonical Fragment Identifiers (CFI) as defined here: http://www.idpf.org/epub/linking/cfi/epub-cfi.html What I'm looking for is generating a CFI for a given character (or string thereof) of an XML tree, or being able to insert elements based on a given CFI. Cheers, Jens -- Jens Tröger http://savage.light-speed.de/

2 1

question about validation TCF xml with rng schemas
by Bene Richárd 06 Feb '15

06 Feb '15

Hello everyone, I have an issues with validation a TCF xml file which my algorithm produces. Here is the validation part of my function for converting to tcf: rngfile = StringIO(file_handling.load_data(pathes['templates']+'TCF/d-spin-local_0_4.rng')) rngtree = lxml.etree.parse(rngfile) validator = lxml.etree.RelaxNG(rngtree) #validator built, validation begins" if validator.validate(template): #converting the xml tree to string xml_string = lxml.etree.tostring(template, pretty_print=True) file_handling.save_data(output_file, xml_string) return True else: raise Exception('not valid TCF xml format') The procedure do not go through validator.validate(template). The files are 4-5 MB large. Do you have any idea how to solve the issue? Thanks, Ricsi

2 1