Mailman 3 July 2014 - lxml - The Python XML Toolkit

[lxml-dev] lxml has its page on launchpad
by Stefan Behnel 11 Apr '23

11 Apr '23

Hi all, I added the lxml project to launchpad, the Ubuntu Bug-Tracker. It also has a FAQ engine and a couple of other goodies. https://launchpad.net/lxml It's easy to sign up for launchpad, BTW, no 90%-footnotes-contract. Have fun, Stefan

9 9

[lxml-dev] Checking whether a node is a comment/element
by Geoffrey Sneddon 10 Apr '23

10 Apr '23

Hi, What's the best way to check whether a given node is a comment or an element? For the former, I'm currently using isinstance(node, etree._Comment), which is rather obviously sub-optimal. -- Geoffrey Sneddon <http://gsnedders.com/>

6 6

[lxml-dev] Reparenting a node
by Lawrence Oluyede 30 Jan '23

30 Jan '23

I have a doc A and a doc B, I'd like to put a node extracted from A in the document B but I always get a ValueError: ValueError: Element is not a child of this node. I didn't find any "setparent" in the API. How can I do this? -- Lawrence, oluyede.org - neropercaso.it "It is difficult to get a man to understand something when his salary depends on not understanding it" - Upton Sinclair

3 2

[lxml-dev] lxml 2.0.5 released
by Stefan Behnel 11 Jan '23

11 Jan '23

Hi all, lxml 2.0.5 is on PyPI. This is a bug-fix-only release of the stable 2.0 series. Have fun, Stefan 2.0.5 (2008-05-01) Bugs fixed * Resolving to a filename in custom resolvers didn't work. * lxml did not honour libxslt's second error state "STOPPED", which let some XSLT errors pass silently. * Memory leak in Schematron with libxml2 >= 2.6.31.

3 4

[lxml-dev] Building LXML Trunk
by Sidnei da Silva 31 Aug '22

31 Aug '22

Hi, I've tried to build lxml from trunk today, on Win32. Got the following error: src\lxml\etree.c(880) : error C2059: syntax error : ')' src\lxml\etree.c(881) : error C2059: syntax error : ')' src\lxml\etree.c(882) : error C2059: syntax error : ')' src\lxml\etree.c(883) : error C2059: syntax error : ')' Any clue? Smells like a Pyrex issue? -- Sidnei da Silva Enfold Systems http://enfoldsystems.com Fax +1 832 201 8856 Office +1 713 942 2377 Ext 214

4 4

generating html with incremental writer
by Burak Arslan 25 Sep '14

25 Sep '14

Hi, Is there a way to pass method='html' to etree.xmlfile()? Or any other way to serialize incrementally to html? Best, Burak

2 9

lxml 3.3.5 + static deps on CentOS 6.5
by Fabio Sangiovanni 08 Aug '14

08 Aug '14

Hi everybody, I'm trying to build lxml 3.3.5 with static deps on a CentOS 6.5 box, so far without success. CentOS 6.5 ships with libxml2 2.7.6 and libxslt 1.1.26. I'd like to get the latest versions of both (libxml2 2.9.1 and libxslt 1.1.28). After a lot of experiments, I came up with the following way to install lxml: CFLAGS="$CFLAGS -fPIC -lgcrypt -ldl -lgpg-error -lrt" STATIC_DEPS=true pip2.7 install lxml -fPIC: seems to be mandatory on x86_64 (build fails otherwise) -lrt: fixes an issue with old versions of GCC (4.4.7 on CentOS 6.5), that results in 'undefined symbol clock_gettime' on import. -lgcrypt -ldl -lgpg-error: fixes similar import issues when libgcrypt is installed on the system (it's the output of libgcrypt-config --libs). But still, I get an error on import, that I couldn't be able to fix: Python 2.7.7 (default, Jun 16 2014, 15:17:40) [GCC 4.4.7 20120313 (Red Hat 4.4.7-4)] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>> from lxml import html Traceback (most recent call last): File "<stdin>", line 1, in <module> File "/usr/local/lib/python2.7/site-packages/lxml/html/__init__.py", line 42, in <module> from lxml import etree ImportError: /usr/local/lib/python2.7/site-packages/lxml/etree.so: undefined symbol: libiconv >>> Is there some safe way to get static deps installed with lxml on a CentOS 6.5 system? Am I on the right track or am I totally missing something? Should I maybe just use the system libs (their versions seems not to be recommended by the docs...)? I'm using CPython 2.7.7. Thanks for your help! -- Fabio Sangiovanni

1 1

How to iterate through nodes of xml through xslt
by varun bhatnagar 01 Aug '14

01 Aug '14

Hi, I have two xml files. *File1.xml* *<?xml version="1.0" encoding="UTF-8"?>* *<InfoTag>* *<Procedure attrProc="TestProcA" attrLevel="1">* * <downTime>* * <acceptableDownTime>* * <all/>* * </acceptableDownTime>* * <downTimePeriod time="600000000"/>* * </downTime>* * </Procedure>* * <Procedure attrProc="TestProcB" attrLevel="2">* * <downTime>* * <acceptableDownTime>* * <all/>* * </acceptableDownTime>* * <downTimePeriod time="600000000"/>* * </downTime>* * </Procedure>* *</InfoTag>* *File2.xml* *<?xml version="1.0" encoding="UTF-8"?>* *<InfoTag>* *<Procedure attrProc="TestProcC" attrLevel="3">* * <downTime>* * <acceptableDownTime>* * <all/>* * </acceptableDownTime>* * <downTimePeriod time="600000000"/>* * </downTime>* * </Procedure>* * <Procedure attrProc="TestProcD" attrLevel="4">* * <downTime>* * <acceptableDownTime>* * <all/>* * </acceptableDownTime>* * <downTimePeriod time="600000000"/>* * </downTime>* * </Procedure>* *</InfoTag>* I am trying to fetch an output file which looks like this: *Output.xml* *<InfoTag>* *<Procedure attrProc="1" attrLevel="### NOT UNIQUE ###">* * <downTime>* * <acceptableDownTime>* * <all/>* * </acceptableDownTime>* * <downTimePeriod time="600000000"/>* * </downTime>* * </Procedure>* * <Procedure attrProc="2" attrLevel="### NOT UNIQUE ###">* * <downTime>* * <acceptableDownTime>* * <all/>* * </acceptableDownTime>* * <downTimePeriod time="600000000"/>* * </downTime>* * </Procedure>* *<Procedure attrProc="3" attrLevel="### NOT UNIQUE ###">* * <downTime>* * <acceptableDownTime>* * <all/>* * </acceptableDownTime>* * <downTimePeriod time="600000000"/>* * </downTime>* * </Procedure>* * <Procedure attrProc="4" attrLevel="### NOT UNIQUE ###">* * <downTime>* * <acceptableDownTime>* * <all/>* * </acceptableDownTime>* * <downTimePeriod time="600000000"/>* * </downTime>* * </Procedure> * *</InfoTag>* The number of Procedure tag (<Procedure>) can be different every time. So I have to read this tag every time from each xml and then merge it sequentially. Can anyone tell me how to achieve this. How can I loop every Procedure tag and and append the attrProc attribute value in a sequential order? Thanks, BR, Varun

2 2

problem with removing elements: bug or feature
by Martin Mueller 31 Jul '14

31 Jul '14

I have run into the following problem with removing an element, and I can't figure out whether it's a bug or whether I'm missing something. I want to make a change to an element, depending on the next element, and delete that next element. The change and deletion occur in a function. The main code program like this: if element.getnext() is not None \ and element.getnext().get('rend') == 'superscript': element = processSuperscripts(element, tree, changelog) The function first spells out the change and then orders the deletion; parent = element.getnext().getparent() parent.remove(element.getnext()) This code works the first time, but it doesn't work on successive occasions. I have used a work around where I add a flag to the next element and delete it in a separate loop. Thus my function ends element.getnext().set('n', 'DELETE') and the main program adds the loop: for element in tree.iter(tei + 'w'): if element.get('n') == 'DELETE': parent = element.getparent() parent.remove(element) This works: all instances of the first element are properly changed and all instances of the second element are properly deleted. Is there a way of making the deletion work inside the function without a second "clean-up" loop? Martin Mueller Professor emeritus of English and Classics Northwestern University

3 4

Use the default namespace from nsmap
by Ivan Pozdeev 30 Jul '14

30 Jul '14

Hello Lxml, A googling confirmes that I'm far from the only one bitten badly by the lack of automatic usage of the default `xmlns'. I've written a wrapper that auto-prefixes the tags (like many others, I suspect. it's in attachment). Since starting usgin them by default right off the bat would be controversial, i suggest to make it an option to `xpath()' and collect user feedback for some time. We can use my code for that. It doesn't handle the case where the automatically-added prefix already exists but is written to be robust otherwise. Afterwards, we could start convincing libxml2 team to support NULL for xmlNs::prefix based on existing user experience. -- Best regards, Ivan mailto:vano@mail.mipt.ru

1 0

Dash in encoding name inconsistency, lxml 3.3.5, OSX
by J. Morris 30 Jul '14

30 Jul '14

Hi, I’ve come across a weird inconsistency with lxml running on OSX 10.9.4. Python : sys.version_info(major=2, minor=7, micro=8, releaselevel='final', serial=0) lxml.etree : (3, 3, 5, 0) libxml used : (2, 9, 1) libxml compiled : (2, 9, 1) libxslt used : (1, 1, 28) libxslt compiled : (1, 1, 28) This code should reproduce the problem: from lxml import etree parser = etree.HTMLParser(encoding='utf32’) --------------------------------------------------------------------------- LookupError Traceback (most recent call last) <ipython-input-1-c072399cda15> in <module>() 1 from lxml import etree ----> 2 parser = etree.HTMLParser(encoding='utf32') /Users/jmorris/anaconda/envs/py27/lib/python2.7/site-packages/lxml/etree.so in lxml.etree.HTMLParser.__init__ (src/lxml/lxml.etree.c:100669)() /Users/jmorris/anaconda/envs/py27/lib/python2.7/site-packages/lxml/etree.so in lxml.etree._BaseParser.__init__ (src/lxml/lxml.etree.c:93393)() LookupError: unknown encoding: ‘utf32' However, the following all work perfectly: parser = etree.HTMLParser(encoding='utf-32') parser = etree.HTMLParser(encoding='utf-16') parser = etree.HTMLParser(encoding='utf16') parser = etree.HTMLParser(encoding='utf8') parser = etree.HTMLParser(encoding='utf-8’) For some reason only the utf-32 encoding requires a dash in its encoding string. This happens in python 3.3 as well. Is this a problem in lxml or is it with one of the underlying libraries? I know it seems pretty minor, but it’s causing my tests to fail. Any thoughts? Thanks, J.

3 2

XSLT and WSGI problems
by Robert Onslow 27 Jul '14

27 Jul '14

Dear All I see that there is a well know problem with XSLT and mod_wsgi in Apache. See http://spyne.io/docs/2.10/faq.html#my-app-freezes-under-mod-*wsgi*-help <http://spyne.io/docs/2.10/faq.html#my-app-freezes-under-mod-wsgi-help> Just to report that I am using XSLT, uwsgi and nginx, and am also getting a problem. Is someone working on a solution. I am having to move off LXML completely for the moment .. Robert

2 1

Removing an element and strip space - XSLT
by varun bhatnagar 25 Jul '14

25 Jul '14

Hi, I am trying to play around with python and xslt. I have an xml and I want to transform it to another xml by deleting its one element. The xml is pasted below: <?xml version="1.0" encoding="UTF-8"?> <testNode> <nodeInfo> <nodePeriod nodeTime="600000000"/> <nodeBase base="0" /> </nodeInfo> </testNode> I want to remove the <nodeBase> tag and this is how my xsl file looks like: <?xml version="1.0" encoding="UTF-8"?> <xsl:stylesheet version="1.0" xmlns:xsl=" http://www.w3.org/1999/XSL/Transform"> <xsl:output method="xml" indent="yes"/> <xsl:template match="@*|node()"> <xsl:copy> <xsl:apply-templates select="@*|node()"/> </xsl:copy> </xsl:template> <xsl:template match="/testNode/nodeInfo/nodeBase"> </xsl:template> </xsl:stylesheet> When I execute it my output looks like this: <?xml version="1.0" encoding="UTF-8"?> <testNode> <nodeInfo> * <nodePeriod nodeTime="600000000"/>* * </nodeInfo>* </testNode> I want to strip the space between *<nodePeriod>* and *</nodeInfo>* Can anyone suggest a way out to do that? Thanks, BR, Varun

2 1

finding siblings that are not next or previous
by Martin Mueller 19 Jul '14

19 Jul '14

Is there a simple way in lxml to say things like "the third house on the block," "the next house but one," or "the last house on the block."? I understand getnext() and getprevious(), and it's possible to concatenate those, but it's not very elegant, and I'm not sure how it scales. I work with TEI documents where <w> elements alternate with <c> element, and very often what you do with a given <w> element depends on the attributes of the next <w> element, which is the "next but one" element. Martin Mueller Professor emeritus of English and Classics Northwestern University

2 1

Apache2 crashes with segmentation fault
by Elhadi Falah 17 Jul '14

17 Jul '14

Hello, We are using lxml in several of our applications with Python 2.6 and from time to time, the application stops responding after a segmentation fault error ( [notice] child pid 10544 exit signal Segmentation fault (11)), and this kind of backtrace: Jul 1 15:24:48 server1 httpd: *** glibc detected *** /usr/sbin/apache2: munmap_chunk(): invalid pointer: 0x00007f6468bf2c00 *** Jul 1 15:24:48 server1 httpd: ======= Backtrace: ========= Jul 1 15:24:48 server1 httpd: /lib/libc.so.6(+0x78bf6)[0x7f64767ecbf6] Jul 1 15:24:48 server1 httpd: /usr/lib/libxml2.so.2(xmlCopyError+0xd1)[0x7f6473311801] Jul 1 15:24:48 server1 httpd: /usr/lib/libxml2.so.2(__xmlRaiseError+0x30b)[0x7f6473312ecb] Jul 1 15:24:48 server1 httpd: /usr/lib/libxml2.so.2(+0x393e5)[0x7f64733173e5] Jul 1 15:24:48 server1 httpd: /usr/lib/libxml2.so.2(xmlParseDocument+0x2dc)[0x7f647332e5cc] Jul 1 15:24:48 server1 httpd: /usr/lib/libxml2.so.2(+0x50895)[0x7f647332e895] Jul 1 15:24:48 server1 httpd: /usr/lib/python2.6/dist-packages/lxml/etree.so(+0x8cbc2)[0x7f645691cbc2] Jul 1 15:24:48 server1 httpd: /usr/lib/python2.6/dist-packages/lxml/etree.so(+0x2c7cf)[0x7f64568bc7cf] After trying several versions of lxml we are still facing the issue.I've checked for the system memory consumption but everything looks fine to me, plenty of memory available, I don't see any process consuming abnormally. The issue is reproducible everytime when we execute the commande apache (apache2 reload or apache2 graceful). As workaround for this issue we execute apache2 restart. We've followed recommendations defined on these 2 links but we're still facing the issue. http://code.google.com/p/modwsgi/wiki/ApplicationIssues#Python_Simplified_G… http://code.google.com/p/modwsgi/wiki/ApplicationIssues#Multiple_Python_Sub… Library version: print("%-20s: %s" % ('Python', sys.version_info)) Python : (2, 6, 5, 'final', 0) print("%-20s: %s" % ('lxml.etree', etree.LXML_VERSION)) lxml.etree : (2, 3, 5, 0) print("%-20s: %s" % ('libxml used', etree.LIBXML_VERSION)) libxml used : (2, 7, 6) print("%-20s: %s" % ('libxml compiled', etree.LIBXML_COMPILED_VERSION)) libxml compiled : (2, 7, 6) print("%-20s: %s" % ('libxslt used', etree.LIBXSLT_VERSION)) libxslt used : (1, 1, 26) print("%-20s: %s" % ('libxslt compiled', etree.LIBXSLT_COMPILED_VERSION)) libxslt compiled : (1, 1, 26) Apache 2.2.14 Here is the source code that generate the issue: ID_TRANSFORM = os.environ['APPLICATION_WORKING_PATH']+'/statics/xsl/list.xsl' styledoc = lxml.etree.parse(ID_TRANSFORM) transform = lxml.etree.XSLT(styledoc) doc_root = lxml.etree.XML(str(atom)) Could you help us on this case? Regards

2 2

How to set xml:base attribute with ElementMaker?
by Arne Neumann 16 Jul '14

16 Jul '14

Dear all, I have to generate XML files that contain snippets like this one, but I can't find a way to produce a "xml:base" attribute: <markList xmlns:xlink="http://www.w3.org/1999/xlink" type="tok" xml:base="maz-1423.text.xml"> <mark id="sTok1" xlink:href="#xpointer(string-range(//body,'',1,3))" /> <mark id="sTok2" xlink:href="#xpointer(string-range(//body,'',5,10))" /> </markList> I tried to setup the namespaces, but instead of "xml:base", I'll only get "ns0:base". NSMAP={None: 'xml', 'xlink': 'http://www.w3.org/1999/xlink', 'xml': 'xml'} E = ElementMaker(nsmap=NSMAP) etree.tostring(E("markList", {'type': 'tok', '{%s}base' % NSMAP['xml']: 'maz-1423.text.xml'})) '<markList xmlns:xlink="http://www.w3.org/1999/xlink" xmlns="xml" xmlns:ns0="xml" ns0:base="maz-1423.text.xml" type="tok"/>' I'd appreciate any help with this. Best regards, Arne Neumann

2 1

Re: [lxml] ImportError when importing etree from lxml
by Scott Young 14 Jul '14

14 Jul '14

Thank you for the pointers, Stefan. - xsltproc works for me, although I have only used it for a simple example - I tried ldd as you suggested, and it does seem to call a link to the installed library, libxml2.so.2.9.1 - I have tried uninstalling and installing libxslt1.1, libxlst1-dev, libxml2, libxml2-dev, and lxml, and then re-installing. I get the same error. Please let me know if anyone has any other suggestions. I will now also post to the xslt list to see what I can find there. Scott

1 0

Can I change maxvars?
by Sam Bull 14 Jul '14

14 Jul '14

I'm trying to process some XML files, and a few of them are several thousand lines long, and with the moderately complicated XSL I'm using, I seem to be hitting recursion limits. I'm currently getting this message: lxml.etree.XSLTApplyError: xsltApplyXSLTTemplate: A potential infinite template recursion was detected. You can adjust maxTemplateVars (--maxvars) in order to raise the maximum number of variables/params (currently set to 15000). It says I can adjust the value, but doesn't explain how, nor is this value mentioned anywhere in the documentation. I've just had to change the maxdepth, which can be done with XSLT.set_global_max_depth(), but there doesn't appear to be an equivalent for maxvars. How can I change this value?

2 1

ImportError when importing etree from lxml
by Scott Young 12 Jul '14

12 Jul '14

Hello, I am trying to run lxml (in order to run python-docx). I can import lxml okay, but whenever I try to import etree from lxml, I get an ImportError: ImportError: /usr/lib/x86_64-linux-gnu/libxslt.so.1: symbol xmlBufUse, version LIBXML2_2.9.0 not defined in file libxml2.so.2 with link time reference Here is my configuration information: - Python 2.7.6 running with Enthought Canopy. - Ubuntu 14.04 on VMWare Worstation 10. - lxml 3.3.5 - libxml2 2.9.1+dfsg1-3ubuntu4 - libxslt1 1.1.28-2build1 It seems that it is calling for version 2.9.0 of libxml2, but I'm not sure why, because it appears that 2.9.1 has been out for a while. I also cannot tell whether this is a problem with lxml, libxslt, or libxml2, so I will post this to each of the lists. Any guidance or suggestions would be appreciated! Thank you! Thanks, Scott

2 1

Problem of charging TEI.rng
by Kun JIN 11 Jul '14

11 Jul '14

Hello Everyone, I'm trying to use the method from lxml website to charge a relaxNG file into lxml : tei_cmr.rng (https://www.dropbox.com/s/9y4aicw5ytg2v4x/tei_cmr.rng.zip). but i was blocked at rng = etree.RelaxNG(rng_doc) i don't know why, i got nothing, even not error. My code: >>> from lxml import etree >>> f = open("tei_cmr.rng","r") >>> n = etree.parse(f) >>> rng = etree.RelaxNG(n) I can't use any XML editor to validate my XML file, cause too big. so i have to use this method, could you give me some solution? Thank you in advance, Best regards Kun

2 1

Windows exes/wheels for 3.3 and 3.4?
by Chris “Kwpolska” Warrick 07 Jul '14

07 Jul '14

Hello, I tried to install lxml on Windows from PyPI, but installer exes and wheels are available for 3.2 only, and there are no 3.3/3.4 builds there. Why is that? Will they ever be available? -- Chris “Kwpolska” Warrick <http://kwpolska.tk> PGP: 5EAAEA16 stop html mail | always bottom-post | only UTF-8 makes sense

3 2

Re: [lxml] lxml.etree.XMLSyntaxError in process 1Gig file
by junkmail＠visumpoint.com 05 Jul '14

05 Jul '14

2 1

lxml.etree.XMLSyntaxError in process 1Gig file
by junkmail＠visumpoint.com 04 Jul '14

04 Jul '14

1 0

Re: [lxml] Extra content at the end of the document
by Stefan Behnel 04 Jul '14

04 Jul '14

junkmail(a)visumpoint.com, 04.07.2014 04:42: > I am not familiar with using iterparse. Are there examples/documentation you > can direct me to? https://duckduckgo.com/?q=lxml+iterparse Stefan

1 0

C-extension and nogil
by Anton Patrushev 02 Jul '14

02 Jul '14

Hi Everyone, We are trying to create c-extension to offload XML parsing from our main thread. We need lxml.etree.Element as result of parsing. Parsing will be done in separate threads (pool of) from in-memory string as input source. How we can easily avoid acquiring GIL in our parsing threads (from pool). Is there any code samples or may You can suggest some directions in lxml code base to look for? AFAICS, lxml release GIL only internally after some preparation steps, so if we want to use fromstring we must acquire GIL before. Thanks in advance, Anton

2 1