Mailman 3 January 2016 - lxml - The Python XML Toolkit

[lxml-dev] lxml has its page on launchpad
by Stefan Behnel April 11, 2023

April 11, 2023

Hi all, I added the lxml project to launchpad, the Ubuntu Bug-Tracker. It also has a FAQ engine and a couple of other goodies. https://launchpad.net/lxml It's easy to sign up for launchpad, BTW, no 90%-footnotes-contract. Have fun, Stefan

9 9

[lxml-dev] Checking whether a node is a comment/element
by Geoffrey Sneddon April 10, 2023

April 10, 2023

Hi, What's the best way to check whether a given node is a comment or an element? For the former, I'm currently using isinstance(node, etree._Comment), which is rather obviously sub-optimal. -- Geoffrey Sneddon <http://gsnedders.com/>

6 6

[lxml-dev] Reparenting a node
by Lawrence Oluyede Jan. 30, 2023

Jan. 30, 2023

I have a doc A and a doc B, I'd like to put a node extracted from A in the document B but I always get a ValueError: ValueError: Element is not a child of this node. I didn't find any "setparent" in the API. How can I do this? -- Lawrence, oluyede.org - neropercaso.it "It is difficult to get a man to understand something when his salary depends on not understanding it" - Upton Sinclair

3 2

[lxml-dev] lxml 2.0.5 released
by Stefan Behnel Jan. 11, 2023

Jan. 11, 2023

Hi all, lxml 2.0.5 is on PyPI. This is a bug-fix-only release of the stable 2.0 series. Have fun, Stefan 2.0.5 (2008-05-01) Bugs fixed * Resolving to a filename in custom resolvers didn't work. * lxml did not honour libxslt's second error state "STOPPED", which let some XSLT errors pass silently. * Memory leak in Schematron with libxml2 >= 2.6.31.

3 4

[lxml-dev] Building LXML Trunk
by Sidnei da Silva Aug. 31, 2022

Aug. 31, 2022

Hi, I've tried to build lxml from trunk today, on Win32. Got the following error: src\lxml\etree.c(880) : error C2059: syntax error : ')' src\lxml\etree.c(881) : error C2059: syntax error : ')' src\lxml\etree.c(882) : error C2059: syntax error : ')' src\lxml\etree.c(883) : error C2059: syntax error : ')' Any clue? Smells like a Pyrex issue? -- Sidnei da Silva Enfold Systems http://enfoldsystems.com Fax +1 832 201 8856 Office +1 713 942 2377 Ext 214

4 4

Lxml fails to parse httpbin.org example utf-8 page
by Austin Platt Jan. 25, 2016

Jan. 25, 2016

Hello all, I was doing some tests with lxml and decided to try it out on the test response pages of httpbin.org. Lxml fails to 'correctly' parse the example utf8 example page supplied by httpbin.org. The page can be found here: http://httpbin.org/encoding/utf8. Here is a reproduction of the case: > import requests > r = requests.get("http://httpbin.org/encoding/utf8") > html = r.text > print(html) [...] > from lxml import etree > etree_parser = etree.HTMLParser(encoding='utf-8') > tree = etree.fromstring(html, parser=etree_parser) > new_html = etree.tostring(tree, method='html', encoding='utf-8') > print(new_html) [...] The new_html is truncated after a `<` character in the `pre` tag of the original response. I presume this is because lxml attempts to interpret the `<` character as the start of an html tag. Does lxml have any heuristics for deciding whether to interpret a lone `<` character as a text character as opposed to a html tag initiator? Cheers Austin

3 3

remove an element without removing its tail?
by Martin Mueller Jan. 25, 2016

Jan. 25, 2016

I found a solution that works for this example: float.tail = q.tail Is that what others would do? I still don't know what kind of animal "tail" is. It's a string. I sort of understand that an element is a list. So an XML document is a list with nested lists. That's easy to get. It's harder to see how the string of tail is globbed on to a particular list. Some element.text is the value of a list item. But how does the string "tail" know where it belongs? Of course, as long as it works, I don't really need to know... I am struggling with a tail problem. Here is the example; <sp> <speaker>Rainoldes.</speaker> <p>You may learne the reason hereof in your <hi>Por‑tesse,</hi> reformed lately by the Pope. In your olde <note n="c" place="margin"> <hi>Portiforium seu breuiarium, ad vsum ecclesiae Sarum: in festo S. Thomae Can‑••ariensis.</hi> </note> <hi>Portesse</hi> there was this prayer to the Popes martyr, <hi>S. Thomas Bec‑ket of Canterbury:</hi> <q> <floatingText> <body> <div type="version"> <l>Christe Iesu,</l> <l>per Thomae vulnera,</l> <l>Quae nos ligant.</l> <l>relaxa scelera.</l> </div> <div type="version"> <l>By Thomas woundes,</l> <l>O Christ Iesus,</l> <l>Loose thou the sinnes▪</l> <l>which do binde vs.</l> </div> </body> </floatingText> </q> Or, if you will haue better ryme, with as bad reason: <q> <pb n="480" facs="tcp:15991:239"/> <floatingText> <body> <div type="version"> <l>Tu per Thomae sanguinem</l> <l>quem pro te impendit,</l> <l>Fac nos Christe scandere</l> <l>quo Thomas a•cendit.</l> </div> <div type="version"> <l>By the blood of Thomas</l> <l>which he for thee did spend,</l> <l>Make vs O Christ to clime</l> <l>whether he did a••end.</l> </div> </body> </floatingText> </q> <q> <l>Mary had a little lamb</l> </q> </p> </sp> In this example (and many others from a larger collection) I want to strip <q> as unnecessary wrappers. In this particular case, I could use strip_tags. But this wouldn't work in cases where q does not wrap a floatingText element. In many cases, it would be simple to do something like q.addprevious(floatingText) parent = q.getparent() parent.remove(q) But this code removes the tail of q, which I want to keep. How do I remove the empty q element without removing the tail as well? MM

2 1

remove an element without removing its tail?
by Martin Mueller Jan. 23, 2016

Jan. 23, 2016

I am struggling with a tail problem. Here is the example; <sp> <speaker>Rainoldes.</speaker> <p>You may learne the reason hereof in your <hi>Por‑tesse,</hi> reformed lately by the Pope. In your olde <note n="c" place="margin"> <hi>Portiforium seu breuiarium, ad vsum ecclesiae Sarum: in festo S. Thomae Can‑••ariensis.</hi> </note> <hi>Portesse</hi> there was this prayer to the Popes martyr, <hi>S. Thomas Bec‑ket of Canterbury:</hi> <q> <floatingText> <body> <div type="version"> <l>Christe Iesu,</l> <l>per Thomae vulnera,</l> <l>Quae nos ligant.</l> <l>relaxa scelera.</l> </div> <div type="version"> <l>By Thomas woundes,</l> <l>O Christ Iesus,</l> <l>Loose thou the sinnes▪</l> <l>which do binde vs.</l> </div> </body> </floatingText> </q> Or, if you will haue better ryme, with as bad reason: <q> <pb n="480" facs="tcp:15991:239"/> <floatingText> <body> <div type="version"> <l>Tu per Thomae sanguinem</l> <l>quem pro te impendit,</l> <l>Fac nos Christe scandere</l> <l>quo Thomas a•cendit.</l> </div> <div type="version"> <l>By the blood of Thomas</l> <l>which he for thee did spend,</l> <l>Make vs O Christ to clime</l> <l>whether he did a••end.</l> </div> </body> </floatingText> </q> <q> <l>Mary had a little lamb</l> </q> </p> </sp> In this example (and many others from a larger collection) I want to strip <q> as unnecessary wrappers. In this particular case, I could use strip_tags. But this wouldn't work in cases where q does not wrap a floatingText element. In many cases, it would be simple to do something like q.addprevious(floatingText) parent = q.getparent() parent.remove(q) But this code removes the tail of q, which I want to keep. How do I remove the empty q element without removing the tail as well? MM

1 0

from lxml cimport etree
by Wietse Jacobs Jan. 22, 2016

Jan. 22, 2016

Hello, I'm experimenting with cython and lxml. I would like to "cimport" lxml in my own project (parsing XBRL files), but when I build my extension I get the following error: ---------------------------------------------------------------------------------------------------- cyxbrl\cy_xbrl.pyx:5:0: 'lxml.pxd' not found Error compiling Cython file: ------------------------------------------------------------ ... # encoding: utf-8 import os from lxml cimport etree ^ ------------------------------------------------------------ cyxbrl\cy_xbrl.pyx:5:0: 'lxml\etree.pxd' not found Traceback (most recent call last): File ".\cysetup.py", line 24, in <module> setup(ext_modules=cythonize("cyxbrl/*.pyx")) File "C:\Users\Wietse\projects\pyenv35\cython\lib\site-packages\Cython\Build\Dependencies.py", line 877, in cythonize cythonize_one(*args) File "C:\Users\Wietse\projects\pyenv35\cython\lib\site-packages\Cython\Build\Dependencies.py", line 997, in cythonize_ one raise CompileError(None, pyx_file) Cython.Compiler.Errors.CompileError: cyxbrl\cy_xbrl.pyx ---------------------------------------------------------------------------------------------------- I've been able to build lxml on windows from source - with some hacks in the setup.py and setupinfo.py scripts - with the following command: python .\setup.py build_ext -i --with-cython --static (I've disabled the automatic download of libxml and libxslt because I wanted to use my own 64 bit build). I've set PYTHONPATH to the lxml.src directory. Everything works fine if I use a normal python import. (This is on Windows 10 with Visual Studio 2015 Community Edition and python 3.5 - 64 bit). Is there a way to fix this? Thanks, Wietse

2 4

Null character problems
by Martin Mueller Jan. 22, 2016

Jan. 22, 2016

In running an lxml on some3,000 TEI files, about 2,000 ended up with string of null characters( \x00) after the closing </TEI> element. They generate the error message "Content is not allowed in training section."The problem doesn't seem to be random. While I can't see the difference between the input files(all of them validate with Jing), files that don't generate the\x00 characters the first time, also don't do it the second time, but files that do it the first time do it the second time. I can remove the offending character manually, but that doesn't scale to 3,000 files. Is there a way of checking for the characters before or after the final serialization and getting rid of them? Martin Mueller

1 0

Building lxml statically for MeVisLab
by Corvus Corax Jan. 21, 2016

Jan. 21, 2016

Hi, I'm trying to build lxml for MeVisLab 2.7 on Windows 7 x64. Therefore I'm using the Python 2.7 delivered with MeVisLab and MS Visual Studio 2013 Professional. As my workplace is behind a proxy I had to configure the setup.py and buildlibxml.py to use the already downloaded libraries. How I did this is described in the appended build-changes.txt. So when trying to build lxml it fails with the output in build.log. It looks like it's trying to rebuild lxml.etree, but does it even have to? Would you please help me with this? I don't know what to do from here on. Sincerely, Corbie

3 5

a question about odd behaviour of 'break' in an lxml loop with iterchildren and itersiblings
by Martin Mueller Jan. 15, 2016

Jan. 15, 2016

I am puzzled by the following behaviour of lxml. I want to group element children with the same tag. Here is a schematic representation of my text: <sp xml:id="sp-0645"> <speaker xml:id="spk-0645"/> <ab xml:id="ftln-0645" n="2.1.1"/> <ab xml:id="ftln-0646" n="2.1.2"/> <ab xml:id="ftln-0647" n="2.1.3"/> <ab xml:id="ftln-0648" n="2.1.4"/> <stage/> <ab xml:id="ftln-0703" n="2.1.59"/> <ab xml:id="ftln-0704" n="2.1.60"/> <stage/> <ab xml:id="ftln-0705" n="2.1.61"/> </sp> I want to end up with something like : <sp xml:id="sp-0645"> <speaker xml:id="spk-0645"/> <p> <lb xml:id="ftln-0645" n="2.1.1"/> <lb xml:id="ftln-0646" n="2.1.2"/> <lb xml:id="ftln-0647" n="2.1.3"/> <lb xml:id="ftln-0648" n="2.1.4"/> </p> <stage/> <p> <lb xml:id="ftln-0703" n="2.1.59"/> <lb xml:id="ftln-0704" n="2.1.60"/> </p> <stage/> <p> <lb xml:id="ftln-0705" n="2.1.61"/> </p> </sp> I try to do this through a combination of iterchildren() and itersiblings() and use the following script: speech = 'speech.xml' tree = etree.parse(speech) for sp in tree.iter('sp'): for child in sp.iterchildren(): if child.tag != 'ab': print('child', child.tag) for sibling in child.itersiblings(): if sibling.tag == 'ab': print('sibling'. sibling.tag) else: break The output of this script suggests that the two loops work properly: child speaker sibling ab sibling ab sibling ab sibling ab child stage sibling ab sibling ab child stage sibling ab Now I complicate things by creating a paragraph element with each child element. The printout shows that each of the paragraphs exists and that after the break statement the program returns to the top child loop: child speaker sibling ab sibling ab sibling ab sibling ab else <Element p at 0x1014cb548> child stage sibling ab sibling ab else <Element p at 0x1014cb508> child stage sibling ab elif <Element p at 0x1014cb608> But if I now add code to the script that fills the created paragraphs with ab elements, the program does this correctly for the first iteration but then it exits. Here is the code and the result: speech = 'speech.xml' tree = etree.parse(speech) for sp in tree.iter('sp'): for child in sp.iterchildren(): if child.tag != 'ab': paragraph = etree.Element('p') print('child', child.tag) for sibling in child.itersiblings(): if sibling.tag == 'ab' and sibling.getnext()is not None: print('sibling', sibling.tag) paragraph.append(sibling) elif sibling.tag == 'ab' and sibling.getnext() is None: print('sibling', sibling.tag) paragraph.append(sibling) print('elif', etree.tostring(paragraph, encoding='unicode', pretty_print=True)) else: print('else',etree.tostring(paragraph, encoding='unicode', pretty_print=True)) Break child speaker sibling ab sibling ab sibling ab sibling ab else <p><ab xml:id="ftln-0645" n="2.1.1"/> <ab xml:id="ftln-0646" n="2.1.2"/> <ab xml:id="ftln-0647" n="2.1.3"/> <ab xml:id="ftln-0648" n="2.1.4"/> </p> Why does the program execute only the first iteration of the loop and exit completely after the 'break', when it doesn't do that in the structurally identical but simpler versions?

2 2

a question about iter
by Martin Mueller Jan. 15, 2016

Jan. 15, 2016

Is there a way of telling etree to iterate only over the immediate children of an element? I know I can do it by enumerating the elements that occur as chid element. In my particular case, it would be simpler if I could exclude elements, as in "cycle through everything except 'w', 'c', and 'pc'. On But it would be more elegant to say "cycle through the elements just one level below the current element". Is that possible? One possibility might be to use the getchildren() function, but I'm not sure whether that's the best way.

2 1

help
by balavignesh Jan. 10, 2016

Jan. 10, 2016

I am trying to build lxml package from the following link https://pypi.python.org/packages/source/l/lxml/lxml-3.5.0.tar.gz#md5=9f0c5f… using pypy. *$ pypy setup.py build* I run this pypy in a virtualenv. I am using pypy 4.0.1 and lxml 3.5.0. The build fails with the following error: cc -O2 -fPIC -Wimplicit -I/usr/include/libxml2 -Isrc/lxml/includes -I/root/venv/pypy/include -c src/lxml/lxml.etree.c -o build/temp.linux-x86_64-2.7/src/lxml/lxml.etree.o -w src/lxml/lxml.etree.c: In function '__Pyx_call_return_trace_func': src/lxml/lxml.etree.c:4323:13: error: 'PyThreadState' has no member named 'tracing' tstate->tracing++; ^ src/lxml/lxml.etree.c:4324:13: error: 'PyThreadState' has no member named 'use_tracing' tstate->use_tracing = 0; ^ src/lxml/lxml.etree.c:4325:33: error: 'PyThreadState' has no member named 'c_tracefunc' if (CYTHON_TRACE && tstate->c_tracefunc) ^ src/lxml/lxml.etree.c:4326:17: error: 'PyThreadState' has no member named 'c_tracefunc' tstate->c_tracefunc(tstate->c_traceobj, frame, PyTrace_RETURN, result); ^ src/lxml/lxml.etree.c:4326:37: error: 'PyThreadState' has no member named 'c_traceobj' tstate->c_tracefunc(tstate->c_traceobj, frame, PyTrace_RETURN, result); ^ src/lxml/lxml.etree.c:4326:58: error: 'PyTrace_RETURN' undeclared (first use in this function) tstate->c_tracefunc(tstate->c_traceobj, frame, PyTrace_RETURN, result); ^ src/lxml/lxml.etree.c:4326:58: note: each undeclared identifier is reported only once for each function it appears in src/lxml/lxml.etree.c:4327:17: error: 'PyThreadState' has no member named 'c_profilefunc' if (tstate->c_profilefunc) ^ src/lxml/lxml.etree.c:4328:17: error: 'PyThreadState' has no member named 'c_profilefunc' tstate->c_profilefunc(tstate->c_profileobj, frame, PyTrace_RETURN, result); ^ src/lxml/lxml.etree.c:4328:39: error: 'PyThreadState' has no member named 'c_profileobj' tstate->c_profilefunc(tstate->c_profileobj, frame, PyTrace_RETURN, result); and this error goes on for thousands of lines. I tried googling this error and I came across this link : https://bitbucket.org/pypy/pypy/issues/1185/cpyext-missing-members-in-pythr… I am not sure if this is the same issue as mine. Any help would be appreciated. *Note*: This lxml package is successfully built using python 2.7, 3.2, 3.4. Thank you. -- Regards, *Balavignesh S*

2 1