[lxml-dev] ElementTree.iterparse segfault
Hi all, There's a test in selftest2.py which fails (results segfault) on my 64-bit linux host: def iterparse(): """ Test iterparse interface. >>> iterparse = ElementTree.iterparse >>> context = iterparse("samples/simple.xml") >>> for action, elem in context: ... print("%s %s" % (action, elem.tag)) end element end element end empty-element end root I've created a standalone script which contains: from lxml import etree as ElementTree iterparse = ElementTree.iterparse context = iterparse("samples/simple.xml") for i in context: print i ...and it also results segmentation fault. Where samples/simple.xml is the xml bundled with lxml: <root> <element key='value'>text</element> <element>text</element>tail <empty-element/> </root> I've tested it with python 2.6.4, 2.5.4, and lxml 2.2.2 and 2.2.4 (2.2.2 is linked with libxml2 2.7.3, libxslt 1.1.24, 2.2.4 is linked with libxml2 2.7.6). It only happens on 64-bit linux (RHEL 4), my 32-bit linux systems (RHEL 3 and RHEL 4) are fine. Any advise on this bug? Could you check that it also happens on you 64-bit linux system? Thanks for advance, Zsolt -------------------------------------------------------------------------- NOTICE: If received in error, please destroy, and notify sender. Sender does not intend to waive confidentiality or privilege. Use of this email is prohibited when received in error. We may monitor and store emails to the extent permitted by applicable law.
Cserna, Zsolt, 23.02.2010 12:25:
There's a test in selftest2.py which fails (results segfault) on my 64-bit linux host: [...] I've tested it with python 2.6.4, 2.5.4, and lxml 2.2.2 and 2.2.4 (2.2.2 is linked with libxml2 2.7.3, libxslt 1.1.24, 2.2.4 is linked with libxml2 2.7.6).
It only happens on 64-bit linux (RHEL 4), my 32-bit linux systems (RHEL 3 and RHEL 4) are fine.
Any advise on this bug? Could you check that it also happens on you 64-bit linux system?
Just tested with the current trunk on 64bit - no problem there. The test suite in the current 2.2 branch also passes for me. Could you run the test through gdb to get a stack trace of the segfault? Stefan
Hi, Thanks for the quick reply. The backtrace is the following: #0 __pyx_pf_4lxml_5etree_9iterparse___next__ (__pyx_v_self=0x2a96d12110) at src/lxml/lxml.etree.c:86174 #1 0x0000002a95725dee in PyEval_EvalFrameEx (f=0x573030, throwflag=Variable "throwflag" is not available. ) at Python/ceval.c:2195 #2 0x0000002a9572c5f5 in PyEval_EvalCodeEx (co=0x2a96040558, globals=Variable "globals" is not available. ) at Python/ceval.c:2875 #3 0x0000002a9572c772 in PyEval_EvalCode (co=Variable "co" is not available. ) at Python/ceval.c:514 #4 0x0000002a9574e60c in PyRun_FileExFlags (fp=0x501010, filename=0x7fbfffed5e "/home/zsolt/devel/python/temp/lxml_test.py", start=Variable "start" is not available. ) at Python/pythonrun.c:1273 #5 0x0000002a9574f293 in PyRun_SimpleFileExFlags (fp=0x501010, filename=0x7fbfffed5e "/home/zsolt/devel/python/temp/lxml_test.py", closeit=1, flags=0x7fbfffe80c) at Python/pythonrun.c:879 #6 0x0000002a9575a548 in Py_Main (argc=Variable "argc" is not available. ) at Modules/main.c:532 I'm using the .c files bundled in the tar file (so I haven't re-built them by cython). Zsolt
-----Original Message----- From: Stefan Behnel [mailto:stefan_ml@behnel.de] Sent: Tuesday, February 23, 2010 13:24 To: Cserna, Zsolt (IDEAS) Cc: lxml-dev@codespeak.net Subject: Re: [lxml-dev] ElementTree.iterparse segfault
Cserna, Zsolt, 23.02.2010 12:25:
There's a test in selftest2.py which fails (results segfault) on my 64-bit linux host: [...] I've tested it with python 2.6.4, 2.5.4, and lxml 2.2.2 and 2.2.4 (2.2.2 is linked with libxml2 2.7.3, libxslt 1.1.24, 2.2.4 is linked with libxml2 2.7.6).
It only happens on 64-bit linux (RHEL 4), my 32-bit linux systems (RHEL 3 and RHEL 4) are fine.
Any advise on this bug? Could you check that it also happens on you 64-bit linux system?
Just tested with the current trunk on 64bit - no problem there. The test suite in the current 2.2 branch also passes for me.
Could you run the test through gdb to get a stack trace of the segfault?
Stefan
-------------------------------------------------------------------------- NOTICE: If received in error, please destroy, and notify sender. Sender does not intend to waive confidentiality or privilege. Use of this email is prohibited when received in error. We may monitor and store emails to the extent permitted by applicable law.
Hm, depending on the -O switch of gcc I have different results. With -O0, the test passes. With -O3 what I used previously it results segfault. Is it possible that this switch caused the problem? Zsolt
-----Original Message----- From: lxml-dev-bounces@codespeak.net [mailto:lxml-dev-bounces@codespeak.net] On Behalf Of Cserna, Zsolt (IDEAS) Sent: Tuesday, February 23, 2010 14:34 To: Stefan Behnel Cc: lxml-dev@codespeak.net Subject: Re: [lxml-dev] ElementTree.iterparse segfault
Hi,
Thanks for the quick reply.
The backtrace is the following:
#0 __pyx_pf_4lxml_5etree_9iterparse___next__ (__pyx_v_self=0x2a96d12110) at src/lxml/lxml.etree.c:86174 #1 0x0000002a95725dee in PyEval_EvalFrameEx (f=0x573030, throwflag=Variable "throwflag" is not available. ) at Python/ceval.c:2195 #2 0x0000002a9572c5f5 in PyEval_EvalCodeEx (co=0x2a96040558, globals=Variable "globals" is not available. ) at Python/ceval.c:2875 #3 0x0000002a9572c772 in PyEval_EvalCode (co=Variable "co" is not available. ) at Python/ceval.c:514 #4 0x0000002a9574e60c in PyRun_FileExFlags (fp=0x501010, filename=0x7fbfffed5e "/home/zsolt/devel/python/temp/lxml_test.py", start=Variable "start" is not available. ) at Python/pythonrun.c:1273 #5 0x0000002a9574f293 in PyRun_SimpleFileExFlags (fp=0x501010, filename=0x7fbfffed5e "/home/zsolt/devel/python/temp/lxml_test.py", closeit=1, flags=0x7fbfffe80c) at Python/pythonrun.c:879 #6 0x0000002a9575a548 in Py_Main (argc=Variable "argc" is not available. ) at Modules/main.c:532
I'm using the .c files bundled in the tar file (so I haven't re-built them by cython).
Zsolt
-----Original Message----- From: Stefan Behnel [mailto:stefan_ml@behnel.de] Sent: Tuesday, February 23, 2010 13:24 To: Cserna, Zsolt (IDEAS) Cc: lxml-dev@codespeak.net Subject: Re: [lxml-dev] ElementTree.iterparse segfault
Cserna, Zsolt, 23.02.2010 12:25:
There's a test in selftest2.py which fails (results segfault) on my 64-bit linux host: [...] I've tested it with python 2.6.4, 2.5.4, and lxml 2.2.2 and 2.2.4 (2.2.2 is linked with libxml2 2.7.3, libxslt 1.1.24, 2.2.4 is linked with libxml2 2.7.6).
It only happens on 64-bit linux (RHEL 4), my 32-bit linux systems (RHEL 3 and RHEL 4) are fine.
Any advise on this bug? Could you check that it also happens on you 64-bit linux system?
Just tested with the current trunk on 64bit - no problem there. The test suite in the current 2.2 branch also passes for me.
Could you run the test through gdb to get a stack trace of the segfault?
Stefan
-------------------------------------------------------------- ------------ NOTICE: If received in error, please destroy, and notify sender. Sender does not intend to waive confidentiality or privilege. Use of this email is prohibited when received in error. We may monitor and store emails to the extent permitted by applicable law. _______________________________________________ lxml-dev mailing list lxml-dev@codespeak.net http://codespeak.net/mailman/listinfo/lxml-dev
-------------------------------------------------------------------------- NOTICE: If received in error, please destroy, and notify sender. Sender does not intend to waive confidentiality or privilege. Use of this email is prohibited when received in error. We may monitor and store emails to the extent permitted by applicable law.
Cserna, Zsolt, 23.02.2010 14:34:
Stefan Behnel wrote:
There's a test in selftest2.py which fails (results segfault) on my 64-bit linux host: [...] I've tested it with python 2.6.4, 2.5.4, and lxml 2.2.2 and 2.2.4 (2.2.2 is linked with libxml2 2.7.3, libxslt 1.1.24, 2.2.4 is linked with libxml2 2.7.6).
It only happens on 64-bit linux (RHEL 4), my 32-bit linux systems (RHEL 3 and RHEL 4) are fine.
Any advise on this bug? Could you check that it also happens on you 64-bit linux system? Just tested with the current trunk on 64bit - no problem
Cserna, Zsolt, 23.02.2010 12:25: there. The test suite in the current 2.2 branch also passes for me.
Could you run the test through gdb to get a stack trace of the segfault?
The backtrace is the following:
#0 __pyx_pf_4lxml_5etree_9iterparse___next__ (__pyx_v_self=0x2a96d12110) at src/lxml/lxml.etree.c:86174 #1 0x0000002a95725dee in PyEval_EvalFrameEx (f=0x573030, throwflag=Variable "throwflag" is not available. ) at Python/ceval.c:2195 #2 0x0000002a9572c5f5 in PyEval_EvalCodeEx (co=0x2a96040558, globals=Variable "globals" is not available. ) at Python/ceval.c:2875 #3 0x0000002a9572c772 in PyEval_EvalCode (co=Variable "co" is not available. ) at Python/ceval.c:514 #4 0x0000002a9574e60c in PyRun_FileExFlags (fp=0x501010, filename=0x7fbfffed5e "/home/zsolt/devel/python/temp/lxml_test.py", start=Variable "start" is not available. ) at Python/pythonrun.c:1273 #5 0x0000002a9574f293 in PyRun_SimpleFileExFlags (fp=0x501010, filename=0x7fbfffed5e "/home/zsolt/devel/python/temp/lxml_test.py", closeit=1, flags=0x7fbfffe80c) at Python/pythonrun.c:879 #6 0x0000002a9575a548 in Py_Main (argc=Variable "argc" is not available. ) at Modules/main.c:532
I'm using the .c files bundled in the tar file (so I haven't re-built them by cython).
You didn't write if this is with 2.2.2 or 2.2.4, but neither of the two release versions shows anything in line 86174 that could potentially segfault. You also omitted the error output of gdb, so I don't know what the actual problem is here. For now, I would suspect that it may even be a gcc problem (what version do you use?), doesn't look like anything is broken in lxml itself. Stefan
You didn't write if this is with 2.2.2 or 2.2.4, but neither of the two release versions shows anything in line 86174 that could potentially segfault. You also omitted the error output of gdb, so I don't know what the actual problem is here.
I've tried 2.2.4. In line 86174 there's an if, and I agree with you that it should not cause any problems. Weird.. Maybe -O3 caused this wrong gcc backtrace.
For now, I would suspect that it may even be a gcc problem (what version do you use?), doesn't look like anything is broken in lxml itself.
Yes, what I forgot to mention that I use different gcc on 32 bit (3.2.3) and 64-bit (3.4.5). So this also could have been the problem - now it works fine for me to compile with -O2 on 64-bit. Thanks, Zsolt -------------------------------------------------------------------------- NOTICE: If received in error, please destroy, and notify sender. Sender does not intend to waive confidentiality or privilege. Use of this email is prohibited when received in error. We may monitor and store emails to the extent permitted by applicable law.
participants (2)
-
Cserna, Zsolt
-
Stefan Behnel