[lxml-dev] PyThread_release_lock segmentation fault

Hello, I'm getting segmentation faults inside PyThread_release_lock when trying to use lxml 1.1.2. The various stack traces I have seen seem to vary, but they are all related to "__xmlRaiseError () from /usr/lib/libxml2.so.2" (I haven't produced very many test cases, so there might be other causes). The original lxml install was done with easy_install, but I get the same results when I test the source directly. I'm using libxml2-2.6.27, gcc 3.4.3, and python 2.4 on a relatively recent redhat install. I am able to use the standard libxml2 python bindings, so I don't think there is an issue with the libxml2 install. I installed the svn trunk-and-pyrex, and was able to compile and run make test, but the tests failed with a number of errors that seemed to be expected for using the dev trunk (wrong functionality, not segfaults). lxml 1.1.1 also segfaults for me, which leads me to believe there might be a bugfix on the trunk that hasn't been released yet. If so, is there a revision that contains the change that works? Below is a log of my session (the most interesting part is probably the gdb stack trace down at the bottom). Here are the commands I entered: gcc -v python ^D make clean make test gdb python run test.py bt Can anyone tell if this is a problem on my end, or on lxml's? Is there any more information that I can provide that might be of use? Sorry if all this is a bit verbose. :) Thanks, Eli --- elis@mapdev4:~/edit/external/tarball/lxml-1.1.2/src/lxml> gcc -v Reading specs from /usr/lib/gcc/i386-redhat-linux/3.4.3/specs Configured with: ../configure --prefix=/usr --mandir=/usr/share/man --infodir=/usr/share/info --enable-shared --enable-threads=posix --disable-checking --with-system-zlib --enable-__cxa_atexit --disable-libunwind-exceptions --enable-java-awt=gtk --host=i386-redhat-linux Thread model: posix gcc version 3.4.3 20050227 (Red Hat 3.4.3-22.1) elis@mapdev4:~/edit/external/tarball/lxml-1.1.2/src/lxml> python Python 2.4 (#1, Nov 30 2004, 11:25:14) [GCC 3.4.2 20041017 (Red Hat 3.4.2-6.fc3)] on linux2 Type "help", "copyright", "credits" or "license" for more information.
elis@mapdev4:~/edit/external/tarball/lxml-1.1.2> make clean find . \( -name '*.o' -o -name '*.c' -o -name '*.so' -o -name '*.py[cod]' -o -name '*.dll' \) -exec rm -f {} \; rm -rf build elis@mapdev4:~/edit/external/tarball/lxml-1.1.2> make test python setup.py build_ext -i Building lxml version 1.1.2 running build_ext building 'lxml.etree' extension creating build creating build/temp.linux-i686-2.4 creating build/temp.linux-i686-2.4/src creating build/temp.linux-i686-2.4/src/lxml gcc -pthread -fno-strict-aliasing -DNDEBUG -g -O3 -Wall -Wstrict-prototypes -fPIC -I/usr/include/python2.4 -c src/lxml/etree.c -o build/temp.linux-i686-2.4/src/lxml/etree.o -w -I/usr/local/include -I/usr/local/include/libxml2 gcc -pthread -shared build/temp.linux-i686-2.4/src/lxml/etree.o -o src/lxml/etree.so -L/usr/local/lib -lexslt -lxslt -lxml2 -lz -lm building 'lxml.objectify' extension gcc -pthread -fno-strict-aliasing -DNDEBUG -g -O3 -Wall -Wstrict-prototypes -fPIC -I/usr/include/python2.4 -c src/lxml/objectify.c -o build/temp.linux-i686-2.4/src/lxml/objectify.o -w -I/usr/local/include -I/usr/local/include/libxml2 gcc -pthread -shared build/temp.linux-i686-2.4/src/lxml/objectify.o -o src/lxml/objectify.so -L/usr/local/lib -lexslt -lxslt -lxml2 -lz -lm python test.py -p -v TESTED VERSION: Python: (2, 4, 0, 'final', 0) lxml.etree: (1, 1, 2, 0) libxml used: (2, 6, 16) libxml compiled: (2, 6, 27) libxslt used: (1, 1, 11) libxslt compiled: (1, 1, 19) 1/656 ( 0.2%): test_attribute_based_lookup (lxml.tests.test_classlookup.ClassLookupTestCase)make: *** [test_inplace] Segmentation fault elis@mapdev4:~/edit/external/tarball/lxml-1.1.2> gdb python GNU gdb Red Hat Linux (6.3.0.0-0.31rh) Copyright 2004 Free Software Foundation, Inc. GDB is free software, covered by the GNU General Public License, and you are welcome to change it and/or distribute copies of it under certain conditions. Type "show copying" to see the conditions. There is absolutely no warranty for GDB. Type "show warranty" for details. This GDB was configured as "i386-redhat-linux-gnu"...(no debugging symbols found) Using host libthread_db library "/lib/tls/libthread_db.so.1". (gdb) run test.py Starting program: /usr/local/bin/python test.py (no debugging symbols found) [Thread debugging using libthread_db enabled] [New Thread -1207994688 (LWP 25687)] (no debugging symbols found) (no debugging symbols found) (no debugging symbols found) (no debugging symbols found) (no debugging symbols found) (no debugging symbols found) (no debugging symbols found) TESTED VERSION: Python: (2, 4, 0, 'final', 0) lxml.etree: (1, 1, 2, 0) libxml used: (2, 6, 16) libxml compiled: (2, 6, 27) libxslt used: (1, 1, 11) libxslt compiled: (1, 1, 19) Program received signal SIGSEGV, Segmentation fault. [Switching to Thread -1207994688 (LWP 25687)] 0x00a10b4a in sem_post@GLIBC_2.0 () from /lib/tls/libpthread.so.0 (gdb) bt #0 0x00a10b4a in sem_post@GLIBC_2.0 () from /lib/tls/libpthread.so.0 #1 0x080de5e2 in PyThread_release_lock () #2 0x0012430f in __pyx_f_5etree__receiveError (__pyx_v_c_log_handler=0xb7c42bcc, __pyx_v_error=0x9b5e308) at src/lxml/etree.c:15789 #3 0x00383ccc in __xmlRaiseError () from /usr/lib/libxml2.so.2 #4 0x0038525e in xmlKeepBlanksDefault () from /usr/lib/libxml2.so.2 #5 0x0040e58f in xmlSkipBlankChars () from /usr/lib/libxml2.so.2 #6 0x0041204a in xmlParseElement () from /usr/lib/libxml2.so.2 #7 0x003749ae in xmlParseDocument () from /usr/lib/libxml2.so.2 #8 0x00374b71 in xmlCleanupParser () from /usr/lib/libxml2.so.2 #9 0x00186726 in __pyx_f_5etree_11_BaseParser__parseDoc (__pyx_v_self=0x0, __pyx_v_c_text=0xb7fda8d4 "<root xmlns=\"myNS\" xmlns:other=\"otherNS\">\n <c1 a1=\"A1\" a2=\"A2\" other:a3=\"A3\">\n <c2 a1=\"C2\">0</c2>\n <c2>1</c2>\n <other:c2>2</other:c2>\n </c1>\n</root>", __pyx_v_c_len=159, __pyx_v_c_filename=0x0) at src/lxml/etree.c:20945 #10 0x00137132 in __pyx_f_5etree__parseMemoryDocument (__pyx_v_text=0xb7fda8c0, __pyx_v_url=0x8118bc0, __pyx_v_parser=0xb7c4466c) at src/lxml/etree.c:22182 #11 0x0016accc in __pyx_f_5etree_XML (__pyx_self=0x0, __pyx_args=0xb7ad0bac, __pyx_kwds=0x0) at src/lxml/etree.c:9712 #12 0x080b5122 in PyEval_EvalFrame () #13 0x080b569c in PyEval_EvalFrame () #14 0x080b5e22 in PyEval_EvalCodeEx () #15 0x080fcd9a in function_call () #16 0x0805bdb0 in PyObject_Call () #17 0x080641d5 in instancemethod_call () #18 0x0805bdb0 in PyObject_Call () #19 0x0808f352 in slot_tp_call () #20 0x0805bdb0 in PyObject_Call () #21 0x080b0e6d in PyEval_EvalFrame () #22 0x080b5e22 in PyEval_EvalCodeEx () #23 0x080fcd9a in function_call () #24 0x0805bdb0 in PyObject_Call () #25 0x080641d5 in instancemethod_call () #26 0x0805bdb0 in PyObject_Call () #27 0x0808f352 in slot_tp_call () #28 0x0805bdb0 in PyObject_Call () #29 0x080b0e6d in PyEval_EvalFrame () #30 0x080b569c in PyEval_EvalFrame () #31 0x080b569c in PyEval_EvalFrame () #32 0x080b5e22 in PyEval_EvalCodeEx () #33 0x080b5f8a in PyEval_EvalCode () #34 0x080d9f4c in PyRun_FileExFlags () #35 0x080da82e in PyRun_SimpleFileExFlags () #36 0x0805560f in Py_Main () #37 0x08054e3f in main ()

Hi, lxml-dev-bounces@codespeak.net schrieb am 09.01.2007 04:31:37:
No segfaults on my solaris system using gcc 2.95.2, libxml-2.6.27, libxslt-1.1.18 and python 2.4.3. There's been some issues with segfaults due to errors raised from libxml lately which seemed to be correlated with python 2.3 and threading, you might want to take a look at the archives. I ran into some myself using python 2.3. Note that - my current version is a 1.1.2 branch snapshot taken some time ago - the "libxml used" and "libxml compiled" output is the same on my system, but not on yours (whatever that's supposed to mean) - I use a more recent Python 2.4 version (2.4.3); maybe the Python2.3-threading-lxml issue also extends to earlier Python 2.4-versions? Holger Here's my test log: $ PATH=/apps/pydev/bin:$PATH make test PYTHON=/apps/pydev/bin/python2.4 /apps/pydev/bin/python2.4 setup.py build_ext -i Building lxml version 1.1.2-33909 running build_ext /apps/pydev/bin/python2.4 test.py -p -v TESTED VERSION: Python: (2, 4, 3, 'final', 0) lxml.etree: (1, 1, 2, 33909) libxml used: (2, 6, 27) libxml compiled: (2, 6, 27) libxslt used: (1, 1, 18) libxslt compiled: (1, 1, 18) 477/477 (100.0%): Doctest: extensions.txt ====================================================================== ERROR: test_object_path_syntax (lxml.tests.test_objectify.ObjectifyTestCase) ---------------------------------------------------------------------- Traceback (most recent call last): File "/apps/prod//lib/python2.4/unittest.py", line 260, in run testMethod() File "/data/pydev/DOWNLOADS/LXML/lxml/lxml-1.1 /src/lxml/tests/test_objectify.py", line 544, in test_object_path_syntax self.assertEquals(root.c1.c2.text, path(root).text) File "objectify.pyx", line 1103, in objectify.ObjectPath.__call__ File "objectify.pyx", line 1244, in objectify._findObjectPath ValueError: root element does not match: need {objectified}root , got {objectified}root ====================================================================== FAIL: test_xslt_parameter_missing (lxml.tests.test_xslt.ETreeXSLTTestCase) ---------------------------------------------------------------------- Traceback (most recent call last): File "/apps/prod//lib/python2.4/unittest.py", line 260, in run testMethod() File "/data/pydev/DOWNLOADS/LXML/lxml/lxml-1.1 /src/lxml/tests/test_xslt.py", line 221, in test_xslt_parameter_missing st.apply, tree) File "/apps/prod//lib/python2.4/unittest.py", line 326, in failUnlessRaises raise self.failureException, "%s not raised" % excName AssertionError: XSLTApplyError not raised ---------------------------------------------------------------------- Ran 477 tests in 1.014s FAILED (failures=1, errors=1) make: *** [test_inplace] Error 1 Der Inhalt dieser E-Mail ist vertraulich. Falls Sie nicht der angegebene Empfänger sind oder falls diese E-Mail irrtümlich an Sie adressiert wurde, verständigen Sie bitte den Absender sofort und löschen Sie die E-Mail sodann. Das unerlaubte Kopieren sowie die unbefugte Übermittlung sind nicht gestattet. Die Sicherheit von Übermittlungen per E-Mail kann nicht garantiert werden. Falls Sie eine Bestätigung wünschen, fordern Sie bitte den Inhalt der E-Mail als Hardcopy an. The contents of this e-mail are confidential. If you are not the named addressee or if this transmission has been addressed to you in error, please notify the sender immediately and then delete this e-mail. Any unauthorized copying and transmission is forbidden. E-Mail transmission cannot be guaranteed to be secure. If verification is required, please request a hard copy version.

Martijn Faassen wrote:
Yes, I've installed the python 2.4.4 on the machine in question and after a recompile of lxml-1.1.2, everything seems to be working, minus a few test failures (that seemed to be the same as the ones on trunk?). Thanks for both your help looking into this, and apologies for using up your time with what was actually an unrelated issue (in my defense, the whole "1.1.2 fails/trunk works" is a nice bit of misdirection! ;). If nothing else, hopefully this thread will help the next person who runs into this corner case. :) Thanks again, Eli

Martijn Faassen wrote:
Could you let us know which tests are failing?
Hello, The test failures that I mentioned in a previous message were due to there being an older version of libxml2 installed on the machine I was using (I don't administer the machine, and I hadn't realized it was lurking around). This was the cause of the mismatched versions that Holger noticed: TESTED VERSION: Python: (2, 4, 0, 'final', 0) lxml.etree: (1, 1, 2, 0) libxml used: (2, 6, 16) libxml compiled: (2, 6, 27) libxslt used: (1, 1, 11) libxslt compiled: (1, 1, 19) I'm getting one more test failure, but since I've got two versions of libxml2, two versions of python 2.4 and two versions of lxml all installed on this machine, I suspect the problem is that just mismatched configurations on my end. Just in case, I've copied the test failure below, but I really think that it's a problem on the machine I'm using. Eli elis@mapdev4:~/edit/external/tarball/lxml-1.1.2> make test /homes/elis/bin/python setup.py build_ext -i Building lxml version 1.1.2 running build_ext /homes/elis/bin/python test.py -p -v TESTED VERSION: Python: (2, 4, 4, 'final', 0) lxml.etree: (1, 1, 2, 0) libxml used: (2, 6, 27) libxml compiled: (2, 6, 27) libxslt used: (1, 1, 19) libxslt compiled: (1, 1, 19) 477/477 (100.0%): Doctest: extensions.txt ====================================================================== FAIL: test_xslt_parameter_missing (lxml.tests.test_xslt.ETreeXSLTTestCase) ---------------------------------------------------------------------- Traceback (most recent call last): File "/homes/elis/lib/python2.4/unittest.py", line 260, in run testMethod() File "/homes/elis/edit/external/tarball/lxml-1.1.2/src/lxml/tests/test_xslt.py", line 221, in test_xslt_parameter_missing st.apply, tree) File "/homes/elis/lib/python2.4/unittest.py", line 326, in failUnlessRaises raise self.failureException, "%s not raised" % excName AssertionError: XSLTApplyError not raised ---------------------------------------------------------------------- Ran 477 tests in 0.464s FAILED (failures=1) make: *** [test_inplace] Error 1

Hi, lxml-dev-bounces@codespeak.net schrieb am 09.01.2007 04:31:37:
No segfaults on my solaris system using gcc 2.95.2, libxml-2.6.27, libxslt-1.1.18 and python 2.4.3. There's been some issues with segfaults due to errors raised from libxml lately which seemed to be correlated with python 2.3 and threading, you might want to take a look at the archives. I ran into some myself using python 2.3. Note that - my current version is a 1.1.2 branch snapshot taken some time ago - the "libxml used" and "libxml compiled" output is the same on my system, but not on yours (whatever that's supposed to mean) - I use a more recent Python 2.4 version (2.4.3); maybe the Python2.3-threading-lxml issue also extends to earlier Python 2.4-versions? Holger Here's my test log: $ PATH=/apps/pydev/bin:$PATH make test PYTHON=/apps/pydev/bin/python2.4 /apps/pydev/bin/python2.4 setup.py build_ext -i Building lxml version 1.1.2-33909 running build_ext /apps/pydev/bin/python2.4 test.py -p -v TESTED VERSION: Python: (2, 4, 3, 'final', 0) lxml.etree: (1, 1, 2, 33909) libxml used: (2, 6, 27) libxml compiled: (2, 6, 27) libxslt used: (1, 1, 18) libxslt compiled: (1, 1, 18) 477/477 (100.0%): Doctest: extensions.txt ====================================================================== ERROR: test_object_path_syntax (lxml.tests.test_objectify.ObjectifyTestCase) ---------------------------------------------------------------------- Traceback (most recent call last): File "/apps/prod//lib/python2.4/unittest.py", line 260, in run testMethod() File "/data/pydev/DOWNLOADS/LXML/lxml/lxml-1.1 /src/lxml/tests/test_objectify.py", line 544, in test_object_path_syntax self.assertEquals(root.c1.c2.text, path(root).text) File "objectify.pyx", line 1103, in objectify.ObjectPath.__call__ File "objectify.pyx", line 1244, in objectify._findObjectPath ValueError: root element does not match: need {objectified}root , got {objectified}root ====================================================================== FAIL: test_xslt_parameter_missing (lxml.tests.test_xslt.ETreeXSLTTestCase) ---------------------------------------------------------------------- Traceback (most recent call last): File "/apps/prod//lib/python2.4/unittest.py", line 260, in run testMethod() File "/data/pydev/DOWNLOADS/LXML/lxml/lxml-1.1 /src/lxml/tests/test_xslt.py", line 221, in test_xslt_parameter_missing st.apply, tree) File "/apps/prod//lib/python2.4/unittest.py", line 326, in failUnlessRaises raise self.failureException, "%s not raised" % excName AssertionError: XSLTApplyError not raised ---------------------------------------------------------------------- Ran 477 tests in 1.014s FAILED (failures=1, errors=1) make: *** [test_inplace] Error 1 Der Inhalt dieser E-Mail ist vertraulich. Falls Sie nicht der angegebene Empfänger sind oder falls diese E-Mail irrtümlich an Sie adressiert wurde, verständigen Sie bitte den Absender sofort und löschen Sie die E-Mail sodann. Das unerlaubte Kopieren sowie die unbefugte Übermittlung sind nicht gestattet. Die Sicherheit von Übermittlungen per E-Mail kann nicht garantiert werden. Falls Sie eine Bestätigung wünschen, fordern Sie bitte den Inhalt der E-Mail als Hardcopy an. The contents of this e-mail are confidential. If you are not the named addressee or if this transmission has been addressed to you in error, please notify the sender immediately and then delete this e-mail. Any unauthorized copying and transmission is forbidden. E-Mail transmission cannot be guaranteed to be secure. If verification is required, please request a hard copy version.

Martijn Faassen wrote:
Yes, I've installed the python 2.4.4 on the machine in question and after a recompile of lxml-1.1.2, everything seems to be working, minus a few test failures (that seemed to be the same as the ones on trunk?). Thanks for both your help looking into this, and apologies for using up your time with what was actually an unrelated issue (in my defense, the whole "1.1.2 fails/trunk works" is a nice bit of misdirection! ;). If nothing else, hopefully this thread will help the next person who runs into this corner case. :) Thanks again, Eli

Martijn Faassen wrote:
Could you let us know which tests are failing?
Hello, The test failures that I mentioned in a previous message were due to there being an older version of libxml2 installed on the machine I was using (I don't administer the machine, and I hadn't realized it was lurking around). This was the cause of the mismatched versions that Holger noticed: TESTED VERSION: Python: (2, 4, 0, 'final', 0) lxml.etree: (1, 1, 2, 0) libxml used: (2, 6, 16) libxml compiled: (2, 6, 27) libxslt used: (1, 1, 11) libxslt compiled: (1, 1, 19) I'm getting one more test failure, but since I've got two versions of libxml2, two versions of python 2.4 and two versions of lxml all installed on this machine, I suspect the problem is that just mismatched configurations on my end. Just in case, I've copied the test failure below, but I really think that it's a problem on the machine I'm using. Eli elis@mapdev4:~/edit/external/tarball/lxml-1.1.2> make test /homes/elis/bin/python setup.py build_ext -i Building lxml version 1.1.2 running build_ext /homes/elis/bin/python test.py -p -v TESTED VERSION: Python: (2, 4, 4, 'final', 0) lxml.etree: (1, 1, 2, 0) libxml used: (2, 6, 27) libxml compiled: (2, 6, 27) libxslt used: (1, 1, 19) libxslt compiled: (1, 1, 19) 477/477 (100.0%): Doctest: extensions.txt ====================================================================== FAIL: test_xslt_parameter_missing (lxml.tests.test_xslt.ETreeXSLTTestCase) ---------------------------------------------------------------------- Traceback (most recent call last): File "/homes/elis/lib/python2.4/unittest.py", line 260, in run testMethod() File "/homes/elis/edit/external/tarball/lxml-1.1.2/src/lxml/tests/test_xslt.py", line 221, in test_xslt_parameter_missing st.apply, tree) File "/homes/elis/lib/python2.4/unittest.py", line 326, in failUnlessRaises raise self.failureException, "%s not raised" % excName AssertionError: XSLTApplyError not raised ---------------------------------------------------------------------- Ran 477 tests in 0.464s FAILED (failures=1) make: *** [test_inplace] Error 1
participants (3)
-
Eli Stevens (WG.c)
-
Holger Joukl
-
Martijn Faassen