Mailman 3 Milter using lxml to parse emails crashes eventually - lxml - The Python XML Toolkit

Nov. 12, 2019

      ...
...
...
import sys
from lxml import etree
print("%-20s: %s" % ('Python',           sys.version_info))
Python              : sys.version_info(major=3, minor=6, micro=8,
releaselevel='final', serial=0)
print("%-20s: %s" % ('lxml.etree',       etree.LXML_VERSION))
lxml.etree          : (4, 4, 1, 0)
print("%-20s: %s" % ('libxml used',      etree.LIBXML_VERSION))
...
...
...
print("%-20s: %s" % ('libxml compiled',  etree.LIBXML_COMPILED_VERSION))
...
...
...
print("%-20s: %s" % ('libxslt used',     etree.LIBXSLT_VERSION))
...
...
...
print("%-20s: %s" % ('libxslt compiled',
etree.LIBXSLT_COMPILED_VERSION))
Hi there!

I'am running into an error when using a python based milter (
https://github.com/spacefreak86/pyquarantine-milter) which utilizes
BeautifulSoup/lxml to parse html emails.
After some mails passed the milter, it segfaults eventually.
For testing purposes I generated the same mail over and over again, its not
reproducible (to me) exactly when it's gonna happen, but somewhere in the
range of 30-250 emails...

Although the project is not mature yet, the developer suspects it's an lxml
bug not related to the project itself - maybe environment related, since
he/she can't reproduce this in his/her gentoo based system. You can see our
conversation here
https://github.com/spacefreak86/pyquarantine-milter/issues/4

I tried different versions of lxml (via PyPI) and compiled the latest
release from github as well (probably about 10 days ago) - to no avail.

I'am not a programmer - so my use usage is rather limited here, but I'll
try my best to provide anything needed.
Thanks for your time!

dennis

## Environment
OS: bionic / LXC
Python: 3.6.8
beautifulsoup4: 4.8.1 (PyPI)
lxml: 4.4.1 (PyPI)

Python 3.6.8 (default, Oct  7 2019, 12:59:55)
[GCC 8.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
libxml used         : (2, 9, 9)
libxml compiled     : (2, 9, 9)
libxslt used        : (1, 1, 33)
libxslt compiled    : (1, 1, 33)

I did some stacktraces - maybe they can shed some light on the issue:

## gdb
(gdb) run /usr/local/bin/pyquarantine-milter -s inet:8899@127.0.0.1 -c
/etc/pyquarantine/pyquarantine.conf -d
Starting program: /usr/bin/python3.6 /usr/local/bin/pyquarantine-milter -s
inet:8899@127.0.0.1 -c /etc/pyquarantine/pyquarantine.conf -d
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".
2019-10-29 12:09:20 pyquarantine-milter[pyquarantine.run]: [INFO]
PyQuarantine-Milter starting
[...]
[New Thread 0x7ffff2784700 (LWP 6450)]
[New Thread 0x7ffff1f83700 (LWP 6451)]
[New Thread 0x7ffff1782700 (LWP 6452)]
[New Thread 0x7ffff0f81700 (LWP 6453)]
[...]
2019-10-29 12:11:03 pyquarantine-milter[pyquarantine.notifications]:
[DEBUG] C312828951: trying to create BeatufilSoup object with parser lib
lxml, text length is 4588 bytes

Thread 4 "python3.6" received signal SIGSEGV, Segmentation fault.
[Switching to Thread 0x7ffff1782700 (LWP 6452)]
0x00007ffff43f108b in __pyx_f_4lxml_5etree__handleSaxTargetStartNoNs.4645
() from /usr/local/lib/python3.6/dist-packages/lxml/
etree.cpython-36m-x86_64-linux-gnu.so

## (gdb) bt
#0  0x00007ffff43f108b in
__pyx_f_4lxml_5etree__handleSaxTargetStartNoNs.4645 () from
/usr/local/lib/python3.6/dist-packages/lxml/
etree.cpython-36m-x86_64-linux-gnu.so
#1  0x00007ffff42d1370 in htmlParseStartTag () from
/usr/local/lib/python3.6/dist-packages/lxml/
etree.cpython-36m-x86_64-linux-gnu.so
#2  0x00007ffff42d39c1 in htmlParseChunk () from
/usr/local/lib/python3.6/dist-packages/lxml/
etree.cpython-36m-x86_64-linux-gnu.so
#3  0x00007ffff42d4718 in __pyx_f_4lxml_5etree_11_FeedParser_feed.8014 ()
from /usr/local/lib/python3.6/dist-packages/lxml/
etree.cpython-36m-x86_64-linux-gnu.so
#4  0x00007ffff42d5f4e in __pyx_pw_4lxml_5etree_11_FeedParser_1feed () from
/usr/local/lib/python3.6/dist-packages/lxml/
etree.cpython-36m-x86_64-linux-gnu.so
#5  0x00007ffff42f5ddf in __Pyx_CyFunction_CallAsMethod.7585 () from
/usr/local/lib/python3.6/dist-packages/lxml/
etree.cpython-36m-x86_64-linux-gnu.so
#6  0x00000000005aa69c in _PyObject_FastCallDict (kwargs=<optimized out>,
nargs=2, args=0x7fffe401b1d0, func=<cython_function_or_method at remote
0x7ffff3eabb10>) at ../Objects/tupleobject.c:131
#7  _PyObject_FastCallKeywords () at ../Objects/abstract.c:2496
#8  0x000000000050ab53 in call_function.lto_priv () at
../Python/ceval.c:4875
#9  0x000000000050c549 in _PyEval_EvalFrameDefault () at
../Python/ceval.c:3335
#10 0x0000000000509ce8 in PyEval_EvalFrameEx (throwflag=0,
   f=Frame 0x7fffe401b038, for file
/usr/local/lib/python3.6/dist-packages/bs4/builder/_lxml.py, line 294, in
feed (self=<LXMLTreeBuilder(_default_parser=None,
soup=<BeautifulSoup(element_classes={}, builder=<...>, is_xml=False,
known_xml=False, _namespaces={'xml': 'http://www.w3.org/XML/1998/namespace'},
parse_only=None, markup=b'<html>\n  <head>\n\n    <meta
http-equiv="content-type" content="text/html; charset=UTF-8">\n  </head>\n
 <body>\n    <h1>Lorem Ipsum</h1>\n    <h4>"Neque porro quisquam est qui
dolorem ipsum quia dolor sit amet,\n      consectetur, adipisci
velit..."</h4>\n    <h5>"There is no one who loves pain itself, who seeks
after it and\n      wants to have it, simply because it is pain..."</h5>\n
   <hr>\n    <div id="Content">\n      <div class="boxed">\n        <div
id="lipsum">\n          <p>\n            Lorem ipsum dolor sit amet,
consectetur adipiscing elit.\n            Proin lobortis quis est et
imperdiet. Curabitur sagittis\n            erat non velit scelerisque,
lobortis commodo sapien...(truncated)) at ../Python/ceval.c:754
#11 _PyFunction_FastCall (globals=<optimized out>, nargs=140737018703928,
args=<optimized out>, co=<optimized out>) at ../Python/ceval.c:4933
#12 fast_function.lto_priv () at ../Python/ceval.c:4968
#13 0x000000000050aa1d in call_function.lto_priv () at
../Python/ceval.c:4872
#14 0x000000000050c549 in _PyEval_EvalFrameDefault () at
../Python/ceval.c:3335
#15 0x0000000000509ce8 in PyEval_EvalFrameEx (throwflag=0,
   f=Frame 0x7ffff0743708, for file
/usr/local/lib/python3.6/dist-packages/bs4/__init__.py, line 382, in _feed
(self=<BeautifulSoup(element_classes={},
builder=<LXMLTreeBuilder(_default_parser=None, soup=<...>, nsmaps=[{'
http://www.w3.org/XML/1998/namespace': 'xml'}], cdata_list_attributes={'*':
['class', 'accesskey', 'dropzone'], 'a': ['rel', 'rev'], 'link': ['rel',
'rev'], 'td': ['headers'], 'th': ['headers'], 'form': ['accept-charset'],
'object': ['archive'], 'area': ['rel'], 'icon': ['sizes'], 'iframe':
['sandbox'], 'output': ['for']}, preserve_whitespace_tags={'pre',
'textarea'}, store_line_numbers=False, processing_instruction_class=<type
at remote 0xce53e8>, parser=<lxml.etree.HTMLParser at remote
0x7ffff0751178>) at remote 0x7ffff06c8588>, is_xml=False, known_xml=False,
_namespaces={'xml': 'http://www.w3.org/XML/1998/namespace'},
parse_only=None, markup=b'<html>\n  <head>\n\n    <meta
http-equiv="content-type" content="text/html; charset=UTF-8">\n  </head>\n
 <body>\n    <h1>Lorem Ipsum</h1>\n    <h4>"Nequ...(truncated)) at
../Python/ceval.c:754
#16 _PyFunction_FastCall (globals=<optimized out>, nargs=140737227536136,
args=<optimized out>, co=<optimized out>) at ../Python/ceval.c:4933
#17 fast_function.lto_priv () at ../Python/ceval.c:4968
#18 0x000000000050aa1d in call_function.lto_priv () at
../Python/ceval.c:4872
#19 0x000000000050c549 in _PyEval_EvalFrameDefault () at
../Python/ceval.c:3335
#20 0x00000000005081d5 in PyEval_EvalFrameEx (throwflag=0,
   f=Frame 0x7fffe40188c8, for file
/usr/local/lib/python3.6/dist-packages/bs4/__init__.py, line 313, in
__init__ (self=<BeautifulSoup(element_classes={},
builder=<LXMLTreeBuilder(_default_parser=None, soup=<...>, nsmaps=[{'
http://www.w3.org/XML/1998/namespace': 'xml'}], cdata_list_attributes={'*':
['class', 'accesskey', 'dropzone'], 'a': ['rel', 'rev'], 'link': ['rel',
'rev'], 'td': ['headers'], 'th': ['headers'], 'form': ['accept-charset'],
'object': ['archive'], 'area': ['rel'], 'icon': ['sizes'], 'iframe':
['sandbox'], 'output':
#21 _PyEval_EvalCodeWithName.lto_priv.1836 () at ../Python/ceval.c:4166
#22 0x0000000000509647 in _PyFunction_FastCallDict () at
../Python/ceval.c:5084
#23 0x00000000005951c1 in _PyObject_FastCallDict (kwargs=0x0, nargs=3,
args=0x7ffff1780af0, func=<function at remote 0x7ffff4816f28>) at
../Objects/abstract.c:2310
#24 _PyObject_Call_Prepend (kwargs=0x0, args=<optimized out>,
obj=<optimized out>, func=<function at remote 0x7ffff4816f28>) at
../Objects/abstract.c:2373
#25 method_call.lto_priv () at ../Objects/classobject.c:314
#26 0x000000000054a11f in PyObject_Call (kwargs=0x0,
   args=(b'<html>\n  <head>\n\n    <meta http-equiv="content-type"
content="text/html; charset=UTF-8">\n  </head>\n  <body>\n    <h1>Lorem
Ipsum</h1>\n    <h4>"Neque porro quisquam est qui dolorem ipsum quia dolor
sit amet,\n      consectetur, adipisci velit..."</h4>\n    <h5>"There is no
one who loves pain itself, who seeks after it and\n      wants to have it,
simply because it is pain..."</h5>\n    <hr>\n    <div id="Content">\n
 <div class="boxed">\n        <div id="lipsum">\n          <p>\n
 Lorem ipsum dolor sit amet, consectetur adipiscing elit.\n
 Proin lobortis quis est et imperdiet. Curabitur sagittis\n            erat
non velit scelerisque, lobortis commodo sapien\n            ullamcorper.
Morbi mollis venenatis dictum. Etiam euismod\n            urna in mi
sagittis, in cursus eros mollis. Aenean convallis\n            elit pretium
quam mollis iaculis. Vestibulum sed sodales\n            tortor. Donec at
convallis diam. Etiam egestas interdum\n            egestas. Pellentesque a
bib...(truncated),
   func=<method at remote 0x7ffff4990388>) at ../Objects/abstract.c:2261
#27 slot_tp_init () at ../Objects/typeobject.c:6420
#28 0x0000000000551761 in type_call.lto_priv () at
../Objects/typeobject.c:915
#29 0x00000000005aa69c in _PyObject_FastCallDict (kwargs=<optimized out>,
nargs=2, args=0x7ffff07483f0, func=<type at remote 0xd6df58>) at
../Objects/tupleobject.c:131
#30 _PyObject_FastCallKeywords () at ../Objects/abstract.c:2496
#31 0x000000000050ab53 in call_function.lto_priv () at
../Python/ceval.c:4875
#32 0x000000000050c549 in _PyEval_EvalFrameDefault () at
../Python/ceval.c:3335
#33 0x0000000000509ce8 in PyEval_EvalFrameEx (throwflag=0,
   f=Frame 0x7ffff0748240, for file
/usr/local/lib/python3.6/dist-packages/pyquarantine/notifications.py, line
240, in get_text (self=<EMailNotification(quarantine_name='virusmail',
global_config={'quarantines': 'virusmail', 'preferred_quarantine_action':
'last'}, config={'ignore_hosts': '10.0.0.0/8,172.16.0.0/12,192.168.0.0/16',
'regex': '^X-Virus-Status: Infected.*', 'smtp_host': '127.0.0.1',
'smtp_port': '10025', 'quarantine_type': 'file', 'quarantine_directory':
'/var/virusmails/', 'action': 'discard', 'reject_reason': 'Message
rejected', 'notification_type': 'email',
'notification_email_envelope_from': 'noreply@domain.tld',
'notification_email_from': 'Virus Alert <noreply@domain.tld>',
'notification_email_subject': 'Virus quarantine notification',
'notification_email_template': 'templates/notification.template',
'notification_email_strip_images': 'False',
'notification_email_replacement_img': 'templates/removed.png',
'notification_email_embedded_imgs': 'templates/random.png',
'whitelist_type': 'none',...(truncated)) at ../Python/ceval.c:754
#34 _PyFunction_FastCall (globals=<optimized out>, nargs=140737227555392,
args=<optimized out>, co=<optimized out>) at ../Python/ceval.c:4933
#35 fast_function.lto_priv () at ../Python/ceval.c:4968
#36 0x000000000050aa1d in call_function.lto_priv () at
../Python/ceval.c:4872
#37 0x000000000050c549 in _PyEval_EvalFrameDefault () at
../Python/ceval.c:3335
#38 0x00000000005081d5 in PyEval_EvalFrameEx (throwflag=0,
   f=Frame 0x7ffff0748048, for file
/usr/local/lib/python3.6/dist-packages/pyquarantine/notifications.py, line
253, in get_text_multipart
(self=<EMailNotification(quarantine_name='virusmail',
global_config={'quarantines': 'virusmail', 'preferred_quarantine_action':
'last'}, config={'ignore_hosts': '10.0.0.0/8,172.16.0.0/12,192.168.0.0/16',
'regex': '^X-Virus-Status: Infected.*', 'smtp_host': '127.0.0.1',
'smtp_port': '10025', 'quarantine_type': 'file', 'quarantine_directory':
'/var/virusmails/', 'action': 'discard', 'reject_reason':
#39 _PyEval_EvalCodeWithName.lto_priv.1836 () at ../Python/ceval.c:4166
#40 0x000000000050a020 in fast_function.lto_priv () at
../Python/ceval.c:4992
#41 0x000000000050aa1d in call_function.lto_priv () at
../Python/ceval.c:4872
#42 0x000000000050c549 in _PyEval_EvalFrameDefault () at
../Python/ceval.c:3335
#43 0x0000000000509ce8 in PyEval_EvalFrameEx (throwflag=0,
   f=Frame 0x7fffe40087c8, for file
/usr/local/lib/python3.6/dist-packages/pyquarantine/notifications.py, line
307, in get_html_text_part
(self=<EMailNotification(quarantine_name='virusmail',
global_config={'quarantines': 'virusmail', 'preferred_quarantine_action':
'last'}, config={'ignore_hosts': '10.0.0.0/8,172.16.0.0/12,192.168.0.0/16',
'regex': '^X-Virus-Status: Infected.*', 'smtp_host': '127.0.0.1',
'smtp_port': '10025', 'quarantine_type': 'file', 'quarantine_directory':
'/var/virusmails/', 'action': 'discard', 'reject_reason': 'Message
rejected', 'notification_type': 'email',
'notification_email_envelope_from': 'noreply@domain.tld',
'notification_email_from': 'Virus Alert <noreply@domain.tld>',
'notification_email_subject': 'Virus quarantine notification',
'notification_email_template': 'templates/notification.template',
'notification_email_strip_images': 'False',
'notification_email_replacement_img': 'templates/removed.png',
'notification_email_embedded_imgs': 'templates/random.png',
'whitelist_type...(truncated)) at ../Python/ceval.c:754
#44 _PyFunction_FastCall (globals=<optimized out>, nargs=140737018628040,
args=<optimized out>, co=<optimized out>) at ../Python/ceval.c:4933
#45 fast_function.lto_priv () at ../Python/ceval.c:4968
#46 0x000000000050aa1d in call_function.lto_priv () at
../Python/ceval.c:4872
#47 0x000000000050c549 in _PyEval_EvalFrameDefault () at
../Python/ceval.c:3335
#48 0x00000000005081d5 in PyEval_EvalFrameEx (throwflag=0,
   f=Frame 0x7fffe4006a18, for file
/usr/local/lib/python3.6/dist-packages/pyquarantine/notifications.py, line
337, in notify (self=<EMailNotification(quarantine_name='virusmail',
global_config={'quarantines': 'virusmail', 'preferred_quarantine_action':
'last'}, config={'ignore_hosts': '10.0.0.0/8,172.16.0.0/12,192.168.0.0/16',
'regex': '^X-Virus-Status: Infected.*', 'smtp_host': '127.0.0.1',
'smtp_port': '10025', 'quarantine_type': 'file', 'quarantine_directory':
'/var/virusmails/', 'action': 'discard', 'reject_reason': 'Message
rejected', 'notification_type': 'email',
'notification_email_envelope_from': 'noreply@domain.tld',
'notification_email_from': 'Virus Alert <noreply@domain.tld>',
'notification_email_subject': 'Virus quarantine notification',
'notification_email_template': 'templates/notification.template',
'notification_email_strip_images': 'False',
'notification_email_replacement_img': 'templates/removed.png',
'notification_email_embedded_imgs': 'templates/random.png',
'whitelist_type': 'none', '...(truncated)) at ../Python/ceval.c:754
#49 _PyEval_EvalCodeWithName.lto_priv.1836 () at ../Python/ceval.c:4166
#50 0x000000000050a020 in fast_function.lto_priv () at
../Python/ceval.c:4992
#51 0x000000000050aa1d in call_function.lto_priv () at
../Python/ceval.c:4872
#52 0x000000000050c549 in _PyEval_EvalFrameDefault () at
../Python/ceval.c:3335
#53 0x00000000005081d5 in PyEval_EvalFrameEx (throwflag=0,
   f=Frame 0x7fffe4003668, for file
/usr/local/lib/python3.6/dist-packages/pyquarantine/__init__.py, line 334,
in eom (quarantines=[({'ignore_hosts': '
10.0.0.0/8,172.16.0.0/12,192.168.0.0/16', 'regex': '^X-Virus-Status:
Infected.*', 'smtp_host': '127.0.0.1', 'smtp_port': '10025',
'quarantine_type': 'file', 'quarantine_directory': '/var/virusmails/',
'action': 'discard', 'reject_reason': 'Message rejected',
'notification_type': 'email', 'notification_email_envelope_from':
'noreply@domain.tld', 'notification_email_from': 'Virus Alert
<noreply@domain.tld>', 'notification_email_subject': 'Virus quarantine
notification', 'notification_email_template':
'templates/notification.template', 'notification_email_strip_images':
'False', 'notification_email_replacement_img': 'templates/removed.png',
'notification_email_embedded_imgs': 'templates/random.png',
'whitelist_type': 'none', 'whitelist_db_connection':
'mysql://user:password@localhost/database', 'whitelist_db_table':
'whitelist', 'name': 'virusmail', 'index': 0,...(truncated)) at
../Python/ceval.c:754
#54 _PyEval_EvalCodeWithName.lto_priv.1836 () at ../Python/ceval.c:4166
#55 0x000000000050a020 in fast_function.lto_priv () at
../Python/ceval.c:4992
#56 0x000000000050aa1d in call_function.lto_priv () at
../Python/ceval.c:4872
#57 0x000000000050c549 in _PyEval_EvalFrameDefault () at
../Python/ceval.c:3335
#58 0x00000000005081d5 in PyEval_EvalFrameEx (throwflag=0, f=Frame
0x7ffff07411f0, for file
/usr/local/lib/python3.6/dist-packages/Milter/__init__.py, line 772, in
<lambda> (ctx=<milter.Context at remote 0x7ffff7ea9c10>)) at
../Python/ceval.c:754
#59 _PyEval_EvalCodeWithName.lto_priv.1836 () at ../Python/ceval.c:4166
#60 0x000000000058952b in PyEval_EvalCodeEx (closure=<optimized out>,
kwdefs=<optimized out>, defcount=0, defs=0x0, kwcount=0, kws=0x0,
argcount=<optimized out>, args=0x7ffff7e4fca0, locals=0x0,
globals=<optimized out>, _co=<optimized out>) at ../Python/ceval.c:4187
#61 function_call.lto_priv () at ../Objects/funcobject.c:604
#62 0x00000000005a04ce in PyObject_Call () at ../Objects/abstract.c:2261
#63 0x00007ffff675dbd9 in _generic_wrapper (self=0x7ffff7ea9c10,
cb=<function at remote 0x7ffff27b4620>, arglist=(<milter.Context at remote
0x7ffff7ea9c10>,)) at miltermodule.c:493
#64 0x00007ffff6545312 in ?? () from
/usr/lib/x86_64-linux-gnu/libmilter.so.1.0.1
#65 0x00007ffff65456c2 in mi_engine () from
/usr/lib/x86_64-linux-gnu/libmilter.so.1.0.1
#66 0x00007ffff654810a in ?? () from
/usr/lib/x86_64-linux-gnu/libmilter.so.1.0.1
#67 0x00007ffff77cc6db in start_thread (arg=0x7ffff1782700) at
pthread_create.c:463
#68 0x00007ffff7b0588f in clone () at
../sysdeps/unix/sysv/linux/x86_64/clone.S:95

## (gdb) py-bt
Traceback (most recent call first):
 File "/usr/local/lib/python3.6/dist-packages/bs4/builder/_lxml.py", line
294, in feed
   self.parser.feed(markup)
 File "/usr/local/lib/python3.6/dist-packages/bs4/__init__.py", line 382,
in _feed
   self.builder.feed(self.markup)
 File "/usr/local/lib/python3.6/dist-packages/bs4/__init__.py", line 313,
in __init__
   self._feed()
 File
"/usr/local/lib/python3.6/dist-packages/pyquarantine/notifications.py",
line 240, in get_text
   soup = BeautifulSoup(text, self.parser_lib)
 File
"/usr/local/lib/python3.6/dist-packages/pyquarantine/notifications.py",
line 253, in get_text_multipart
   soup = self.get_text(queueid, part)
 File
"/usr/local/lib/python3.6/dist-packages/pyquarantine/notifications.py",
line 307, in get_html_text_part
   soup = self.get_text_multipart(queueid, msg)
 File
"/usr/local/lib/python3.6/dist-packages/pyquarantine/notifications.py",
line 337, in notify
   queueid, email.message_from_binary_file(fp))
 File "/usr/local/lib/python3.6/dist-packages/pyquarantine/__init__.py",
line 334, in eom
   subgroups, named_subgroups)
 File "/usr/local/lib/python3.6/dist-packages/Milter/__init__.py", line
772, in <lambda>
   milter.set_eom_callback(lambda ctx: ctx.getpriv().eom())

## (gdb) info threads
 Id   Target Id         Frame
 1    Thread 0x7ffff7fe7740 (LWP 6445) "python3.6" 0x00007ffff7af8bf9 in
__GI___poll (fds=0x7fffffffd998, nfds=1, timeout=5000) at
../sysdeps/unix/sysv/linux/poll.c:29
 2    Thread 0x7ffff2784700 (LWP 6450) "python3.6" 0x00007ffff7a2426c in
__GI___sigtimedwait (set=<optimized out>, set@entry=0x7ffff2783e40,
info=info@entry=0x7ffff2783d80, timeout=timeout@entry=0x0) at
../sysdeps/unix/sysv/linux/sigtimedwait.c:42
 3    Thread 0x7ffff1f83700 (LWP 6451) "python3.6" 0x00007ffff7af8bf9 in
__GI___poll (fds=0x7fffec000b20, nfds=1, timeout=10000) at
../sysdeps/unix/sysv/linux/poll.c:29
* 4    Thread 0x7ffff1782700 (LWP 6452) "python3.6" 0x00007ffff43f108b in
__pyx_f_4lxml_5etree__handleSaxTargetStartNoNs.4645 () from
/usr/local/lib/python3.6/dist-packages/lxml/
etree.cpython-36m-x86_64-linux-gnu.so
 5    Thread 0x7ffff0f81700 (LWP 6453) "python3.6" 0x00007ffff77d2f85 in
futex_abstimed_wait_cancelable (private=<optimized out>,
abstime=0x7ffff0f80e90, expected=0, futex_word=0x7ffff674e1a0) at
../sysdeps/unix/sysv/linux/futex-internal.h:205
 6    Thread 0x7fffe3fff700 (LWP 6464) "python3.6" 0x00007ffff77d56d6 in
futex_abstimed_wait_cancelable (private=0, abstime=0x0, expected=0,
futex_word=0x7fffdc0011b0) at
../sysdeps/unix/sysv/linux/futex-internal.h:205

## (gdb) py-list
289
290        def feed(self, markup):
291            encoding = self.soup.original_encoding
292            try:
293                self.parser = self.parser_for(encoding)
...
294                self.parser.feed(markup)
295                self.parser.close()
296            except (UnicodeDecodeError, LookupError, etree.ParserError)
as e:
297                raise ParserRejectedMarkup(e)
298
299
## (gdb) thread apply all py-list
Thread 6 (Thread 0x7fffe3fff700 (LWP 6464)):
290            self._waiters.append(waiter)
291            saved_state = self._release_save()
292            gotit = False
293            try:    # restore state no matter what (e.g.,
KeyboardInterrupt)
294                if timeout is None:
...
295                    waiter.acquire()
296                    gotit = True
297                else:
298                    if timeout > 0:
299                        gotit = waiter.acquire(True, timeout)
300                    else:
Thread 5 (Thread 0x7ffff0f81700 (LWP 6453)):
Unable to locate gdb frame for python bytecode interpreter

Thread 4 (Thread 0x7ffff1782700 (LWP 6452)):
289
290        def feed(self, markup):
291            encoding = self.soup.original_encoding
292            try:
293                self.parser = self.parser_for(encoding)
...
294                self.parser.feed(markup)
295                self.parser.close()
296            except (UnicodeDecodeError, LookupError, etree.ParserError)
as e:
297                raise ParserRejectedMarkup(e)
298
299
Thread 3 (Thread 0x7ffff1f83700 (LWP 6451)):
---Type <return> to continue, or q <return> to quit---
Unable to locate gdb frame for python bytecode interpreter

Thread 2 (Thread 0x7ffff2784700 (LWP 6450)):
Unable to locate gdb frame for python bytecode interpreter

Thread 1 (Thread 0x7ffff7fe7740 (LWP 6445)):
793      # processed.  It's safer to shutdown sendmail, kill the filter
process,
794      # restart the filter, and then restart sendmail.
795      milter.opensocket(rmsock)
796      start_seq = _seq
797      try:
...
798        milter.main()
799      except milter.error:
800        if start_seq == _seq: raise # couldn't start
801        # milter has been running for a while, but now it can't start
new threads
802        raise milter.error("out of thread resources")
803

Milter using lxml to parse emails crashes eventually

Dennis |

tags

participants (1)