[New-bugs-announce] [issue33005] 3.7.0b2 Interpreter crash in dev mode (or with PYTHONMALLOC=debug) with 'python -X dev -c 'import os; os.fork()'

Mon Mar 5 17:40:09 EST 2018

New submission from Jason Madden <jason at nextthought.com>:

At the request of Victor Stinner on twitter, I ran the gevent test suite with Python 3.7.0b2 with the new '-X dev' argument and discovered an interpreter crash. With a bit of work, it boiled down to a very simple command:

$ env -i .runtimes/snakepit/python3.7.0b2 -X dev -c 'import os; os.fork()'
*** Error in `.runtimes/snakepit/python3.7.0b2': munmap_chunk(): invalid pointer: 0x0000000001c43a80 ***
======= Backtrace: =========
/lib/x86_64-linux-gnu/libc.so.6(+0x777e5)[0x7f5a971607e5]
/lib/x86_64-linux-gnu/libc.so.6(cfree+0x1a8)[0x7f5a9716d698]
.runtimes/snakepit/python3.7.0b2(_PyRuntimeState_Fini+0x30)[0x515d90]
.runtimes/snakepit/python3.7.0b2[0x51445f]
.runtimes/snakepit/python3.7.0b2[0x42ce40]
.runtimes/snakepit/python3.7.0b2(_Py_UnixMain+0x7b)[0x42eaab]
/lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xf0)[0x7f5a97109830]
.runtimes/snakepit/python3.7.0b2(_start+0x29)[0x42a0d9]
======= Memory map: ========
00400000-00689000 r-xp 00000000 08:01 177409                             //.runtimes/versions/python3.7.0b2/bin/python3.7
00888000-00889000 r--p 00288000 08:01 177409                             //.runtimes/versions/python3.7.0b2/bin/python3.7
00889000-008f3000 rw-p 00289000 08:01 177409                             //.runtimes/versions/python3.7.0b2/bin/python3.7
008f3000-00914000 rw-p 00000000 00:00 0
01b84000-01c64000 rw-p 00000000 00:00 0                                  [heap]
7f5a96052000-7f5a96068000 r-xp 00000000 08:01 265946                     /lib/x86_64-linux-gnu/libgcc_s.so.1
7f5a96068000-7f5a96267000 ---p 00016000 08:01 265946                     /lib/x86_64-linux-gnu/libgcc_s.so.1
7f5a96267000-7f5a96268000 rw-p 00015000 08:01 265946                     /lib/x86_64-linux-gnu/libgcc_s.so.1
7f5a96268000-7f5a96273000 r-xp 00000000 08:01 268943                     /lib/x86_64-linux-gnu/libnss_files-2.23.so
7f5a96273000-7f5a96472000 ---p 0000b000 08:01 268943                     /lib/x86_64-linux-gnu/libnss_files-2.23.so
7f5a96472000-7f5a96473000 r--p 0000a000 08:01 268943                     /lib/x86_64-linux-gnu/libnss_files-2.23.so
7f5a96473000-7f5a96474000 rw-p 0000b000 08:01 268943                     /lib/x86_64-linux-gnu/libnss_files-2.23.so
7f5a96474000-7f5a9647a000 rw-p 00000000 00:00 0
7f5a9647a000-7f5a96485000 r-xp 00000000 08:01 268947                     /lib/x86_64-linux-gnu/libnss_nis-2.23.so
7f5a96485000-7f5a96684000 ---p 0000b000 08:01 268947                     /lib/x86_64-linux-gnu/libnss_nis-2.23.so
7f5a96684000-7f5a96685000 r--p 0000a000 08:01 268947                     /lib/x86_64-linux-gnu/libnss_nis-2.23.so
7f5a96685000-7f5a96686000 rw-p 0000b000 08:01 268947                     /lib/x86_64-linux-gnu/libnss_nis-2.23.so
7f5a96686000-7f5a9669c000 r-xp 00000000 08:01 268927                     /lib/x86_64-linux-gnu/libnsl-2.23.so
7f5a9669c000-7f5a9689b000 ---p 00016000 08:01 268927                     /lib/x86_64-linux-gnu/libnsl-2.23.so
7f5a9689b000-7f5a9689c000 r--p 00015000 08:01 268927                     /lib/x86_64-linux-gnu/libnsl-2.23.so
7f5a9689c000-7f5a9689d000 rw-p 00016000 08:01 268927                     /lib/x86_64-linux-gnu/libnsl-2.23.so
7f5a9689d000-7f5a9689f000 rw-p 00000000 00:00 0
7f5a9689f000-7f5a968a7000 r-xp 00000000 08:01 268938                     /lib/x86_64-linux-gnu/libnss_compat-2.23.so
7f5a968a7000-7f5a96aa6000 ---p 00008000 08:01 268938                     /lib/x86_64-linux-gnu/libnss_compat-2.23.so
7f5a96aa6000-7f5a96aa7000 r--p 00007000 08:01 268938                     /lib/x86_64-linux-gnu/libnss_compat-2.23.so
7f5a96aa7000-7f5a96aa8000 rw-p 00008000 08:01 268938                     /lib/x86_64-linux-gnu/libnss_compat-2.23.so
7f5a96acc000-7f5a96b4c000 rw-p 00000000 00:00 0
7f5a96b4c000-7f5a96b4e000 r-xp 00000000 08:01 184551                     //.runtimes/versions/python3.7.0b2/lib/python3.7/lib-dynload/_heapq.cpython-37m-x86_64-linux-gnu.so
7f5a96b4e000-7f5a96d4e000 ---p 00002000 08:01 184551                     //.runtimes/versions/python3.7.0b2/lib/python3.7/lib-dynload/_heapq.cpython-37m-x86_64-linux-gnu.so
7f5a96d4e000-7f5a96d4f000 r--p 00002000 08:01 184551                     //.runtimes/versions/python3.7.0b2/lib/python3.7/lib-dynload/_heapq.cpython-37m-x86_64-linux-gnu.so
7f5a96d4f000-7f5a96d51000 rw-p 00003000 08:01 184551                     //.runtimes/versions/python3.7.0b2/lib/python3.7/lib-dynload/_heapq.cpython-37m-x86_64-linux-gnu.so
7f5a96d51000-7f5a96e11000 rw-p 00000000 00:00 0
7f5a96e11000-7f5a970e9000 r--p 00000000 08:01 133586                     /usr/lib/locale/locale-archive
7f5a970e9000-7f5a972a9000 r-xp 00000000 08:01 268930                     /lib/x86_64-linux-gnu/libc-2.23.so
7f5a972a9000-7f5a974a9000 ---p 001c0000 08:01 268930                     /lib/x86_64-linux-gnu/libc-2.23.so
7f5a974a9000-7f5a974ad000 r--p 001c0000 08:01 268930                     /lib/x86_64-linux-gnu/libc-2.23.so
7f5a974ad000-7f5a974af000 rw-p 001c4000 08:01 268930                     /lib/x86_64-linux-gnu/libc-2.23.so
7f5a974af000-7f5a974b3000 rw-p 00000000 00:00 0
7f5a974b3000-7f5a975bb000 r-xp 00000000 08:01 268926                     /lib/x86_64-linux-gnu/libm-2.23.so
7f5a975bb000-7f5a977ba000 ---p 00108000 08:01 268926                     /lib/x86_64-linux-gnu/libm-2.23.so
7f5a977ba000-7f5a977bb000 r--p 00107000 08:01 268926                     /lib/x86_64-linux-gnu/libm-2.23.so
7f5a977bb000-7f5a977bc000 rw-p 00108000 08:01 268926                     /lib/x86_64-linux-gnu/libm-2.23.so
7f5a977bc000-7f5a977be000 r-xp 00000000 08:01 268937                     /lib/x86_64-linux-gnu/libutil-2.23.so
7f5a977be000-7f5a979bd000 ---p 00002000 08:01 268937                     /lib/x86_64-linux-gnu/libutil-2.23.so
7f5a979bd000-7f5a979be000 r--p 00001000 08:01 268937                     /lib/x86_64-linux-gnu/libutil-2.23.so
7f5a979be000-7f5a979bf000 rw-p 00002000 08:01 268937                     /lib/x86_64-linux-gnu/libutil-2.23.so
7f5a979bf000-7f5a979c2000 r-xp 00000000 08:01 268932                     /lib/x86_64-linux-gnu/libdl-2.23.so
7f5a979c2000-7f5a97bc1000 ---p 00003000 08:01 268932                     /lib/x86_64-linux-gnu/libdl-2.23.so
7f5a97bc1000-7f5a97bc2000 r--p 00002000 08:01 268932                     /lib/x86_64-linux-gnu/libdl-2.23.so
7f5a97bc2000-7f5a97bc3000 rw-p 00003000 08:01 268932                     /lib/x86_64-linux-gnu/libdl-2.23.so
7f5a97bc3000-7f5a97bdb000 r-xp 00000000 08:01 268929                     /lib/x86_64-linux-gnu/libpthread-2.23.so
7f5a97bdb000-7f5a97dda000 ---p 00018000 08:01 268929                     /lib/x86_64-linux-gnu/libpthread-2.23.so
7f5a97dda000-7f5a97ddb000 r--p 00017000 08:01 268929                     /lib/x86_64-linux-gnu/libpthread-2.23.so
7f5a97ddb000-7f5a97ddc000 rw-p 00018000 08:01 268929                     /lib/x86_64-linux-gnu/libpthread-2.23.so
7f5a97ddc000-7f5a97de0000 rw-p 00000000 00:00 0
7f5a97de0000-7f5a97e06000 r-xp 00000000 08:01 268928                     /lib/x86_64-linux-gnu/ld-2.23.so
7f5a97e10000-7f5a97fb5000 rw-p 00000000 00:00 0
7f5a97fb5000-7f5a97fdc000 r--p 00000000 08:01 135047                     /usr/lib/locale/C.UTF-8/LC_CTYPE
7f5a97fdc000-7f5a97fe1000 rw-p 00000000 00:00 0
7f5a97ffd000-7f5a97ffe000 rw-p 00000000 00:00 0
7f5a97ffe000-7f5a98005000 r--s 00000000 08:01 529048                     /usr/lib/x86_64-linux-gnu/gconv/gconv-modules.cache
7f5a98005000-7f5a98006000 r--p 00025000 08:01 268928                     /lib/x86_64-linux-gnu/ld-2.23.so
7f5a98006000-7f5a98007000 rw-p 00026000 08:01 268928                     /lib/x86_64-linux-gnu/ld-2.23.so
7f5a98007000-7f5a98008000 rw-p 00000000 00:00 0
7fff79aeb000-7fff79b0c000 rw-p 00000000 00:00 0                          [stack]
7fff79b1d000-7fff79b20000 r--p 00000000 00:00 0                          [vvar]
7fff79b20000-7fff79b22000 r-xp 00000000 00:00 0                          [vdso]
ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0                  [vsyscall]

The crash is reproducible on Ubuntu 16.04 with a pyenv-built 3.7.0b2 and on macOS 10.13 with the python.org build.

Individually setting PYTHONMALLOC=debug also triggers the crash:

$ PYTHONMALLOC=debug /usr/local/bin/python3.7 -c 'import os; os.fork()'
Python(16996,0x7fffb1879340) malloc: *** error for object 0x7f90e6d01ff0: pointer being freed was not allocated
*** set a breakpoint in malloc_error_break to debug

----------
components: Interpreter Core
messages: 313296
nosy: jmadden
priority: normal
severity: normal
status: open
title: 3.7.0b2 Interpreter crash in dev mode (or with PYTHONMALLOC=debug) with 'python -X dev -c 'import os; os.fork()'
versions: Python 3.7

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33005>
_______________________________________