I wrote the feature (both tracemalloc and query tracemalloc when a buffer overflow is detected), so I should be able to help you ;-)
Le dim. 5 avr. 2020 à 00:27, Skip Montanaro firstname.lastname@example.org a écrit :
Looking at the tracemalloc module docs and trying various command line args (-X tracemalloc=5) or environment variables (PYTHONTRACEMALLOC=5)
Yes, that's the right way to enable it. The env var is inherited by subprocesses.
I'm unable to provoke any different output.
Maybe your test runs Python with -I or -E which ignores the environment variable. Which command do do you run to run tests?
It's unclear from your output which test and which process triggers the "Debug memory block at address" bug.
Maybe you are out of lock and the buffer corruption is only detected (the memory is only deallocated) after tracemalloc was disabled by _PyTraceMalloc_Fini() call in Py_FinalizeEx(). You may try to hack Python by commenting this _PyTraceMalloc_Fini() call.
FYI there is an unit test on the debug hooks on memory allocator to ensure that it detects buffer overflow: test_capi.test_buffer_overflow() checks for "Debug memory block at address (...)".
Example: ------------------------ $ cat bug.py import _testcapi _testcapi.pymem_buffer_overflow()
$ python3 bug.py Segmentation fault (core dumped)
$ python3 -X tracemalloc=5 -X dev bug.py Debug memory block at address p=0x7f225fd1c100: API 'm' 16 bytes originally requested The 7 pad bytes at p-7 are FORBIDDENBYTE, as expected. The 8 pad bytes at tail=0x7f225fd1c110 are not all FORBIDDENBYTE (0xfd): at tail+0: 0x78 *** OUCH at tail+1: 0xfd at tail+2: 0xfd at tail+3: 0xfd at tail+4: 0xfd at tail+5: 0xfd at tail+6: 0xfd at tail+7: 0xfd The block was made by call #56367 to debug malloc/realloc. Data at p: cd cd cd cd cd cd cd cd cd cd cd cd cd cd cd cd
Memory block allocated at (most recent call first): File "bug.py", line 2
Fatal Python error: bad trailing pad byte
Current thread 0x00007f226cda56c0 (most recent call first): File "bug.py", line 2 in <module> Aborted (core dumped) ------------------------
Tracemalloc adds "Memory block allocated at" traceback.
By the way, I started to also suggest enabling tracemalloc when a ResourceWarning is logged, to see where the leaked resource was allocated.