[issue13044] pdb throws AttributeError at end of debugging session

[Xavier de Gaye]

Xavier de Gaye <xdegaye at gmail.com> added the comment:

Actually the segfault can be avoided by running the garbage collection before _PyState_ClearModules() in PyImport_Cleanup() and one can trace the C destructor with the attached patch global_destructors.diff applied on top of PR 6730.

_PyState_ClearModules() clears the state->modules_by_index list and causes the readline module to be non-functional. The same problem occurs in test_create_at_shutdown_without_encoding() in test_io.py when it is run with the io module: the C destructor fails before hitting the "LookupError: unknown encoding: ascii" error message because the garbage collection is made after _PyState_ClearModules() and the _io module is non-functional (PyState_FindModule() returns NULL).

So the destructors can be traced during finalization provided the tracer does not access any of the sys attributes that are set to None at the start of PyImport_Cleanup().  This is true for the globals of the modules that are only owned by sys.modules at the time 'weaklist' is about to be built in PyImport_Cleanup(). But it is not the case of '_ag' and that explains at last why the destructor was called and failed:
'_ag' is a global of the types module which is owned (imported) by the inspect module, which is owned (imported) by the bdb module which is owned by PyThreadState through its c_profileobj member. So the types module, and all modules imported directly or indirectly by pdb, can only be cleared at the end of PyImport_Cleanup() when 'weaklist' is browsed for the last modules to clear. At that point any module may be non-functional and tracing must not be allowed.

The global_destructors.diff patch ensures that no access is made by the pdb and bdb modules (but imported modules have not been checked) to the the sys attributes that are set to None at the start of PyImport_Cleanup():
* Pdb.lookupmodule() accesses sys.path and is used to set breakpoints so we forbid setting breakpoint when sys.path is None.
* All the values of sys.modules are set to None when 'weaklist' is built. This causes the lazy imports to fail and the patch removes them except the lazy import of pydoc in pdb.py because pydoc imports threading and when threading has been imported wait_for_thread_shutdown() in Py_FinalizeEx() runs the threading._shutdown() Python code which is traced (this happens in Python 3.6 too whenever threading is imported). Since it is annoying to systematically trace this function and pydoc is a heavy module, pdb.help() is disabled on Python finalization.

----------
Added file: https://bugs.python.org/file47581/global_destructors.diff

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue13044>
_______________________________________