[New-bugs-announce] [issue38266] Revert bpo-37878: Make PyThreadState_DeleteCurrent() Internal

[Wenzel Jakob]

New submission from Wenzel Jakob <wenzel at inf.ethz.ch>:

A commit from a few days ago and discussed in issue #37878 removed an undocumented function PyThreadState_DeleteCurrent() from Python's public API.

This function was exposed for good reasons and simply removing it because it is undocumented strikes me as a somewhat brutal solution to achieve documentation coverage. This function is useful -- even essential -- to delete the current thread's thread state, which cannot be done using the still-public PyThreadState_Delete(). The documentation could simply be a line stating this explicitly.

I was asked to provide an example of an actual usage: this function is used by pybind11, which is a widely used library for creating Python bindings to C++ code. pybind11 only calls PyThreadState_DeleteCurrent() in a rare set of circumstances, but there is no alternative in this case. Specifically, pybind11 can intercept a Python function call on the main() thread and delete the associated thread state, launching a new thread and continuing Python execution there (with a newly created thread state).

Kind of crazy, so why is this useful? The answer is UI libraries. On some platforms, it is not legal to poll UI events on any thread other than the main thread. This means that it's impossible to implement a proper asynchronous UI event loop because Python is hogging the main thread. With the functionality in pybind11's gil_scoped_acquire, it is possible can launch an event polling loop on the main thread, continue running an interactive Python REPL on another thread, and even swap them back e.g. when the user interface is no longer needed.

Best,
Wenzel

----------
components: Interpreter Core
messages: 353114
nosy: eric.snow, nanjekyejoannah, ncoghlan, tcaswell, vstinner, wenzel
priority: normal
severity: normal
status: open
title: Revert bpo-37878: Make PyThreadState_DeleteCurrent() Internal
type: behavior
versions: Python 3.8, Python 3.9

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue38266>
_______________________________________