[issue37878] sub-ineterpreters : Document PyThreadState_DeleteCurrent() ?
New submission from Joannah Nanjekye <nanjekyejoannah@gmail.com>: I just noticed that PyThreadState_DeleteCurrent()in Python/pystate.c is not documented. Relevant documentation should go in Doc/c-api/init.rst If no one objects to this. ---------- assignee: docs@python components: Documentation keywords: easy messages: 349881 nosy: docs@python, eric.snow, nanjekyejoannah, ncoghlan, vstinner priority: normal severity: normal stage: needs patch status: open title: sub-ineterpreters : Document PyThreadState_DeleteCurrent() ? versions: Python 3.8, Python 3.9 _______________________________________ Python tracker <report@bugs.python.org> <https://bugs.python.org/issue37878> _______________________________________
Change by Joannah Nanjekye <nanjekyejoannah@gmail.com>: ---------- title: sub-ineterpreters : Document PyThreadState_DeleteCurrent() ? -> Sub-ineterpreters : Document PyThreadState_DeleteCurrent() ? _______________________________________ Python tracker <report@bugs.python.org> <https://bugs.python.org/issue37878> _______________________________________
Change by Joannah Nanjekye <nanjekyejoannah@gmail.com>: ---------- title: Sub-ineterpreters : Document PyThreadState_DeleteCurrent() ? -> Sub-Interpreters : Document PyThreadState_DeleteCurrent() _______________________________________ Python tracker <report@bugs.python.org> <https://bugs.python.org/issue37878> _______________________________________
Change by Joannah Nanjekye <nanjekyejoannah@gmail.com>: ---------- keywords: +patch pull_requests: +15034 stage: needs patch -> patch review pull_request: https://github.com/python/cpython/pull/15315 _______________________________________ Python tracker <report@bugs.python.org> <https://bugs.python.org/issue37878> _______________________________________
STINNER Victor <vstinner@redhat.com> added the comment: I'm not sure why PyThreadState_DeleteCurrent() is exposed. Why would anyone use it? It is used internally by the _thread module when the thread function completes. I suggest to make the function internal instead of documenting it. Eric, Joannah: what do you think? ---------- _______________________________________ Python tracker <report@bugs.python.org> <https://bugs.python.org/issue37878> _______________________________________
Joannah Nanjekye <nanjekyejoannah@gmail.com> added the comment:
From what I know the GIL is released by calling PyEval_ReleaseThread() or PyEval_SaveThread().
I see we can use PyThreadState_Delete() to clean up-specifically destroy thread state but does not release the GIL and requires a thread state. On the other hand, PyThreadState_DeleteCurrent() allows to both clean up the current thread state - no thread state is required as well release the GIL which is convenient. I would not fight so much to keep it public given we have PyEval_ReleaseThread() or PyEval_SaveThread(). ---------- _______________________________________ Python tracker <report@bugs.python.org> <https://bugs.python.org/issue37878> _______________________________________
STINNER Victor <vstinner@redhat.com> added the comment: Is it safe to call PyThreadState_DeleteCurrent()? ---------- keywords: -easy _______________________________________ Python tracker <report@bugs.python.org> <https://bugs.python.org/issue37878> _______________________________________
Joannah Nanjekye <nanjekyejoannah@gmail.com> added the comment: I updated the PR to make the function internal. ---------- _______________________________________ Python tracker <report@bugs.python.org> <https://bugs.python.org/issue37878> _______________________________________
STINNER Victor <vstinner@python.org> added the comment: I did a GitHub code search on "PyThreadState_DeleteCurrent" in C code: https://github.com/search?l=C&q=PyThreadState_DeleteCurrent&type=Code I looked at the first 6 pages: I only found copies of the Python source code, but no *call* to this function. It seems like this function is not used outside Python, so it's fine to remove it. Anyway, if an user complains lately, we can add it back. It's fine. In case of doubt, in this case, I'm fine with removing the function. ---------- _______________________________________ Python tracker <report@bugs.python.org> <https://bugs.python.org/issue37878> _______________________________________
STINNER Victor <vstinner@python.org> added the comment: New changeset 2bc43cdc015eda4f1a651bb2b308a17a83c38e14 by Victor Stinner (Joannah Nanjekye) in branch 'master': bpo-37878: Remove PyThreadState_DeleteCurrent() function (GH-15315) https://github.com/python/cpython/commit/2bc43cdc015eda4f1a651bb2b308a17a83c... ---------- _______________________________________ Python tracker <report@bugs.python.org> <https://bugs.python.org/issue37878> _______________________________________
STINNER Victor <vstinner@python.org> added the comment: Thanks Joannah. ---------- resolution: -> fixed stage: patch review -> resolved status: open -> closed _______________________________________ Python tracker <report@bugs.python.org> <https://bugs.python.org/issue37878> _______________________________________
Joannah Nanjekye <nanjekyejoannah@gmail.com> added the comment: Update, Some user of this function just complained on the merged pull requests here : https://github.com/python/cpython/pull/15315#issuecomment-534522962. Do we revert and deprecate ? cc @victor ---------- _______________________________________ Python tracker <report@bugs.python.org> <https://bugs.python.org/issue37878> _______________________________________
Wenzel Jakob <wenzel@inf.ethz.ch> added the comment: Hi, pybind11 developer here. A bit on context of our usage of this function: Pybind11 supports some advanced GIL-related tricks that *require* access this function. For instance, pybind11 can intercept a Python function call on the `main()` thread and delete the associated thread state, launching a new thread and continuing Python execution there (with a newly created thread state). Kind of crazy, but why is this useful? UI libraries. On some platforms, it is not legal to poll UI events on any thread other than the main thread. This means that it's impossible to implement a proper UI event loop because Python is hogging the main thread. With the functionality in pybind11's ``gil_scoped_acquire``, I can launch an event polling loop on the main thread, continue running an interactive Python session on another thread, and even swap them back e.g. when the user interface is no longer needed. Best, Wenzel ---------- nosy: +wenzel _______________________________________ Python tracker <report@bugs.python.org> <https://bugs.python.org/issue37878> _______________________________________
Eric Snow <ericsnowcurrently@gmail.com> added the comment: I've re-opened this issue to address the original point: documenting PyThreadState_DeleteCurrent(). FWIW, I don't see a problem with documenting it if we're keeping it around (which it looks like we are). We will address reverting GH-15315 in #38266. ---------- resolution: fixed -> stage: resolved -> needs patch status: closed -> open _______________________________________ Python tracker <report@bugs.python.org> <https://bugs.python.org/issue37878> _______________________________________
Change by Joannah Nanjekye <nanjekyejoannah@gmail.com>: ---------- pull_requests: +16149 stage: needs patch -> patch review pull_request: https://github.com/python/cpython/pull/16558 _______________________________________ Python tracker <report@bugs.python.org> <https://bugs.python.org/issue37878> _______________________________________
STINNER Victor <vstinner@python.org> added the comment: New changeset 8855e47d09d4db1206c65b24efc8ad0df585ac46 by Victor Stinner (Joannah Nanjekye) in branch 'master': bpo-38266: Revert bpo-37878: Make PyThreadState_DeleteCurrent() Internal (GH-16558) https://github.com/python/cpython/commit/8855e47d09d4db1206c65b24efc8ad0df58... ---------- _______________________________________ Python tracker <report@bugs.python.org> <https://bugs.python.org/issue37878> _______________________________________
STINNER Victor <vstinner@python.org> added the comment:
bpo-38266: Revert bpo-37878: Make PyThreadState_DeleteCurrent() Internal (GH-16558)
Oh, I didn't notice something in the review: the revert moved PyThreadState_DeleteCurrent() definition from Include/pystate.h to Include/cpython/pystate.h. Previously, it was defined in Include/pystate.h, see the commit removing it: https://github.com/python/cpython/commit/2bc43cdc015eda4f1a651bb2b308a17a83c... Joannah: can you please move the definition back to Include/pystate.h, at the same place it was before the removal, please? The revert excluded PyThreadState_DeleteCurrent() from the stable API (Py_LIMITED_API). ---------- versions: -Python 3.8 _______________________________________ Python tracker <report@bugs.python.org> <https://bugs.python.org/issue37878> _______________________________________
STINNER Victor <vstinner@python.org> added the comment: To be clear: this issue only impacts Python 3.8. The function was only removed from the master branch. It was still in the 3.8 branch. ---------- _______________________________________ Python tracker <report@bugs.python.org> <https://bugs.python.org/issue37878> _______________________________________
Joannah Nanjekye <nanjekyejoannah@gmail.com> added the comment:
Oh, I didn't notice something in the review: the revert moved >PyThreadState_DeleteCurrent() definition from Include/pystate.h to >Include/cpython/pystate.h. Previously, it was defined in > Include/pystate.h, see the commit removing it:
I made this change as it was requested by Eric snow in issue38266. "when we un-revert we should be sure to move PyThreadState_DeleteCurrent() from Include/pystate.h to Include/cpython/pystate.h." ---------- _______________________________________ Python tracker <report@bugs.python.org> <https://bugs.python.org/issue37878> _______________________________________
STINNER Victor <vstinner@python.org> added the comment:
"when we un-revert we should be sure to move PyThreadState_DeleteCurrent() from Include/pystate.h to Include/cpython/pystate.h."
Ah, I didn't see this comment. If Eric endorses this change, I'm fine with it :-) I close the issue. If someone wants to modify PyThreadState_DeleteCurrent() to automatically call PyThreadState_Clear(), please open a new issue for that. ---------- resolution: -> fixed stage: patch review -> resolved status: open -> closed _______________________________________ Python tracker <report@bugs.python.org> <https://bugs.python.org/issue37878> _______________________________________
Change by STINNER Victor <vstinner@python.org>: ---------- pull_requests: +19738 pull_request: https://github.com/python/cpython/pull/20489 _______________________________________ Python tracker <report@bugs.python.org> <https://bugs.python.org/issue37878> _______________________________________
STINNER Victor <vstinner@python.org> added the comment: New changeset fda7f6d61b13c68f59806db674e892fda4013348 by Victor Stinner in branch 'master': bpo-37878: PyThreadState_DeleteCurrent() was not removed (GH-20489) https://github.com/python/cpython/commit/fda7f6d61b13c68f59806db674e892fda40... ---------- _______________________________________ Python tracker <report@bugs.python.org> <https://bugs.python.org/issue37878> _______________________________________
Change by miss-islington <mariatta.wijaya+miss-islington@gmail.com>: ---------- nosy: +miss-islington nosy_count: 6.0 -> 7.0 pull_requests: +19745 pull_request: https://github.com/python/cpython/pull/20496 _______________________________________ Python tracker <report@bugs.python.org> <https://bugs.python.org/issue37878> _______________________________________
STINNER Victor <vstinner@python.org> added the comment: I was very confused by the history of this issue. To be clear, PyThreadState_DeleteCurrent() is now documented in Python 3.9: https://docs.python.org/dev/c-api/init.html#c.PyThreadState_DeleteCurrent ---------- _______________________________________ Python tracker <report@bugs.python.org> <https://bugs.python.org/issue37878> _______________________________________
miss-islington <mariatta.wijaya+miss-islington@gmail.com> added the comment: New changeset 6a478641c00e5a52be9d595ad804bf848a498d96 by Miss Islington (bot) in branch '3.9': bpo-37878: PyThreadState_DeleteCurrent() was not removed (GH-20489) https://github.com/python/cpython/commit/6a478641c00e5a52be9d595ad804bf848a4... ---------- _______________________________________ Python tracker <report@bugs.python.org> <https://bugs.python.org/issue37878> _______________________________________
participants (5)
-
Eric Snow -
Joannah Nanjekye
-
miss-islington
-
STINNER Victor
-
Wenzel Jakob