Hi, folks.
I found PyEval_AcquireLock and PyEval_ReleaseLock are deprecated since Python 3.2. But the same time, stable ABI is defined in Python 3.2 too. The deprecated APIs are stable ABI too because `ceval.h` is not excluded from the stable ABI PEP.
As far as my understanding, we can not remove them until Python 4.0. Am I right?
I will add comment like this. /* This is a part of stable ABI. Do not remove until Python 4.0 */
Regards,
01.07.20 04:35, Inada Naoki пише:
Hi, folks.
I found PyEval_AcquireLock and PyEval_ReleaseLock are deprecated since Python 3.2. But the same time, stable ABI is defined in Python 3.2 too. The deprecated APIs are stable ABI too because `ceval.h` is not excluded from the stable ABI PEP.
As far as my understanding, we can not remove them until Python 4.0. Am I right?
I will add comment like this. /* This is a part of stable ABI. Do not remove until Python 4.0 */
We can remove them from public headers, but should keep their implementation and export their names to preserve the binary compatibility.
Thanks. I will do it.
On Wed, Jul 1, 2020 at 5:50 PM Serhiy Storchaka storchaka@gmail.com wrote:
01.07.20 04:35, Inada Naoki пише:
Hi, folks.
I found PyEval_AcquireLock and PyEval_ReleaseLock are deprecated since Python 3.2. But the same time, stable ABI is defined in Python 3.2 too. The deprecated APIs are stable ABI too because `ceval.h` is not excluded from the stable ABI PEP.
As far as my understanding, we can not remove them until Python 4.0. Am I right?
I will add comment like this. /* This is a part of stable ABI. Do not remove until Python 4.0 */
We can remove them from public headers, but should keep their implementation and export their names to preserve the binary compatibility. _______________________________________________ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-leave@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/255WEQX2... Code of Conduct: http://python.org/psf/codeofconduct/
Hi,
Last time I looked at PyEval_AcquireLock(), it was used in the wild, but I don't recall exactly where, sorry :-( Before removing the functions, I suggest to first notify impacted projects of the incoming removal, and maybe even propose a fix. I suggest to attempt to follow the process that I proposed in the PEP 620: https://www.python.org/dev/peps/pep-0620/#process-to-reduce-the-number-of-br...
For a concrete example, see this JEP issue: https://github.com/ninia/jep/issues/229
Getting rid of PyEval_AcquireLock() and PyEval_ReleaseLock() in JEP doesn't seem trivial. This project uses subinterpreters and uses/used daemon threads.
At least, I tested with my pythonci project (*) and it's possible to build Cython, numpy and lxml without PyEval_AcquireLock() and PyEval_ReleaseLock() (they don't use these functions).
(*) https://github.com/vstinner/pythonci
Previously, PyEval_ReleaseLock() was used by PyThreadState_DeleteCurrent(). I introduced a new _PyEval_ReleaseLock() function which takes a tstate parameter, so Python no longer uses PyEval_AcquireLock() and PyEval_ReleaseLock().
commit 23ef89db7ae46d160650263cc80479c2ed6693fb Author: Victor Stinner vstinner@python.org Date: Wed Mar 18 02:26:04 2020 +0100
bpo-39984: _PyThreadState_DeleteCurrent() takes tstate (GH-19051)
* _PyThreadState_DeleteCurrent() now takes tstate rather than runtime. * Add ensure_tstate_not_null() helper to pystate.c. * Add _PyEval_ReleaseLock() function. * _PyThreadState_DeleteCurrent() now calls _PyEval_ReleaseLock(tstate) and frees PyThreadState memory after this call, not before. * PyGILState_Release(): rename "tcur" variable to "tstate".
Victor
Le mer. 1 juil. 2020 à 03:39, Inada Naoki songofacandy@gmail.com a écrit :
Hi, folks.
I found PyEval_AcquireLock and PyEval_ReleaseLock are deprecated since Python 3.2. But the same time, stable ABI is defined in Python 3.2 too. The deprecated APIs are stable ABI too because `ceval.h` is not excluded from the stable ABI PEP.
As far as my understanding, we can not remove them until Python 4.0. Am I right?
I will add comment like this. /* This is a part of stable ABI. Do not remove until Python 4.0 */
Regards,
Inada Naoki songofacandy@gmail.com _______________________________________________ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-leave@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/EJF67ZM2... Code of Conduct: http://python.org/psf/codeofconduct/
-- Night gathers, and now my watch begins. It shall not end until my death.
On Thu, Jul 2, 2020 at 7:28 PM Victor Stinner vstinner@python.org wrote:
Hi,
Last time I looked at PyEval_AcquireLock(), it was used in the wild, but I don't recall exactly where, sorry :-( Before removing the functions, I suggest to first notify impacted projects of the incoming removal, and maybe even propose a fix.
Thank you for suggestion.
I had grepped PyEval_AcquireLock in top4000 packages and I confirmed it is not used. But I had not checked PyEval_ReleaseLock because I thought it is used only pair with PyEval_AcquireLock.
Actually, PyEval_ReleaseLock is used in three packages:
pydevd-pycharm-202.5103.19/pydevd_attach_to_process/windows/attach.cpp 330: DEFINE_PROC(releaseLock, PyEval_Lock*, "PyEval_ReleaseLock", -160);
jep-3.9.0/src/main/c/Jep/pyembed.c 836: PyEval_ReleaseLock();
ptvsd-4.3.2.zip/ptvsd-4.3.2/src/ptvsd/_vendored/pydevd/pydevd_attach_to_process/windows/attach.cpp 330: DEFINE_PROC(releaseLock, PyEval_Lock*, "PyEval_ReleaseLock", -160);
I will keep PyEval_ReleaseLock.
Getting rid of PyEval_AcquireLock() and PyEval_ReleaseLock() in JEP doesn't seem trivial. This project uses subinterpreters and uses/used daemon threads.
I think they use only PyEval_ReleaseLock(). Do they use PyEval_AcquireLock() too?
Regards, -- Inada Naoki songofacandy@gmail.com
Le ven. 3 juil. 2020 à 04:51, Inada Naoki songofacandy@gmail.com a écrit :
Actually, PyEval_ReleaseLock is used in three packages:
pydevd-pycharm-202.5103.19/pydevd_attach_to_process/windows/attach.cpp 330: DEFINE_PROC(releaseLock, PyEval_Lock*, "PyEval_ReleaseLock", -160);
jep-3.9.0/src/main/c/Jep/pyembed.c 836: PyEval_ReleaseLock();
ptvsd-4.3.2.zip/ptvsd-4.3.2/src/ptvsd/_vendored/pydevd/pydevd_attach_to_process/windows/attach.cpp 330: DEFINE_PROC(releaseLock, PyEval_Lock*, "PyEval_ReleaseLock", -160);
I will keep PyEval_ReleaseLock.
The PEP 620 doesn't require to keep a deprecated forever: only to notify projects that the function is going to be removed, and try to help them to replace it.
pydevd-pycharm
I think they use only PyEval_ReleaseLock(). Do they use PyEval_AcquireLock() too?
I checked with GitHub code search on the JEP project: PyEval_AcquireLock() is not used.
PyEval_ReleaseLock() is called in a single function: https://github.com/ninia/jep/blob/639f6cbe512b5ce4b51412564dba8db5bbbf1e3b/s...
Also, I'm not sure that JEP uses PyEval_ReleaseLock() properly: https://github.com/ninia/jep/issues/229#issuecomment-635467616
I don't see why a lock would hold the GIL but release it before exiting.
Maybe JEP should call PyThreadState_DeleteCurrent(), which calls PyEval_ReleaseLock() internally, in pyembed_thread_close().
The _thread.start_new_thread() calls _PyThreadState_DeleteCurrent(tstate) when a thread completes, and this function calls _PyEval_ReleaseLock(tstate).
Maybe PyEval_ReleaseLock() documentation should explain that.
Victor
Le ven. 3 juil. 2020 à 11:19, Victor Stinner vstinner@python.org a écrit :
Le ven. 3 juil. 2020 à 04:51, Inada Naoki songofacandy@gmail.com a écrit :
Actually, PyEval_ReleaseLock is used in three packages:
pydevd-pycharm-202.5103.19/pydevd_attach_to_process/windows/attach.cpp 330: DEFINE_PROC(releaseLock, PyEval_Lock*, "PyEval_ReleaseLock", -160);
jep-3.9.0/src/main/c/Jep/pyembed.c 836: PyEval_ReleaseLock();
ptvsd-4.3.2.zip/ptvsd-4.3.2/src/ptvsd/_vendored/pydevd/pydevd_attach_to_process/windows/attach.cpp 330: DEFINE_PROC(releaseLock, PyEval_Lock*, "PyEval_ReleaseLock", -160);
I will keep PyEval_ReleaseLock.
The PEP 620 doesn't require to keep a deprecated forever: only to notify projects that the function is going to be removed, and try to help them to replace it.
pydevd-pycharm
Oops, I sent the email by mistake while I was typing :-(
pydevd-pycharm doesn't *need* PyEval_ReleaseLock() on Python 3.10: it only uses PyEval_ReleaseLock() on Python 3.1 and older!
if (version >= PythonVersion_32) { // we will release the GIL here gilRelease(gilState); } else { releaseLock(); }
Maybe it's time for pydevd-pycharm to remove support for Python 2.7. Or the code can be made optional using "#if PY_VERSION_HEX < 0x03020000".
So it seems possible to fix JEP and pydevd-pycharm. IMHO it's fine to remove PyEval_ReleaseLock() in Python 3.10. The deprecation warning is there since Python 3.2.
Victor
On Fri, Jul 3, 2020 at 6:23 PM Victor Stinner vstinner@python.org wrote:
So it seems possible to fix JEP and pydevd-pycharm. IMHO it's fine to remove PyEval_ReleaseLock() in Python 3.10. The deprecation warning is there since Python 3.2.
While PyEval_AcquireLock is deprecated, PyEval_ReleaseLock is not deprecated yet in C. https://github.com/python/cpython/blob/master/Include/ceval.h#L132
Maybe, we can uncomment Py_DEPRECATE in 3.10, and remove it from header file in 3.12.
Regards,
PyEval_ReleaseLock() is deprecated since Python 3.2 in the documentation: https://docs.python.org/dev/c-api/init.html#c.PyEval_ReleaseLock
PyEval_AcquireLock() was annotated with Py_DEPRECATED() by Serhiy Storchaka in https://bugs.python.org/issue19569#msg280110 where he wrote:
"PyEval_ReleaseLock() is used in Python/pystate.c. It can't be replaced with PyEval_ReleaseThread() since the latter don't accept NULL."
I wrote in a previous email, I modified Python internals to no longer call PyEval_ReleaseLock(), but a new internal _PyEval_ReleaseLock(tstate) function instead.
Maybe we can still mark PyEval_ReleaseLock() as deprecated in Python 3.9, since it's just compiler warning, it's less intrusive than a warning emitted at runtime.
Victor
Le ven. 3 juil. 2020 à 11:28, Inada Naoki songofacandy@gmail.com a écrit :
On Fri, Jul 3, 2020 at 6:23 PM Victor Stinner vstinner@python.org wrote:
So it seems possible to fix JEP and pydevd-pycharm. IMHO it's fine to remove PyEval_ReleaseLock() in Python 3.10. The deprecation warning is there since Python 3.2.
While PyEval_AcquireLock is deprecated, PyEval_ReleaseLock is not deprecated yet in C. https://github.com/python/cpython/blob/master/Include/ceval.h#L132
Maybe, we can uncomment Py_DEPRECATE in 3.10, and remove it from header file in 3.12.
Regards,
Inada Naoki songofacandy@gmail.com
03.07.20 12:34, Victor Stinner пише:
PyEval_ReleaseLock() is deprecated since Python 3.2 in the documentation: https://docs.python.org/dev/c-api/init.html#c.PyEval_ReleaseLock
PyEval_AcquireLock() was annotated with Py_DEPRECATED() by Serhiy Storchaka in https://bugs.python.org/issue19569#msg280110 where he wrote:
"PyEval_ReleaseLock() is used in Python/pystate.c. It can't be replaced with PyEval_ReleaseThread() since the latter don't accept NULL."
I wrote in a previous email, I modified Python internals to no longer call PyEval_ReleaseLock(), but a new internal _PyEval_ReleaseLock(tstate) function instead.
Maybe we can still mark PyEval_ReleaseLock() as deprecated in Python 3.9, since it's just compiler warning, it's less intrusive than a warning emitted at runtime.
+1 for adding Py_DEPRECATED() in 3.9.
On Thu, Jul 2, 2020 at 8:05 PM Inada Naoki songofacandy@gmail.com wrote:
On Thu, Jul 2, 2020 at 7:28 PM Victor Stinner vstinner@python.org wrote:
Hi,
Last time I looked at PyEval_AcquireLock(), it was used in the wild, but I don't recall exactly where, sorry :-( Before removing the functions, I suggest to first notify impacted projects of the incoming removal, and maybe even propose a fix.
Thank you for suggestion.
I had grepped PyEval_AcquireLock in top4000 packages and I confirmed it is not used. But I had not checked PyEval_ReleaseLock because I thought it is used only pair with PyEval_AcquireLock.
Actually, PyEval_ReleaseLock is used in three packages:
pydevd-pycharm-202.5103.19/pydevd_attach_to_process/windows/attach.cpp 330: DEFINE_PROC(releaseLock, PyEval_Lock*, "PyEval_ReleaseLock", -160);
jep-3.9.0/src/main/c/Jep/pyembed.c 836: PyEval_ReleaseLock();
ptvsd-4.3.2.zip/ptvsd-4.3.2/src/ptvsd/_vendored/pydevd/pydevd_attach_to_process/windows/attach.cpp 330: DEFINE_PROC(releaseLock, PyEval_Lock*, "PyEval_ReleaseLock", -160);
As the dev manager for ptvsd I can officially tell not to worry about it as the project has been superseded by debugpy.
-Brett
I will keep PyEval_ReleaseLock.
Getting rid of PyEval_AcquireLock() and PyEval_ReleaseLock() in JEP doesn't seem trivial. This project uses subinterpreters and uses/used daemon threads.
I think they use only PyEval_ReleaseLock(). Do they use PyEval_AcquireLock() too?
Regards,
Inada Naoki songofacandy@gmail.com _______________________________________________ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-leave@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/HMDNL4MC... Code of Conduct: http://python.org/psf/codeofconduct/
-
Brett Cannon -
Inada Naoki -
Serhiy Storchaka -
Victor Stinner