Hi, folks.
I found PyEval_AcquireLock and PyEval_ReleaseLock are deprecated since
Python 3.2.
But the same time, stable ABI is defined in Python 3.2 too.
The deprecated APIs are stable ABI too because ceval.h is not
excluded from the stable ABI PEP.
As far as my understanding, we can not remove them until Python 4.0. Am I right?
I will add comment like this. / This is a part of stable ABI. Do not remove until Python 4.0 /
Regards,
Inada Naoki songofacandy@gmail.com
01.07.20 04:35, Inada Naoki пише:
Hi, folks.
I found PyEval_AcquireLock and PyEval_ReleaseLock are deprecated since
Python 3.2.
But the same time, stable ABI is defined in Python 3.2 too.
The deprecated APIs are stable ABI too because ceval.h is not
excluded from the stable ABI PEP.
As far as my understanding, we can not remove them until Python 4.0. Am I right?
I will add comment like this. / This is a part of stable ABI. Do not remove until Python 4.0 /
We can remove them from public headers, but should keep their implementation and export their names to preserve the binary compatibility.
Thanks. I will do it.
On Wed, Jul 1, 2020 at 5:50 PM Serhiy Storchaka storchaka@gmail.com wrote: >
01.07.20 04:35, Inada Naoki пише:
Hi, folks.
I found PyEval_AcquireLock and PyEval_ReleaseLock are deprecated since
Python 3.2.
But the same time, stable ABI is defined in Python 3.2 too.
The deprecated APIs are stable ABI too because ceval.h is not
excluded from the stable ABI PEP.
As far as my understanding, we can not remove them until Python 4.0. Am I right?
I will add comment like this. / This is a part of stable ABI. Do not remove until Python 4.0 /
We can remove them from public headers, but should keep their implementation and export their names to preserve the binary compatibility.
Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-leave@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/255WEQX2... Code of Conduct: http://python.org/psf/codeofconduct/
-- Inada Naoki songofacandy@gmail.com
Hi,
Last time I looked at PyEval_AcquireLock(), it was used in the wild, but I don't recall exactly where, sorry :-( Before removing the functions, I suggest to first notify impacted projects of the incoming removal, and maybe even propose a fix. I suggest to attempt to follow the process that I proposed in the PEP 620: https://www.python.org/dev/peps/pep-0620/#process-to-reduce-the-number-of-br...
For a concrete example, see this JEP issue: https://github.com/ninia/jep/issues/229
Getting rid of PyEval_AcquireLock() and PyEval_ReleaseLock() in JEP doesn't seem trivial. This project uses subinterpreters and uses/used daemon threads.
At least, I tested with my pythonci project (*) and it's possible to build Cython, numpy and lxml without PyEval_AcquireLock() and PyEval_ReleaseLock() (they don't use these functions).
(*) https://github.com/vstinner/pythonci
Previously, PyEval_ReleaseLock() was used by PyThreadState_DeleteCurrent(). I introduced a new _PyEval_ReleaseLock() function which takes a tstate parameter, so Python no longer uses PyEval_AcquireLock() and PyEval_ReleaseLock().
commit 23ef89db7ae46d160650263cc80479c2ed6693fb Author: Victor Stinner vstinner@python.org Date: Wed Mar 18 02:26:04 2020 +0100
bpo-39984: _PyThreadState_DeleteCurrent() takes tstate (GH-19051)
* _PyThreadState_DeleteCurrent() now takes tstate rather than
runtime.
* Add ensure_tstate_not_null() helper to pystate.c.
* Add _PyEval_ReleaseLock() function.
* _PyThreadState_DeleteCurrent() now calls
_PyEval_ReleaseLock(tstate) and frees PyThreadState memory after
this call, not before.
* PyGILState_Release(): rename "tcur" variable to "tstate".
Victor
Le mer. 1 juil. 2020 à 03:39, Inada Naoki songofacandy@gmail.com a écrit : >
Hi, folks.
I found PyEval_AcquireLock and PyEval_ReleaseLock are deprecated since
Python 3.2.
But the same time, stable ABI is defined in Python 3.2 too.
The deprecated APIs are stable ABI too because ceval.h is not
excluded from the stable ABI PEP.
As far as my understanding, we can not remove them until Python 4.0. Am I right?
I will add comment like this. / This is a part of stable ABI. Do not remove until Python 4.0 /
Regards,
Inada Naoki songofacandy@gmail.com
Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-leave@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/EJF67ZM2... Code of Conduct: http://python.org/psf/codeofconduct/
-- Night gathers, and now my watch begins. It shall not end until my death.
On Thu, Jul 2, 2020 at 7:28 PM Victor Stinner vstinner@python.org wrote: >
Hi,
Last time I looked at PyEval_AcquireLock(), it was used in the wild, but I don't recall exactly where, sorry :-( Before removing the functions, I suggest to first notify impacted projects of the incoming removal, and maybe even propose a fix.
Thank you for suggestion.
I had grepped PyEval_AcquireLock in top4000 packages and I confirmed it is not used. But I had not checked PyEval_ReleaseLock because I thought it is used only pair with PyEval_AcquireLock.
Actually, PyEval_ReleaseLock is used in three packages:
pydevd-pycharm-202.5103.19/pydevd_attach_to_process/windows/attach.cpp 330: DEFINE_PROC(releaseLock, PyEval_Lock*, "PyEval_ReleaseLock", -160);
jep-3.9.0/src/main/c/Jep/pyembed.c 836: PyEval_ReleaseLock();
ptvsd-4.3.2.zip/ptvsd-4.3.2/src/ptvsd/_vendored/pydevd/pydevd_attach_to_process/windows/attach.cpp 330: DEFINE_PROC(releaseLock, PyEval_Lock*, "PyEval_ReleaseLock", -160);
I will keep PyEval_ReleaseLock.
>
Getting rid of PyEval_AcquireLock() and PyEval_ReleaseLock() in JEP doesn't seem trivial. This project uses subinterpreters and uses/used daemon threads.
I think they use only PyEval_ReleaseLock(). Do they use PyEval_AcquireLock() too?
Regards,
Inada Naoki songofacandy@gmail.com
Le ven. 3 juil. 2020 à 04:51, Inada Naoki songofacandy@gmail.com a écrit :
Actually, PyEval_ReleaseLock is used in three packages:
pydevd-pycharm-202.5103.19/pydevd_attach_to_process/windows/attach.cpp 330: DEFINE_PROC(releaseLock, PyEval_Lock*, "PyEval_ReleaseLock", -160);
jep-3.9.0/src/main/c/Jep/pyembed.c 836: PyEval_ReleaseLock();
ptvsd-4.3.2.zip/ptvsd-4.3.2/src/ptvsd/_vendored/pydevd/pydevd_attach_to_process/windows/attach.cpp 330: DEFINE_PROC(releaseLock, PyEval_Lock*, "PyEval_ReleaseLock", -160);
I will keep PyEval_ReleaseLock.
The PEP 620 doesn't require to keep a deprecated forever: only to notify projects that the function is going to be removed, and try to help them to replace it.
pydevd-pycharm
I think they use only PyEval_ReleaseLock(). Do they use PyEval_AcquireLock() too?
I checked with GitHub code search on the JEP project: PyEval_AcquireLock() is not used.
PyEval_ReleaseLock() is called in a single function: https://github.com/ninia/jep/blob/639f6cbe512b5ce4b51412564dba8db5bbbf1e3b/s...
Also, I'm not sure that JEP uses PyEval_ReleaseLock() properly: https://github.com/ninia/jep/issues/229#issuecomment-635467616
I don't see why a lock would hold the GIL but release it before exiting.
Maybe JEP should call PyThreadState_DeleteCurrent(), which calls PyEval_ReleaseLock() internally, in pyembed_thread_close().
The _thread.start_new_thread() calls _PyThreadState_DeleteCurrent(tstate) when a thread completes, and this function calls _PyEval_ReleaseLock(tstate).
Maybe PyEval_ReleaseLock() documentation should explain that.
Victor
Night gathers, and now my watch begins. It shall not end until my death.
Le ven. 3 juil. 2020 à 11:19, Victor Stinner vstinner@python.org a écrit :
Le ven. 3 juil. 2020 à 04:51, Inada Naoki songofacandy@gmail.com a écrit :
Actually, PyEval_ReleaseLock is used in three packages:
pydevd-pycharm-202.5103.19/pydevd_attach_to_process/windows/attach.cpp 330: DEFINE_PROC(releaseLock, PyEval_Lock*, "PyEval_ReleaseLock", -160);
jep-3.9.0/src/main/c/Jep/pyembed.c 836: PyEval_ReleaseLock();
ptvsd-4.3.2.zip/ptvsd-4.3.2/src/ptvsd/_vendored/pydevd/pydevd_attach_to_process/windows/attach.cpp 330: DEFINE_PROC(releaseLock, PyEval_Lock*, "PyEval_ReleaseLock", -160);
I will keep PyEval_ReleaseLock.
The PEP 620 doesn't require to keep a deprecated forever: only to notify projects that the function is going to be removed, and try to help them to replace it.
pydevd-pycharm
Oops, I sent the email by mistake while I was typing :-(
pydevd-pycharm doesn't need PyEval_ReleaseLock() on Python 3.10: it only uses PyEval_ReleaseLock() on Python 3.1 and older!
if (version >= PythonVersion_32) {
// we will release the GIL here
gilRelease(gilState);
} else {
releaseLock();
}
Maybe it's time for pydevd-pycharm to remove support for Python 2.7. Or the code can be made optional using "#if PY_VERSION_HEX < 0x03020000".
So it seems possible to fix JEP and pydevd-pycharm. IMHO it's fine to remove PyEval_ReleaseLock() in Python 3.10. The deprecation warning is there since Python 3.2.
Victor
Night gathers, and now my watch begins. It shall not end until my death.
On Fri, Jul 3, 2020 at 6:23 PM Victor Stinner vstinner@python.org wrote: > >
So it seems possible to fix JEP and pydevd-pycharm. IMHO it's fine to remove PyEval_ReleaseLock() in Python 3.10. The deprecation warning is there since Python 3.2.
While PyEval_AcquireLock is deprecated, PyEval_ReleaseLock is not deprecated yet in C. https://github.com/python/cpython/blob/master/Include/ceval.h#L132
Maybe, we can uncomment Py_DEPRECATE in 3.10, and remove it from header file in 3.12.
Regards,
Inada Naoki songofacandy@gmail.com
PyEval_ReleaseLock() is deprecated since Python 3.2 in the documentation: https://docs.python.org/dev/c-api/init.html#c.PyEval_ReleaseLock
PyEval_AcquireLock() was annotated with Py_DEPRECATED() by Serhiy Storchaka in https://bugs.python.org/issue19569#msg280110 where he wrote:
"PyEval_ReleaseLock() is used in Python/pystate.c. It can't be replaced with PyEval_ReleaseThread() since the latter don't accept NULL."
I wrote in a previous email, I modified Python internals to no longer call PyEval_ReleaseLock(), but a new internal _PyEval_ReleaseLock(tstate) function instead.
Maybe we can still mark PyEval_ReleaseLock() as deprecated in Python 3.9, since it's just compiler warning, it's less intrusive than a warning emitted at runtime.
Victor
Le ven. 3 juil. 2020 à 11:28, Inada Naoki songofacandy@gmail.com a écrit : >
On Fri, Jul 3, 2020 at 6:23 PM Victor Stinner vstinner@python.org wrote: > >
So it seems possible to fix JEP and pydevd-pycharm. IMHO it's fine to remove PyEval_ReleaseLock() in Python 3.10. The deprecation warning is there since Python 3.2.
While PyEval_AcquireLock is deprecated, PyEval_ReleaseLock is not deprecated yet in C. https://github.com/python/cpython/blob/master/Include/ceval.h#L132
Maybe, we can uncomment Py_DEPRECATE in 3.10, and remove it from header file in 3.12.
Regards,
Inada Naoki songofacandy@gmail.com
-- Night gathers, and now my watch begins. It shall not end until my death.
03.07.20 12:34, Victor Stinner пише:
PyEval_ReleaseLock() is deprecated since Python 3.2 in the documentation: https://docs.python.org/dev/c-api/init.html#c.PyEval_ReleaseLock
PyEval_AcquireLock() was annotated with Py_DEPRECATED() by Serhiy Storchaka in https://bugs.python.org/issue19569#msg280110 where he wrote:
"PyEval_ReleaseLock() is used in Python/pystate.c. It can't be replaced with PyEval_ReleaseThread() since the latter don't accept NULL."
I wrote in a previous email, I modified Python internals to no longer call PyEval_ReleaseLock(), but a new internal _PyEval_ReleaseLock(tstate) function instead.
Maybe we can still mark PyEval_ReleaseLock() as deprecated in Python 3.9, since it's just compiler warning, it's less intrusive than a warning emitted at runtime.
+1 for adding Py_DEPRECATED() in 3.9.
On Thu, Jul 2, 2020 at 8:05 PM Inada Naoki songofacandy@gmail.com wrote:
On Thu, Jul 2, 2020 at 7:28 PM Victor Stinner vstinner@python.org wrote: >
Hi,
Last time I looked at PyEval_AcquireLock(), it was used in the wild, but I don't recall exactly where, sorry :-( Before removing the functions, I suggest to first notify impacted projects of the incoming removal, and maybe even propose a fix.
Thank you for suggestion.
I had grepped PyEval_AcquireLock in top4000 packages and I confirmed it is not used. But I had not checked PyEval_ReleaseLock because I thought it is used only pair with PyEval_AcquireLock.
Actually, PyEval_ReleaseLock is used in three packages:
pydevd-pycharm-202.5103.19/pydevd_attach_to_process/windows/attach.cpp 330: DEFINE_PROC(releaseLock, PyEval_Lock*, "PyEval_ReleaseLock", -160);
jep-3.9.0/src/main/c/Jep/pyembed.c 836: PyEval_ReleaseLock();
ptvsd-4.3.2.zip/ptvsd-4.3.2/src/ptvsd/_vendored/pydevd/pydevd_attach_to_process/windows/attach.cpp 330: DEFINE_PROC(releaseLock, PyEval_Lock*, "PyEval_ReleaseLock", -160);
As the dev manager for ptvsd I can officially tell not to worry about it as the project has been superseded by debugpy.
-Brett
> >
I will keep PyEval_ReleaseLock.
>
Getting rid of PyEval_AcquireLock() and PyEval_ReleaseLock() in JEP doesn't seem trivial. This project uses subinterpreters and uses/used daemon threads.
I think they use only PyEval_ReleaseLock(). Do they use PyEval_AcquireLock() too?
Regards,
Inada Naoki songofacandy@gmail.com
Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-leave@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/HMDNL4MC... Code of Conduct: http://python.org/psf/codeofconduct/
-
Brett Cannon -
Inada Naoki -
Serhiy Storchaka -
Victor Stinner