Re: [Python-Dev] cpython: Issue #18408: Fix PyUnicode_AsUTF8AndSize(), raise MemoryError exception on
Am 29.10.2013 01:46, schrieb victor.stinner:
http://hg.python.org/cpython/rev/e1d51c42e5a1 changeset: 86716:e1d51c42e5a1 user: Victor Stinner
date: Tue Oct 29 01:28:23 2013 +0100 summary: Issue #18408: Fix PyUnicode_AsUTF8AndSize(), raise MemoryError exception on memory allocation failure files: Objects/unicodeobject.c | 1 + 1 files changed, 1 insertions(+), 0 deletions(-)
diff --git a/Objects/unicodeobject.c b/Objects/unicodeobject.c --- a/Objects/unicodeobject.c +++ b/Objects/unicodeobject.c @@ -3766,6 +3766,7 @@ return NULL; _PyUnicode_UTF8(unicode) = PyObject_MALLOC(PyBytes_GET_SIZE(bytes) + 1); if (_PyUnicode_UTF8(unicode) == NULL) { + PyErr_NoMemory(); Py_DECREF(bytes); return NULL; }
Shouldn't this (and related commits from #18408) have been committed to the 3.3 branch? Georg
2013/10/29 Georg Brandl
diff --git a/Objects/unicodeobject.c b/Objects/unicodeobject.c --- a/Objects/unicodeobject.c +++ b/Objects/unicodeobject.c @@ -3766,6 +3766,7 @@ return NULL; _PyUnicode_UTF8(unicode) = PyObject_MALLOC(PyBytes_GET_SIZE(bytes) + 1); if (_PyUnicode_UTF8(unicode) == NULL) { + PyErr_NoMemory(); Py_DECREF(bytes); return NULL; }
Shouldn't this (and related commits from #18408) have been committed to the 3.3 branch?
All changes of #18408 "should" be backported, but I don't plan to backport them. It is not trivial to backport them. Nobody complained before (memory allocation failure are usually bad handled anyway, it will crash later if it does not crash here). And I'm not 100% confident that these changes would not break anything. Examples of possible regression: - PyEval_GetLocals() now raises an exception in case of an error. This change "should" not break anything, because callers were already raising an exception when PyEval_GetLocals() returns NULL. The builtin locals() function was not raising an exception in case of error, but it's probably because it is very unlikely that the function is called without any frame (PyEval_GetFrame(); returns NULL). - many functions now fail with an assertion error when they are called with an exception set (assert(!PyErr_Occurred());) because they may replace the exception without noticing the caller - I tried to check reference counters, but I may have introduce a regression leak in the error handling code If there is really a regression, I prefer to limit it to the new version, not to a stable version. Note: I'm not saying that I'm 0% confident in my changes :-) Victor
Gesendet: Dienstag, 29. Oktober 2013 um 10:54 Uhr Von: "Victor Stinner"
An: "Georg Brandl" Cc: "Python Dev" Betreff: Re: [Python-Dev] cpython: Issue #18408: Fix PyUnicode_AsUTF8AndSize(), raise MemoryError exception on 2013/10/29 Georg Brandl
: diff --git a/Objects/unicodeobject.c b/Objects/unicodeobject.c --- a/Objects/unicodeobject.c +++ b/Objects/unicodeobject.c @@ -3766,6 +3766,7 @@ return NULL; _PyUnicode_UTF8(unicode) = PyObject_MALLOC(PyBytes_GET_SIZE(bytes) + 1); if (_PyUnicode_UTF8(unicode) == NULL) { + PyErr_NoMemory(); Py_DECREF(bytes); return NULL; }
Shouldn't this (and related commits from #18408) have been committed to the 3.3 branch?
All changes of #18408 "should" be backported, but I don't plan to backport them. It is not trivial to backport them. Nobody complained before (memory allocation failure are usually bad handled anyway, it will crash later if it does not crash here). And I'm not 100% confident that these changes would not break anything.
OK, that's a good enough reason, if it's "only" no-memory-related errors.
Examples of possible regression:
- PyEval_GetLocals() now raises an exception in case of an error. This change "should" not break anything, because callers were already raising an exception when PyEval_GetLocals() returns NULL. The builtin locals() function was not raising an exception in case of error, but it's probably because it is very unlikely that the function is called without any frame (PyEval_GetFrame(); returns NULL).
- many functions now fail with an assertion error when they are called with an exception set (assert(!PyErr_Occurred());) because they may replace the exception without noticing the caller
- I tried to check reference counters, but I may have introduce a regression leak in the error handling code
If there is really a regression, I prefer to limit it to the new version, not to a stable version.
Note: I'm not saying that I'm 0% confident in my changes :-)
And I certainly think they have a better than 0% chance of being correct ;) Georg
On 29 October 2013 19:54, Victor Stinner
If there is really a regression, I prefer to limit it to the new version, not to a stable version.
Note: I'm not saying that I'm 0% confident in my changes :-)
Sounds like a solid rationale for limiting the changes to 3.4 to me :) /me should take another run at the known crasher tests one of these days. Maybe during the beta cycle... Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia
participants (3)
-
Georg Brandl
-
Nick Coghlan
-
Victor Stinner