[Python-bugs-list] [Bug #125719] malloc() is called when _PyThreadState_Current is NULL
noreply@sourceforge.net
noreply@sourceforge.net
Thu, 14 Dec 2000 07:09:01 -0800
Bug #125719, was updated on 2000-Dec-13 12:50
Here is a current snapshot of the bug.
Project: Python
Category: Threads
Status: Closed
Resolution: Invalid
Bug Group: Not a Bug
Priority: 5
Submitted by: adeutsch
Assigned to : nobody
Summary: malloc() is called when _PyThreadState_Current is NULL
Details: We are implementing an embedded version of Python on an platform that does not have an unlimited amount of memory. In order to catch out-of-memory situations I defined the macro PyCore_MALLOC_FUNC to be equal to my own routine d_malloc(). d_malloc calls malloc() and checks the return value before returning it. If a NULL pointer is returned by malloc(), d_malloc() calls PyErr_NoMemory.
During testing, I discovered that PyCore_MALLOC_FUNC is called in PyOS_StdioReadline, which in turn is called by PyOS_Readline right after a call to Py_BEGIN_ALLOW_THREADS. In other words, at a time when the variable _PyThreadState_Current is set to NULL.
If this particular malloc() call fails, my routine will call PyErr_NoMemory(), which in turn calls PyErr_SetObject(), which calls PyErr_SetObject(), which calls PyErr_Restore(), which calls PyThreadState_GET(). Now if PyThreadState_GET() is called at a time when _PyThreadState_Current is equal to NULL, it will generate a fatal error about there being no current thread.
The net effect is that an out-of-memory situation can result in a misleading fatal error message about no current thread.
Perhaps, I should not be calling PyErr_NoMemory() in this situation, but after all that is what the routine is for. Another alternative would be for the Python source code not to call PyCore_MALLOC_FUNC from PyOS_Readline at all, but instead to call it from PyOS_StdioReadline just before the call to fgets().
Follow-Ups:
Date: 2000-Dec-14 07:09
By: gvanrossum
Comment:
Thanks for the report.
I'm checking in fixes for range and slice. Note that PyObject_Init() checks for a NULL argument, so it is safe already -- but PyRange_New() and PySlice_New() should not dereference the result if it i NULL!
-------------------------------------------------------
Date: 2000-Dec-14 05:44
By: adeutsch
Comment:
I just realized that I missed a layer in my previous comment, PyRange_New and PySlice_New both call PyObject_NEW which is defined in terms of PyObject_Init(PyObject_MALLOC(...)).
Here is the source code for both:
PyObject *
PyRange_New(long start, long len, long step, int reps)
{
rangeobject *obj = PyObject_NEW(rangeobject, &PyRange_Type);
obj->start = start;
obj->len = len;
obj->step = step;
obj->reps = reps;
return (PyObject *) obj;
}
PyObject *
PySlice_New(PyObject *start, PyObject *stop, PyObject *step)
{
PySliceObject *obj = PyObject_NEW(PySliceObject, &PySlice_Type);
if (step == NULL) step = Py_None;
Py_INCREF(step);
if (start == NULL) start = Py_None;
Py_INCREF(start);
if (stop == NULL) stop = Py_None;
Py_INCREF(stop);
obj->step = step;
obj->start = start;
obj->stop = stop;
return (PyObject *) obj;
}
-------------------------------------------------------
Date: 2000-Dec-14 05:37
By: adeutsch
Comment:
It is certainly possible that I am misguided. The reason I got into the whole issue of trapping on out-of-memory errors was that I found that on some occasions Python was not doing so and, as a result, was locking up our unit.
I just redownloaded the version 2.0 source code and found that in PyRange_New and PySlice_New the pointer returned by PyObject_Init is dereferenced and used without first checking to ensure that it is not NULL.
-------------------------------------------------------
Date: 2000-Dec-13 13:47
By: tim_one
Comment:
Unclear why you defined PyCore_MALLOC_FUNC: Python internals always check for a null malloc return already. If you bumped into a case where Python didn't, it's a Python bug.
-------------------------------------------------------
Date: 2000-Dec-13 13:14
By: gvanrossum
Comment:
I believe you are misguided. Python already checks the return value from malloc(), and calls PyErr_NoMemory(). If you find an instance of malloc() or realloc() that is not properly checked, I'd like to hear about it -- *that* would be a bug worth reporting.
So you shouldn't be calling PyErr_NoMemory() from inside your d_malloc() function.
-------------------------------------------------------
For detailed info, follow this link:
http://sourceforge.net/bugs/?func=detailbug&bug_id=125719&group_id=5470