[Python-bugs-list] [Bug #125719] malloc() is called when _PyThreadState_Current is NULL

[noreply@sourceforge.net]

Bug #125719, was updated on 2000-Dec-13 12:50
Here is a current snapshot of the bug.

Project: Python
Category: Threads
Status: Closed
Resolution: Invalid
Bug Group: Not a Bug
Priority: 5
Submitted by: adeutsch
Assigned to : nobody
Summary: malloc() is called when _PyThreadState_Current  is NULL

Details: We are implementing an embedded version of Python on an platform that does not have an unlimited amount of memory.  In order to catch out-of-memory situations I defined the macro PyCore_MALLOC_FUNC to be equal to my own routine d_malloc().  d_malloc calls malloc() and checks the return value before returning it.  If a NULL pointer is returned by malloc(), d_malloc() calls PyErr_NoMemory.

During testing, I discovered that PyCore_MALLOC_FUNC is called in PyOS_StdioReadline, which in turn is called by PyOS_Readline right after a call to Py_BEGIN_ALLOW_THREADS.  In other words, at a time when the variable _PyThreadState_Current is set to NULL.

If this particular malloc() call fails, my routine will call PyErr_NoMemory(), which in turn calls PyErr_SetObject(), which calls PyErr_SetObject(), which calls PyErr_Restore(), which calls PyThreadState_GET().  Now if PyThreadState_GET() is called at a time when _PyThreadState_Current is equal to NULL, it will generate a fatal error about there being no current thread.

The net effect is that an out-of-memory situation can result in a misleading fatal error message about no current thread.

Perhaps, I should not be calling PyErr_NoMemory() in this situation, but after all that is what the routine is for.  Another alternative would be for the Python source code not to call PyCore_MALLOC_FUNC from PyOS_Readline at all, but instead to call it from PyOS_StdioReadline just before the call to fgets().



Follow-Ups:

Date: 2000-Dec-14 05:44
By: adeutsch

Comment:
I just realized that I missed a layer in my previous comment, PyRange_New and PySlice_New both call PyObject_NEW which is defined in terms of PyObject_Init(PyObject_MALLOC(...)).

Here is the source code for both:

PyObject *
PyRange_New(long start, long len, long step, int reps)
{
	rangeobject *obj = PyObject_NEW(rangeobject, &PyRange_Type);

	obj->start = start;
	obj->len   = len;
	obj->step  = step;
	obj->reps  = reps;

	return (PyObject *) obj;
}


PyObject *
PySlice_New(PyObject *start, PyObject *stop, PyObject *step)
{
	PySliceObject *obj = PyObject_NEW(PySliceObject, &PySlice_Type);

	if (step == NULL) step = Py_None;
	Py_INCREF(step);
	if (start == NULL) start = Py_None;
	Py_INCREF(start);
	if (stop == NULL) stop = Py_None;
	Py_INCREF(stop);

	obj->step = step;
	obj->start = start;
	obj->stop = stop;

	return (PyObject *) obj;
}

-------------------------------------------------------

Date: 2000-Dec-14 05:37
By: adeutsch

Comment:
It is certainly possible that I am misguided.  The reason I got into the whole issue of trapping on out-of-memory errors was that I found that on some occasions Python was not doing so and, as a result, was locking up our unit.

I just redownloaded the version 2.0 source code and found that in PyRange_New and PySlice_New the pointer returned by PyObject_Init is dereferenced and used without first checking to ensure that it is not NULL.
-------------------------------------------------------

Date: 2000-Dec-13 13:47
By: tim_one

Comment:
Unclear why you defined PyCore_MALLOC_FUNC:  Python internals always check for a null malloc return already.  If you bumped into a case where Python didn't, it's a Python bug.

-------------------------------------------------------

Date: 2000-Dec-13 13:14
By: gvanrossum

Comment:
I believe you are misguided.  Python already checks the return value from malloc(), and calls PyErr_NoMemory().  If you find an instance of malloc() or realloc() that is not properly checked, I'd like to hear about it -- *that* would be a bug worth reporting.

So you shouldn't be calling PyErr_NoMemory() from inside your d_malloc() function.

-------------------------------------------------------

For detailed info, follow this link:
http://sourceforge.net/bugs/?func=detailbug&bug_id=125719&group_id=5470