Memory management question
I'm trying to port Don Beaudry's objectmodule to Python 2.0 and encountered the following problem. Here is a part from Don's code: co = (MetaClassObject*) PyClass_New(NULL, methods, name); co = PyObject_Realloc(co, sizeof (MetaClassObject)); As you see, he is trying to achive cheap code reuse. This code does not work. I have tracked it down to the following sample, which does also not work. In the debug version on windows, the PyObject_REALLOC fails with an assertion-error: _CrtIsValidHeapPointer(pUserData) op = PyObject_NEW(PyClassObject, &PyClass_Type); op = PyObject_REALLOC(op, sizeof(PyClassObject) + 20); Should the above work or am I doing something wrong? Regards, Thomas
Thomas Heller wrote:
I'm trying to port Don Beaudry's objectmodule to Python 2.0 and encountered the following problem. Here is a part from Don's code:
co = (MetaClassObject*) PyClass_New(NULL, methods, name); co = PyObject_Realloc(co, sizeof (MetaClassObject));
As you see, he is trying to achive cheap code reuse. This code does not work.
I have tracked it down to the following sample, which does also not work. In the debug version on windows, the PyObject_REALLOC fails with an assertion-error: _CrtIsValidHeapPointer(pUserData)
op = PyObject_NEW(PyClassObject, &PyClass_Type); op = PyObject_REALLOC(op, sizeof(PyClassObject) + 20);
Should the above work or am I doing something wrong?
Wouldn't it be safer to first create a MetaClassObject and then let the standard ClassObject initialize it ? -- Marc-Andre Lemburg ______________________________________________________________________ Company & Consulting: http://www.egenix.com/ Python Software: http://www.lemburg.com/python/
I have tracked it down to the following sample, which does also not work. In the debug version on windows, the PyObject_REALLOC fails with an assertion-error: _CrtIsValidHeapPointer(pUserData)
op = PyObject_NEW(PyClassObject, &PyClass_Type); op = PyObject_REALLOC(op, sizeof(PyClassObject) + 20);
Should the above work or am I doing something wrong?
Probably this doesn't work because of the GC header. The 'op' pointer does not point to the start of the allocated memory block. PyObject_NEW and friends know about this, but PyObject_REALLOC doesn't. That's what the assertion is trying to tell you. --Guido van Rossum (home page: http://www.python.org/~guido/)
I have tracked it down to the following sample, which does also not work. In the debug version on windows, the PyObject_REALLOC fails with an assertion-error: _CrtIsValidHeapPointer(pUserData)
op = PyObject_NEW(PyClassObject, &PyClass_Type); op = PyObject_REALLOC(op, sizeof(PyClassObject) + 20);
Should the above work or am I doing something wrong?
Probably this doesn't work because of the GC header. The 'op' pointer does not point to the start of the allocated memory block. PyObject_NEW and friends know about this, but PyObject_REALLOC doesn't. That's what the assertion is trying to tell you. Yes, I've also trakced it down to this.
I assume, PyObject_NEW is a friend of PyObject_DEL, and so are PyObject_MALLOC, PyObject_REALLOC, and PyObject_FREE - but the relationship between the first and the second category is somewhat cooler... Is there any (official) way to realloc the memory returned by PyObject_NEW ? (Always pushing the apis beyond their limits :-) Thomas
Is there any (official) way to realloc the memory returned by PyObject_NEW ?
Not if the object participates in GC. --Guido van Rossum (home page: http://www.python.org/~guido/)
I'll shut up after this one:
Would a PyObject_RESIZE function/macro make sense?
Thomas
Not for anybody else besides you, I think. --Guido van Rossum (home page: http://www.python.org/~guido/)
Guido van Rossum wrote:
Is there any (official) way to realloc the memory returned by PyObject_NEW ?
Not if the object participates in GC.
I going to see if I can fix this for 2.2. My current thinking is that there should be memory management APIs for GC objects, something like: PyObject_GC_New() PyObject_GC_NewVar() PyObject_GC_Realloc() PyObject_GC_Del() The non-GC APIs would no longer have to check the type flags which would be a bit of a speed win. The _AS_GC, _FROM_GC macros would not have to be used as much and the GC implementation would have more freedom about how to allocate things. Neil
I going to see if I can fix this for 2.2. My current thinking is that there should be memory management APIs for GC objects, something like:
PyObject_GC_New() PyObject_GC_NewVar() PyObject_GC_Realloc() PyObject_GC_Del()
The non-GC APIs would no longer have to check the type flags which would be a bit of a speed win. The _AS_GC, _FROM_GC macros would not have to be used as much and the GC implementation would have more freedom about how to allocate things.
Looks good! --Guido van Rossum (home page: http://www.python.org/~guido/)
participants (4)
-
Guido van Rossum -
M.-A. Lemburg -
Neil Schemenauer -
Thomas Heller