How properly manage memory of this PyObject* array?? (C extension)

[John Machin]

On 18/07/2006 1:45 PM, seberino at spawar.navy.mil wrote:
>> Let's try reductio ad adsurdum on that one. Suppose that instead of
>> filling in a malloced chunk of memory, you had stored those gizmoids in
>> local variables foo0, foo1, foo2, etc. Using your reasoning: we can't
>> ever return from our function (which frees up the stack memory
>> containing foo0 etc) because we don't know when garbage collector will
>> free foo0 etc. Avoiding that problem would require a Python/C API
>> function with a name like Py_AwaitGarbageCollection() ... but there
>> isn't one.
> 
> There are 2 memory components that make up a Python object.
> First, there is the object in memory itself.  Secondly, there
> is the 4 bytes or so that hold the address to the object in memory.
> (the 'pointer')

*WRONG*. The object exists in and of itself. There may be one *or more* 
references to it, via pointers, scattered about in memory; they are 
*NOT* components of the object. A reference count is maintained inside 
the object and manipulated by Py_INCREF etc. The Python garbage 
collector knows *nothing* about the memory occupied by those pointers; 
it is *your* responsibility to allocate and free them. If the pointers 
are in a function's local variables, that memory is automatically 
returned when you return from the function. If the pointers are in a 
chunk of memory that you grabbed using malloc(), then you must free() 
the chunk as soon as you are finished with it.

Like I said:

You malloced it? You free it.
You didn't malloc it? You don't free it.

> 
> I think what I hear you saying is that garbage collection has
> to automagically take care of my second component above as well.

I'm terribly sorry, but there is no magic involved with memory 
management at this level; only hard, tedious, error-prone work.

Python garbage collection has neither the responsibility nor the 
necessary information for carrying out that task. *YOU* have that 
information; it is *YOUR* responsibility.

> 
> So can I assume you are also saying our original malloc'd pointer
> to Python objects will be take care of as well as the objects?

*NO* You must free() any memory that you malloc()ed.

> (Hence, no free(my_array) necessary?)

*NO* You must free() any memory that you malloc()ed.

> (Only Py_INCREFs and Py_DECREFs need apply?)

*NO* You must free() any memory that you malloc()ed.

"What I tell you three times is true" -- the Bellman in "The Hunting of 
the Snark", by Lewis Carroll.

HTH,
John