Auto free Buffer objects in C API?

David M. Cooke cookedm+news at physics.mcmaster.ca
Thu Jul 3 21:44:27 EDT 2003


At some point, "Stuart D. Gathman" <stuart at bmsi.com> wrote:

> I have a Python C module (dspam.c - http://bmsi.com/python/milter.html).
> It needs to return some large buffers allocated by the library I am
> wrapping.  The buffers must be freed by the caller when finished.  At
> present, I do the following to copy, then free the buffer:
>
> PyObject *
> toBuffer(char *buf;int len) {
>   PyObject *sig = PyBuffer_New(len);
>   if (sig) {
>     void *data;
>     int dlen;
>     if (!PyObject_AsWriteBuffer(sig,&data,&dlen))
>       memcpy(data,buf,dlen);
>     else {
>       Py_DECREF(sig);
>       sig = 0;
>     }
>   }
>   free(buf);
>   return sig;
> }
>
> Now, I would like to use PyBuffer_FromMemory(buf,len), but then I have no
> way of knowing when to free(buf).  Or do I?  Am I missing something?  Is
> there a way for me to know when the Buffer object is being deleted so
> that I can free the underlying C data?

Staring at the definition of PyBufferObject in bufferobject.c, I see
it has a member 'b_base' of type PyObject *, which is DECREF'd when
the buffer is destroyed (by the buffer dealloc function). Looks to me
like you could do something using a CObject with a passed destructor:

/**********/
/* WARNING: UNTESTED CODE */

/* the contents of PyBufferObject are supposed to be private, so you
   need to include this */
typedef struct {
        PyObject_HEAD
        PyObject *b_base;
        void *b_ptr;
        int b_size;
        int b_readonly;
        long b_hash;
} PyBufferObject;

PyObject *
toBuffer(char *buf, int len) {
   PyObject *sig = PyBuffer_FromMemory(buf, len);
   if (sig) {
      /* pass 'free' as the destructor for the passed memory */
      (PyBufferObject *)sig->b_base = PyCObject_FromVoidPtr(buf, free);
   }
   return sig;
}
/**********/

Now, when the buffer is deallocated, the CObject at b_base will be
DECREF'd, which will free your memory.

This is a bit ugly, as it depends on internal details. Another,
cleaner, method is to define an object which obeys the buffer
protocol, and handles destruction like you want. (This is annoying, as
you'll end up copying all the code in bufferobject.c).

-- 
|>|\/|<
/--------------------------------------------------------------------------\
|David M. Cooke
|cookedm(at)physics(dot)mcmaster(dot)ca




More information about the Python-list mailing list