Q: memory ownership in python extensions

[Bernhard Herzog]

fstajano at uk.research.att.com (Frank Stajano) writes:

> In the C code, I malloc some memory, perform some processing that
> writes a string in it, then turn this C string into a Python string
> via Py_BuildValue. The question is: Who is responsible for freeing
> that memory? Example code fragment follows.
> 
>              ...
> 	ciphertext = (byte*) malloc(pLen+1); /* who frees this? */
> 	if (!ciphertext) {
> 		PyErr_NoMemory();
> 		return NULL;
> 	}
> 	encrypt_block(key, kLen, plaintext, pLen, ciphertext);
> 	return Py_BuildValue("s#", ciphertext, pLen);
> 
> It would be logical and symmetrical for the code who malloc'ed to be
> the one responsible for free'ing. 

And that's exactly what you have to do.

> For this to happen, Py_BuildValue would have to take a copy. But I
> strongly suspect that, for efficiency reasons, Py_BuildValue does not
> take a copy and just reuses the memory passed to it; in which case the
> free would be done by the Python system when the reference count of
> the Python object goes to 0.

Py_BuildValue simply passes the char* to PyString_FromStringAndSize
which copies the data pointed to. 

A string object is one contiguous chunk of memory which includes the
Python object head (for refcount, type pointer etc) and the actual
string content. There's no provision for a string object using memory
allocated outside the string object code.

> Which one of these is the actual behaviour of Py_BuildValue is
> something that isn't at all apparent from Chapter 1.9 of "Extending
> and embedding" version 1.5.2 (hint hint). 

Well, one could argue that the examples suggest that the strings are
copied. Some of them use string literals.

-- 
Bernhard Herzog   | Sketch, a drawing program for Unix
herzog at online.de  | http://sketch.sourceforge.net/