Python C API String Memory Consumption

Carl Banks pavlovevidence at gmail.com
Tue Apr 7 04:40:27 EDT 2009


On Apr 7, 12:01 am, k3xji <sum... at gmail.com> wrote:
> When I run the following function, I seem to have a mem leak, a 20 mb
> of memory
> is allocated and is not freed. Here is the code I run:
>
> >>> import esauth
> >>> for i in range(1000000):
>
> ...     ss = esauth.penc('sumer')
> ...
>
> >>> for i in range(1000000):
>
> ...     ss = esauth.penc('sumer')
> ...
>
> And here is the penc() function.
>
> static PyObject *
> penc(PyObject *self, PyObject *args)
> {
>         unsigned char *s= NULL;
>         unsigned char *buf = NULL;
>         PyObject * result = NULL;
>         unsigned int v,len,i = 0;
>
>         if (!PyArg_ParseTuple(args, "s#", &s, &len))
>         return NULL;
>
>         buf = strdup(s);
>         if (!buf) {
>                 PyErr_SetString(PyExc_MemoryError,
>                         "Out of memory: strdup failed");
>                 return NULL;
>         }
>
>         /*string manipulation*/
>
>         result = PyString_FromString(buf);
>         free(buf);
>         return result;
>
> }
>
> Am I doing something wrong?


It might just be an unfortunate case where malloc keeps allocating
memory higher and higher on the heap even though it frees all the
memory.  And since it doesn't give it back to the OS, it runs out.

However, Python apparently does leak a reference if passed a Unicode
object; PyArg_ParseTuple automatically creates an encoded string but
never decrefs it.  (That might be necessary evil to preserve
compatibility, though.  PyString_AS_STRING does it too.)


Carl Banks



More information about the Python-list mailing list