Tuple size and memory allocation for embedded Python

[Craig Ringer]

On Fri, 2005-01-21 at 16:03 -0600, Jinming Xu wrote:
> Hi Folks,
> 
> Python seems unstable, when allocating big memory.  For example, the 
> following C++ code creates a tuple of tuples:
> 
>   PyObject* arCoord = PyTuple_New(n);
>   double d = 1.5;
>   for(int i=0; i<n; i++)
>     {
>       PyObject* coord = PyTuple_New(2);
>       PyTuple_SetItem(coord,0, PyFloat_FromDouble(d));//x
>       PyTuple_SetItem(coord,1, PyFloat_FromDouble(d));//y
>       PyTuple_SetItem(arCoord,i, coord);
>     }
> 
> When the n is small, say 100, the code works fine.  when n is big, say 
> 10,000, Python has trouble allocating memory, saying:
> 
> "Exception exceptions.IndexError: 'tuple index out of range' in 'garbage 
> collection' ignored
> Fatal Python error: unexpected exception during garbage collection
> Aborted"

You're not checking for errors from PyTuple_SetItem. You need to do so,
otherwise exceptions will go uncaught and may pop up at weird points
later in your software's execution, or crash things.


int PyTuple_SetItem(
PyObject *p, int pos, PyObject *o)
        Inserts a reference to object o at position pos of the tuple
        pointed to by p. It returns 0 on success. Note: This function
        ``steals'' a reference to o.
        
It returns an int result code, so you should probably be checking it.
You CERTAINLY should be ensuring that PyTuple_New() doesn't return NULL
(meaning a failure, probably of memory allocation).

I also don't see anything in there to resize the tuple.
 
http://docs.python.org/api/tupleObjects.html

--
Craig Ringer