Tuple size and memory allocation for embedded Python

[Jinming Xu]

>From: Tim Peters <tim.peters at gmail.com>
>Reply-To: Tim Peters <tim.peters at gmail.com>
>To: python-list at python.org
>Subject: Re: Tuple size and memory allocation for embedded Python
>Date: Fri, 21 Jan 2005 17:51:21 -0500
>
>[Jinming Xu]
> >> Python seems unstable, when allocating big memory.  For
> >> example, the following C++ code creates a tuple of tuples:
> >>
> >>   PyObject* arCoord = PyTuple_New(n);
> >>   double d = 1.5;
> >>   for(int i=0; i<n; i++)
> >>     {
> >>       PyObject* coord = PyTuple_New(2);
> >>       PyTuple_SetItem(coord,0, PyFloat_FromDouble(d));//x
> >>       PyTuple_SetItem(coord,1, PyFloat_FromDouble(d));//y
> >>       PyTuple_SetItem(arCoord,i, coord);
> >>     }
> >>
> >> When the n is small, say 100, the code works fine.  when n
> >> is big, say > 10,000, Python has trouble allocating
> >> memory, saying:
> >>
> >> "Exception exceptions.IndexError: 'tuple index out of range'
> >> in 'garbage collection' ignored
> >> Fatal Python error: unexpected exception during garbage
> >> collection
> >> Aborted"
>
>[Craig Ringer]
> > You're not checking for errors from PyTuple_SetItem.
>
>Or from PyTuple_New(), or from PyFloat_FromDouble().  They can all
>fail, and indeed:
>
> > You need to do so, otherwise exceptions will go uncaught
> > and may pop up at weird points later in your software's
> > execution, or crash things.
>
>That's right.  There's no point thinking about this at all before
>every C API call is checked for an error return.
>
>BTW, since the error occurred during garbage collection, there's
>really no reason to believe that the true cause of the problem is in
>the code shown.  It could simply be that allocating a whole lot of
>tuples here triggers a round of garbage collection, which in turn
>reveals an error in code we haven't been shown.  In fact, I expect
>that's actually the case.  The possible errors the OP is ignoring here
>are overwhemingly possible failures of malloc() to find more memory,
>and in those cases the C API calls shown would return NULL, and a
>segfault would be very likely soon after.
>--
>http://mail.python.org/mailman/listinfo/python-list

Hi folks,

Thank you all for your prompt answers.  I'll take a look for my code in the 
days coming.

Have a good weekend!

Jinming

_________________________________________________________________
Express yourself instantly with MSN Messenger! Download today - it's FREE! 
hthttp://messenger.msn.click-url.com/go/onm00200471ave/direct/01/