Passing back an array from an extension module?

[Bob Greschke]

Hmmm...I was wrong.  Creating a tuple object, then creating long
objects and adding them to the tuple 1,600,000 times doesn't take very
long.  On
the order of about 5 seconds (on a Sun Ultra10).  But it takes it
about 10 seconds if I do the same thing a 2nd, 3rd, 4th, etc. time.
Is it leaking memory, or is it just taking that long to garbage
collect the old tuple?  There is a lot of more disk activity on the
sebsequent runs than there is the first time through.

Bob

"Bob Greschke" <bob at passcal.nmt.edu> wrote in message
news:9fr8mp$219g$1 at newshost.nmt.edu...
> "This" is an array, but it doesn't look like I can either pass an
> array object to the C extension module, and pass an array back to
the
> Python code, or create an array in C and pass it back to the Python
> code.  I don't see anything in the Format Specifiers lists for
> PtArg_Parse() or Py_BuildValue() functions to handle arrays.
However,
> I do see C functions (Low-Level Functions on Built-in Types, Python
> Essential Reference, pg 261) for creating tuples in a C function,
and
> then functions for creating long integers, and then functions for
> adding items (_SetItem) to a tuple, and then the ability to pass
back
> an 'any object' in Py_BuildValue (the "O" format specifier).  Is
that
> a sensible thing to do?  It feels like it would chew up too much
time
> given the amount of data involved.
>
>
> "Martin von Loewis" <loewis at informatik.hu-berlin.de> wrote in
message
> news:j4ae3iev04.fsf at informatik.hu-berlin.de...
> > "Bob Greschke" <bob at passcal.nmt.edu> writes:
> >
> > > Then I need to pass that back to the Python side for graphing
> > > the values.  I initially thought I could just create a tuple and
> pass
> > > that back, since nothing is going to be done with the values
> except to
> > > draw a graph, but there is the possibility of there being
> 16,000,000
> > > data points...that makes for a slightly large statement, doesn't
> it?
> > > It doesn't look like I can do this with arrays in stock Python,
> but it
> > > does look like I can do it with the NumPy arrays.  Is that
> correct?
> >
> > Not sure what 'this' is here. You certainly can create 'custom'
> array
> > types in Python, see Modules/arraymodule for an example of an
array
> > type that is implemented using a C memory buffer of primitive C
> > values.
> >
> > I don't know whether you can use a NumPy array off-the-shelf; I
> assume
> > no since it probably manages its own memory.
> >
> > So you need to create a new type (see xxmodule.c for an example),
> and
> > implement its tp_as_sequence slot.
> >
> > > Now what about Py_DECREF, INCREF and all of that stuff in these
> > > situations?  I read through a number of posts on dejagoogle that
> were
> > > talking about this, but they only confused me more. :-)
> >
> > I recommend you read the "Embedding and Extending
> > tutorial". Basically, you need to implement an array accessor,
which
> > creates and returns a PyInt_FromLong every time you access the nth
> > element of your array. When the reference counter of your Python
> > wrapper object drops to zero, you should free(3) the memory of the
> > very large array.
> >
> > Regards,
> > Martin
> >
>
>