Howto create a record array from arrays without copying their data
Hi, I'm creating (actually calculating) a set of very large 1-d arrays (vectors), which I would like to assemble into a record array so I can access the data row-wise. Unfortunately it seems that all data of my original 1-d arrays are getting copied in memory during that process. Is there a way to get around that? Basically what I do is: arr1 = numpy.array([1, 4, 5], dtype=int) arr2 = numpy.array([5.5, 6.6, 9.9], dtype=float) recarray = numpy.core.rec.fromarrays([arr1, arr2], names=['col1', 'col2']) When I now make a change in recarray like recarray.col1[0] = 5000 I cannot see the change in arr1. (So that's why I assume that the data is copied). Also in the numy book it tells that would be copied. Thanks, Ralph
On Wed, Aug 12, 2009 at 10:22 AM, Ralph Heinkel <ralph@dont-mind.de> wrote:
Hi,
I'm creating (actually calculating) a set of very large 1-d arrays (vectors), which I would like to assemble into a record array so I can access the data row-wise. Unfortunately it seems that all data of my original 1-d arrays are getting copied in memory during that process. Is there a way to get around that?
I don't think so, because fundamentally numpy assumes array elements are packed together in memory. If you know C, record arrays are pretty much arrays of structures. You could try just using a python dictionary to hold the arrays, depending on you motives behind using a record array. Ryan -- Ryan May Graduate Research Assistant School of Meteorology University of Oklahoma Sent from Norman, Oklahoma, United States
On Wed, Aug 12, 2009 at 11:28 AM, Ryan May<rmay31@gmail.com> wrote:
On Wed, Aug 12, 2009 at 10:22 AM, Ralph Heinkel <ralph@dont-mind.de> wrote:
Hi,
I'm creating (actually calculating) a set of very large 1-d arrays (vectors), which I would like to assemble into a record array so I can access the data row-wise. Unfortunately it seems that all data of my original 1-d arrays are getting copied in memory during that process. Is there a way to get around that?
I don't think so, because fundamentally numpy assumes array elements are packed together in memory. If you know C, record arrays are pretty much arrays of structures. You could try just using a python dictionary to hold the arrays, depending on you motives behind using a record array.
Ryan
Can you preallocate the record array and fill it up as you calculate the values? You can use the reference to the recarray columns. I never tried with recarrays. Josef
participants (3)
-
josef.pktd＠gmail.com -
Ralph Heinkel -
Ryan May