[Numpy-discussion] ragged array implimentation
jswhit at fastmail.fm
Mon Mar 7 14:05:06 EST 2011
On 3/7/11 11:42 AM, Christopher Barker wrote:
> On 3/7/11 9:33 AM, Francesc Alted wrote:
>> A Monday 07 March 2011 18:28:11 Christopher Barker escrigué:
>>> I'm setting out to write some code to access and work with ragged
>>> arrays stored in netcdf files. It dawned on me that ragged arrays
>>> are not all that uncommon, so I'm wondering if any of you have any
>>> code you've developed that I could learn-from borrow from, etc.
>> A list of numpy arrays would not be enough? Or you want something more
> Maybe that would, but in mapping to the netcdf data model, I'm thinking
> more like a big 1-d numpy array, with a way to index into it. Also, that
> would allow you to do some math with the arrays, if the broad casting
> made sense, anyway.
> But now that you've entered the conversation, does HDF and/or pytables
> have a standard way of dealing with this?
> On 3/7/11 9:37 AM, Jeff Whitaker wrote:
>> Chris: The netcdf4-python modules reads netcdf vlen arrays
> arethose a netcdf4 feature?
Yes, although I don't think many people are using it.
See section 10 in
> So far, I'm still workign with netcdf3 --
> though this could be a compelling reason to move on!
> We've talked about this some on the CF list, and I don't think anyone
> brought that up.
>> and returns
>> numpy object arrays, where the elements of the object arrays are
>> themselves 1d numpy arrays. I don't think there is any other way to do
>> it. In your example, the 'ragged' array would be a 1d numpy array with
>> dtype='O', and the individual elements would be 1d numpy arrays with
>> dtype=int. Of course, these arrays are very awkward to deal with and
>> operations will be slow.
> Depends some of the operations, but yes.
> That still may be the best option, but I'm exploring others.
> is a "vlen array" stored contiguously in netcdf?
Probably, although I don't know for sure what the underlying HDF5 layer
Jeffrey S. Whitaker Phone : (303)497-6313
Meteorologist FAX : (303)497-6449
NOAA/OAR/PSD R/PSD1 Email : Jeffrey.S.Whitaker at noaa.gov
325 Broadway Office : Skaggs Research Cntr 1D-113
Boulder, CO, USA 80303-3328 Web : http://tinyurl.com/5telg
More information about the NumPy-Discussion