[Numpy-discussion] ragged array implimentation

Jeff Whitaker jswhit at fastmail.fm
Mon Mar 7 14:05:06 EST 2011

On 3/7/11 11:42 AM, Christopher Barker wrote:
> On 3/7/11 9:33 AM, Francesc Alted wrote:
>> A Monday 07 March 2011 18:28:11 Christopher Barker escrigué:
>>> I'm setting out to write some code to access and work with ragged
>>> arrays stored in netcdf files. It dawned on me that ragged arrays
>>> are not all that uncommon, so I'm wondering if any of you have any
>>> code you've developed that I could learn-from borrow from, etc.
>> A list of numpy arrays would not be enough?  Or you want something more
>> specific?
> Maybe that would, but in mapping to the netcdf data model, I'm thinking
> more like a big 1-d numpy array, with a way to index into it. Also, that
> would allow you to do some math with the arrays, if the broad casting
> made sense, anyway.
> But now that you've entered the conversation, does HDF and/or pytables
> have a standard way of dealing with this?
> On 3/7/11 9:37 AM, Jeff Whitaker wrote:
>> Chris:  The netcdf4-python modules reads netcdf vlen arrays
> arethose a netcdf4 feature?

Yes, although I don't think many people are using it.

See section 10 in 

> So far, I'm still workign with netcdf3 --
> though this could be a compelling reason to move on!
> We've talked about this some on the CF list, and I don't think anyone
> brought that up.
>> and returns
>> numpy object arrays, where the elements of the object arrays are
>> themselves 1d numpy arrays. I don't think there is any other way to do
>> it.  In your example, the 'ragged' array would be a 1d numpy array with
>> dtype='O', and the individual elements would be 1d numpy arrays with
>> dtype=int.  Of course, these arrays are very awkward to deal with and
>> operations will be slow.
> Depends some of the operations, but yes.
> That still may be the best option, but I'm exploring others.
> is a "vlen array" stored contiguously in netcdf?
Probably, although I don't know for sure what the underlying HDF5 layer 
is doing.

> -Chris

Jeffrey S. Whitaker         Phone  : (303)497-6313
Meteorologist               FAX    : (303)497-6449
NOAA/OAR/PSD  R/PSD1        Email  : Jeffrey.S.Whitaker at noaa.gov
325 Broadway                Office : Skaggs Research Cntr 1D-113
Boulder, CO, USA 80303-3328 Web    : http://tinyurl.com/5telg

More information about the NumPy-Discussion mailing list