On Tue, Sep 6, 2011 at 9:32 AM, Derek Homeier derek@astro.physik.uni-goettingen.de wrote:
On 02.09.2011, at 11:45PM, Christopher Jordan-Squire wrote:
and unfortunately it's for 1D-arrays only).
That's not bad for this use -- make a row a struct dtype, and you've got a 1-d array anyway -- you can optionally convert to a 2-d array after the fact.
I don't know why I didn't think of using fromiter() when I build accumulator. Though what I did is a bit more flexible -- you can add stuff later on, too, you don't need to do it allat once.
I'm unsure how to use fromiter for missing data. It sounds like a potential solution when no data is missing, though.
Strange I haven't thought about it before either; I guess for record arrays it comes more natural to view them as a collection of 1D arrays. However, you'd need to construct a list or something of ncolumn iterators from the input - should not be too hard; but then how do you feed the ncolumn fromiter() instances synchronously from that?? As far as I can see there is no way to make them read one item at a time, row by row. Then there are additional complications with multi-D dtypes, and in your case, especially datetime instances, but the problem that all columns have to be read in in parallel really seems to be the showstopper here. Of course for "flat" 2D arrays of data (all the same dtype) this would work with simply reshaping the array - that's probably even the most common use case for loadtxt, but that method lacks way too much generality for my taste. Back to accumulator, I suppose.
Yes, I believe the thinking was that if your data is all one dtype that's simple enough to figure out, and there are other method for reading in such an array to produce a 2-d array. This is strictly for structured arrays currently, though I suppose that could change.
-Chris
Cheers, Derek
--
Derek Homeier Centre de Recherche Astrophysique de Lyon ENS Lyon 46, Allée d'Italie 69364 Lyon Cedex 07, France +33 1133 47272-8894
NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion