[Numpy-discussion] convert csv file into recarray without pre-specifying dtypes and variable names
v-nijs at kellogg.northwestern.edu
Mon Jul 9 09:55:25 EDT 2007
Cool! Thanks Tim.
On 7/8/07 10:25 PM, "Timothy Hochberg" <tim.hochberg at ieee.org> wrote:
> On 7/8/07, Vincent Nijs <v-nijs at kellogg.northwestern.edu> wrote:
>> Thanks for looking into this Torgil! I agree that this is a much more
>> complicated setup. I'll check if there is anything I can do on the data end.
>> Otherwise I'll go with Timothy's suggestion and read in numbers as floats
>> and convert to int later as needed.
> Here is a strategy that should allow auto detection without too much in the
> way of inefficiency. The basic idea is to convert till you run into a problem,
> store that data away, and continue the conversion with a new dtype. At the end
> you assemble all the chunks of data you've accumulated into one large array.
> It should be reasonably efficient in terms of both memory and speed.
> The implementation is a little rough, but it should get the idea across.
-------------- next part --------------
An HTML attachment was scrubbed...
More information about the NumPy-Discussion