[Numpy-discussion] convert csv file into recarray without pre-specifying dtypes and variable names
torgil.svensson at gmail.com
Wed Jul 18 20:57:44 EDT 2007
I haven't gone through all details. That's a nice new "missing"
feature, maybe all instances where we can't find a conversion should
be "nan". A few comments:
1. The "load_search" functions contains all memory/performance
overhead that we wanted to avoid with the fromiter function. Does this
mean that you no longer have large text-files that change sting
representation in the columns (aka "0" floats) ?
2. ident=" "*4
This has the same spelling error as in my first compile try .. it was
meant to be "indent"
3. types = list((i,j) for i, j in zip(varnm, types2))
Isn't this the same as "types = zip(varnm, types2)" ?
4. return N.fromiter(iter(reader),dtype = types)
Isn't "reader" an iterator already? What does the "iter()" operator do
in this case?
On 7/18/07, Vincent Nijs <v-nijs at kellogg.northwestern.edu> wrote:
> I combined some of the very useful comments/code from Tim and Torgil and
> came-up with the attached program to read csv files and convert the data
> into a recarray. I couldn't use all of their suggestions because, frankly, I
> didn't understand all of them :)
> The program use variable names if provided in the csv-file and can
> auto-detect data types. However, I also wanted to make it easy to specify
> data types and/or variables names if so desired. Examples are at the bottom
> of the file. Comments are very welcome.
> Numpy-discussion mailing list
> Numpy-discussion at scipy.org
More information about the NumPy-Discussion