[Tutor] Most efficient way to read large csv files with properly converted mixed data types.
Ek Esawi
esawiek at gmail.com
Sat Jun 25 03:04:32 EDT 2016
Hi All--
My work involves reading large csv files with mixed data types (integer,
float, string, time and date). I was able to accomplish the task using (1)
genfromtxt or (2) looping through each line in the file and split, strip,
and assign data type to each entry.
I am wondering if there is a better and more efficient alternative,
especially to method 2 without using numpy or pandas. Alan Gauld mentioned
namedtuples for another question. I read a little about collections and in
particular namedtuples but was not sure how to apply theme here, if they
are applicable to begin with.
Thanks in advance--EKE
An example of a file:
A B C D E
1 2.3 ‘aa’ 10/01/2016 12:30
4 25.6 ‘bb’ 02/02/2015 1:30
More information about the Tutor
mailing list