[Tutor] Most efficient way to read large csv files with properly converted mixed data types.

Ek Esawi esawiek at gmail.com
Sat Jun 25 03:04:32 EDT 2016


Hi All--



My work involves reading large csv files with mixed data types (integer,
float, string, time and date). I was able to accomplish the task using (1)
genfromtxt or (2) looping through each line in the file and split, strip,
and assign data type to each entry.



I am wondering if there is a better and more efficient alternative,
especially to method 2 without using numpy or pandas. Alan Gauld mentioned
namedtuples for another question. I read a little about collections and in
particular namedtuples but was not sure how to apply theme here, if they
are applicable to begin with.



Thanks in advance--EKE



An example of a file:



A         B         C         D                     E

1          2.3       ‘aa’      10/01/2016      12:30

4          25.6     ‘bb’      02/02/2015      1:30


More information about the Tutor mailing list