[Tutor] Processing CSV files

Leena Gupta gupta.leena at gmail.com
Tue Oct 8 22:46:12 CEST 2013


Hello,

Looking for some inputs on Python's csv processing feature.

I need to process a large csv file every 5-10 minutes. The file could
contain 3mill to 10 mill rows and size could be 6MB to 10MB(+). As part of
the processing, I need to sum up a number value by grouping on certain
attributes and store the output in a datastore. I wanted to know if Python
is recommended and can it be used for processing data in csv files of this
size? Any issues that we need to be aware of? I believe Python has a csv
library as well.

Thanks!
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/tutor/attachments/20131008/16c10881/attachment.html>


More information about the Tutor mailing list