best way to read a huge ascii file.
Marko Rauhamaa
marko at pacujo.net
Fri Nov 25 10:29:44 EST 2016
Heli <hemla21 at gmail.com>:
> I have a huge ascii file(40G) and I have around 100M lines. I read this
> file using :
>
> [...]
>
> The problem is reading the file takes around 80 min while the
> interpolation only takes 15 mins.
>
> I was wondering if there is a more optimized way to read the file that
> would reduce the time to read the input file?
It looks pretty fast to me already; that's 20,000 lines per second, or
65 megabits per second.
Marko
More information about the Python-list
mailing list