[Numpy-discussion] Memory efficient alternative for np.loadtxt and np.genfromtxt
Saullo Castro
saullogiovani at gmail.com
Sun Oct 26 04:46:48 EDT 2014
I would like to start working on a memory efficient alternative for
np.loadtxt and np.genfromtxt that uses arrays instead of lists to store the
data while the file iterator is exhausted.
The motivation came from this SO question:
http://stackoverflow.com/q/26569852/832621
where for huge arrays the current NumPy ASCII readers are really slow and
require ~6 times more memory. This case I tested with Pandas' read_csv()
and it required 2 times more memory.
I would be glad if you could share your experience on this matter.
Greetings,
Saullo
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20141026/cc67acfa/attachment.html>
More information about the NumPy-Discussion
mailing list