I think it would be a great idea to have pylab.load in numpy. It also seems to be a lot faster than scipy.io.
One thing that is very nice about pylab.load is that it can read-in dates. However, it can't, as far a I know, handle other non-float data.
I played around with python's csv module and pylab.load for a while resulting in a database class I posted in the cookbook section:
http://www.scipy.org/Cookbook/dbase
This class can read any type of data in a csv file, including dates, into a dictionary but is based on both pylab.load and the csv module. I use cPickle for storing the data once it is read-in once. I haven't tried PyTables but hear a lot of good things about it.
Vincent
On 4/19/07 10:58 AM, "Christopher Barker" Chris.Barker@noaa.gov wrote:
Lisandro Dalcin wrote:
I am also +1 on this, but this functionality should be implemented in C, I think.
well, maybe.
I've just tested numpy.fromfile('name.txt', sep=' ') against pylab.load('name.txt') for a 35MB text file, the number are:
numpy.fromfile: 2.66 sec. pylab.load: 16.64 sec.
exactly that's expected. fromfile is designed to do the easy cases as fast as possible, pylab.load is designed to be be flexible, I'm not user you need both the speed and flexibility at the same time.
By the way, I haven't looked at pylab.load() for a while, but it could perhaps be sped up by using fromfile() and or fromstring internally. There may be some opportunity to special case the easy ones too (i.e. all columns desired, etc.)
-Chris