[MATRIX-SIG] Data import into Numeric arrays

[Janko Hauser]

Hi, attached is a much faster version for reading, than the one of
Konrad. It also seems to me, that the most inefficient part of data
reading is the conversion from strings to numbers. 

I think the time for str=file.read() is reasonable fast. The
conversion took most of the time. This is where sscanf is really good
at. It is also nice to to use sscanf as an filter like in 

    def scan(self, lineformat, typecode = Float):
        saved = []
        line = 1
        while line:
            try:
                data = sscanf(line, lineformat)
                if data:
                    saved.append(data)
                line = self.file.readline()
            except:
                line = self.file.readline()
                pass

        return asarray(saved, typecode)

with a lineformat of:
        a = self.file.scan(" Day = %f  at iic = %f avgKE = %f avgPE = %f")

This skips all lines, which are not of this format, good for report
creation. You see, I have build a class for this, but this is rather
clumsy at the moment, but I can mail it to you if you want. The bad
part at the moment is, that I don't have something equivilant for
writing.

If something more general should be implemented, then the question
arises, what we want.

o One 1d or 2d array per file

o More of these but all with the same length in the last dimension.

o Can text be inserted? Also between the numbers? TeX-tables ...

o Should this be a new file object, or some new methods for the
arraytype?

Besides Yoricks read is only ca. 2 times faster than read_table. And
Matlab very similar to read_table. Not so bad at all.

__Janko



def read_table(fname,header=0):
    """read_table(filename,header=number_of_header_lines)

    Reads from filename after number_of_header_lines
    """

    from sscanf import sscanf
    import string

    try:
	fp=open(fname,'r')
    except:
	print "Can't open file or wrong filename in function read_table"
	return

    scanline='%f'
    lines = fp.readlines()
    fp.close()
    n=len(string.split(lines[header]))
    new_table=ones((len(lines)-header,n),typecode = Float)
    while n-1:
	n=n-1
	scanline=scanline+' %f'
    for item in xrange(len(lines)-header):
        a=array(sscanf(lines[item],scanline))
        new_table[item,:] = a

    return(new_table)

_______________
MATRIX-SIG  - SIG on Matrix Math for Python

send messages to: matrix-sig@python.org
administrivia to: matrix-sig-request@python.org
_______________