[Matrix-SIG] Reading and handling matrices.

Travis Oliphant Oliphant.Travis@mayo.edu
Fri, 23 Apr 1999 13:24:31 -0500 (CDT)


> I'm just starting with python and I'm having a few problems, which I =
> suspect are rather simple-minded.

Welcome aboard, starting with a new system can be intimidating.

> 1. I was looking for some sparse matrix functions that would let me =
> compute the eigen vectors of a rather large matrix in a reasonable =
> amount of time.  I have found the eigen value functions in the numeric =
> library and I have found some linear algebra routines that allow sparse
> matrices, but I do not see a way to combine these.  I ended up saving =
> the matrix as ascii, editing it using word, reading it using excel, and
> then importing it to matlab.  There has to be a better way.

Sparse matrices:

As Konrad said there are several ways to handle sparse matrices, but
no "comprehensive" package. There is a sparse matrix module that I can't
say anything about except that it exists.  It defines a sparse matrix
object and some methods for linear equation solving.  Look on
http://www.python.org/topics/scicomp/numbercrunching.html
for the Sparse Linear Equation Solver module by Neil Schemenauer

Adding good sparse matrix support is something I'll be trying to add to my
collection of useful packages for NumPy over the next year or two.  When I
get to that point I'll likely start with the module on that page.

It sounds like you went through a lot of work getting data in and out of
NumPy arrays.  This was a source of confusion for me at first, too, so I
understand a bit of the frustration.  The frustration does disappear
as you get a feel for the structure of Python and the Numeric extensions.

I wanted to read arbitrary binary files and so wrote a C-extension module
that reads arbitrary binary data into Python NumPy arrays.  It turns out,
I could have accomplished what I wanted to do with the fromstring function
from the Numeric module and the read method of any file object.

One method to get your data into MATLAB is to open a file object in
Python and write the data in your NumPy array as a "string" (i.e. a
sequence of raw bytes).

>>> import Numeric
>>> a = Numeric.ones((512,512))  # an uninteresting array of integers
>>> fid = open("myfile","w")
>>> fid.write(a.tostring())
>>> fid.close()

If you are on a 32-bit machine, you will now have a file called "myfile"
in the current directory that should contain (512*512*4) bytes of 4-byte
integer ones.

You can read this data into MATLAB using matlab's fread function:

>> fid = fopen('myfile');
>> a = fread(fid,512*512,'int');
>> fclose(fid);

The same sequence works in reverse use MATLAB's fwrite function and
Numeric Python's fromstring function and the read method of an open file
object.

You should NEVER have to edit numbers by hand in Word.  It makes me sad
that you felt the need to do that.  This is a pretty low level way to read
and write data, but is indispensable when interfacing with other code and
outputs from other code.  A better solution is to use a higher level file
format like NETCDF which you can read and write to from MATLAB and NumPy.

Of course, I would wonder why you were doing much work in MATLAB anyway
:-) (and if you are then what is NumPy missing and we'll add it.)

If you want it I could send you a module I rarely use now that makes the
NumPy way of reading and writing binary data more like MATLAB's which I
was used, too when I started.

Best of luck on your newfound tool.  Feel free to ask anymore questions as
they arise.

Sincerely,

Travis Oliphant.

(just a run-o'-the-mill NumPy fan)