[SciPy-User] saving a sparse matrix to file

[srean]

If you want to share sparse matrices across different applications a
good bet is the matrix market file format. Most sparse matrix
processing software will accept it.

There are examples here

http://www.scipy.org/SciPyPackages/Sparse#head-bb8f5e0195ae126e95a6be0c92de803887e9f300

But do note that its a ascii file format so the output file can be
long and can take a while to read or write.
We have used it and it worked great.

On some of the larger files however the writing was taking close to an
hour, so we sped it up by using multiprocessing to write different
parts of it. But becuse we needed only limited subset of all features
provided by the matrix market format our code only handles that
restricted subset and wont be generally useful.

Best
  srean

On Mon, Jun 25, 2012 at 1:34 PM, Pengkui Luo <pengkui.luo at gmail.com> wrote:
> Is there a unified API to save a sparse matrix of any type (dok, lil, csc,
> ...) to file, as simple as pickle dump/load?
>
> e.g. Can I do something like the following?
>
> import numpy as np
> from scipy.sparse import dok_matrix
> from scipy.io import save, load
> mat = dok_matrix (np.zeros ((80000, 80000)), np.int)
> save (file ('foobar.mat', 'wb'), mat)  # or mat.save ('foobar.mat')
> mat2 = load (file ('foobar.mat'))
>
>
> It seems that there was a .save method for a sparse matrix several years
> ago, but was later deprecated. Why?
> http://projects.scipy.org/scipy/ticket/590
>
> http://mail.scipy.org/pipermail/scipy-user/2012-January/031326.html
> I came across this email response stating that for csr_matrix one can
> save.data, .indices, and .indptr attributes individually to three different
> files and load them all to reassemble the matrix. Is it a hack only suitable
> for csr_matrix, or one that can be applied to all types of sparse matrices?
> Can I save a matrix to a single file instead of three?
>
> http://docs.scipy.org/doc/scipy/reference/generated/scipy.io.savemat.html
> I also came across the scipy.io.savemat API, but had no idea how to use
> it.... What does "mdict : Dictionary from which to save matfile variables."
> mean? What is this dict for? Can someone provide an example on how to use
> it?
>
> Thanks,
> Pengkui
> _______________________________________________
> SciPy-User mailing list
> SciPy-User at scipy.org
> http://mail.scipy.org/mailman/listinfo/scipy-user
>