[SciPy-User] SpMV operation performance

[Wenlei Xie]

Thank you guys! However it looks like none of these holds:

In [50]: np.isnan(V).any()
Out[50]: False

In [51]: (abs(V) < 1e-100).any()
Out[51]: False

In [52]: (abs(V) > 1e+100).any()
Out[52]: False

In [53]: V.dtype
Out[53]: dtype('float64')


I think it's probably the matrix contains some strange values that slowed
things down. But I don't have a clear clue (it is generated by scipy.svd).

Can any one give me a reference to the implantation to the .dot() for
sparse matrices?

Another interesting observation is that if I .dot() each column of the
matrix V, the running time is much shorter than .dot() all of them:

In [97]: time T = A.dot(V[:, 0])
CPU times: user 0.00 s, sys: 0.00 s, total: 0.00 s
Wall time: 0.00 s

In [98]: time T = A.dot(V[:, 1])
CPU times: user 0.00 s, sys: 0.00 s, total: 0.00 s
Wall time: 0.00 s

In [99]: time T = A.dot(V[:, 0:2])
CPU times: user 0.06 s, sys: 0.01 s, total: 0.07 s
Wall time: 0.07 s

In [100]: time T = A.dot(V)
CPU times: user 3.38 s, sys: 0.52 s, total: 3.90 s
Wall time: 3.91 s

(And, most of the time seems to be spent on user-mode?)


Thank you!

Best,
Wenlei



On Tue, May 6, 2014 at 1:06 PM, Pauli Virtanen <pav at iki.fi> wrote:

> 06.05.2014 02:35, Wenlei Xie kirjoitti:
> > I got very strange behavior of SpMV operations on SciPy. The operation
> is A
> > * b, where A is a 160 * 3494258  sparse matrix with 134K non-zero
> entries,
> > b is a 3494258 * 100 dense matrix. I tried to use a generated b and
> random
> > b, which got very different performance:
> >
> > PROFILE: A.dot(b1) finished, times taken 3.31 sec.
> > PROFILE: A.dot(b2) finished, times taken 0.06 sec.
> [clip]
>
> Some possibilities that may slow down floating point math:
>
> - denormal numbers (|a| < ~1e-300 and |a| > ~1e300)
> - NaNs
>
> The first can in principle be controlled by compiler flags used when
> compiling Scipy.
>
> --
> Pauli Virtanen
>
> _______________________________________________
> SciPy-User mailing list
> SciPy-User at scipy.org
> http://mail.scipy.org/mailman/listinfo/scipy-user
>



-- 
Wenlei Xie (谢文磊)

Department of Computer Science
5132 Upson Hall, Cornell University
Ithaca, NY 14853, USA
Phone: (607) 255-5577
Email: wenlei.xie at gmail.com
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.scipy.org/pipermail/scipy-user/attachments/20140506/40bd0f2c/attachment.html>