[SciPy-User] question: OLS with sparse design and dense parameter

[josef.pktd at gmail.com]

(sunday night quest for another sparse recipe)

This is a sparse algorithm question:

scipy.sparse.linalg has a choice of linear system solvers, but I have not
much idea about the relative merits. Is there a recommendation for the
following case?

suppose I want to do estimate a standard linear model by OLS
y = x * b,  find b = argmin(y - x*b)

x is sparse, with blocks, medium large (20000 by 2000) but if it works
quite a bit larger,
but we don't have a small number of dominant singular values, the solution
of the linear system will be dense.

the basic parts of my minimum example

nobsi, n_groups, k_vars = 10, 2000, 3

#nobsi, n_groups, k_vars = 2, 3, 3


xi = np.arange(nobsi)[:,None]**np.arange(3) #vander

x_sp = sparse.kron(sparse.eye(n_groups), xi).tocsr()

(balanced kron structure is just to make a quick example)


x_sp.shape

(20000, 6000)


first try


xpx = x_sp.T.dot(x_sp) # sparse

xpy = x_sp.T.dot(y) # dense


p_dense = np.linalg.solve(xpx.toarray(), xpy)

p_sp1 = sparse.linalg.spsolve(xpx, xpy)

p_sp2 = sparse.linalg.lsmr(x_sp, y)[0]


timing: 15.6659998894 0.00500011444092 0.00600004196167


Is this too small to worry, and anything works, or is there a
recommendation?

definitely don't use dense.


Thanks,


Josef
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.scipy.org/pipermail/scipy-user/attachments/20140427/dff3c453/attachment.html>