[SciPy-user] combine two sparse matrices

[Robin]

On Sat, May 3, 2008 at 4:24 PM, Nathan Bell <wnbell at gmail.com> wrote:
>
>  If you're worried about speed or memory consumption, then you should
>  avoid lil_matrix and dok_matrix.
>
>  The fastest and most memory efficient approach uses coo_matrix:
>
>  row = array([2,3,1,4])
>  col = array([4,2,3,1])
>  data = array([5,5,5,5])
>  A = coo_matrix( (data,(row,col)), shape=(5,5)).tocsr()
>
>  The conversion to CSR forces a copy, however CSR and COO are so much
>  more memory efficient than LIL/DOK that it shouldn't matter.

Thanks - I was just thinking about COO matrices. The problem is I have
to build it up in parts - I know lists are more efficient to extend
than arrays so I could store the data, row, col as lists and extend
with each chunk - then I think I can construct the COO matrix from the
list. I was worried that the lists would stay around though - even if
deleted so then I wouldn't have enough memory to do the COO-CSC
conversion.

Perhaps I could add another layer of indirection with another fork()
though to make sure the lists are cleaned up and just pass back the
COO matrix - hopefully the COO matrix will pickle OK.

Thanks,

Robin