[SciPy-Dev] Minimal sparse array interface

[CJ Carey]

I wrote a lightweight sparse ndarray library a few years ago:
https://github.com/perimosocordiae/sparray

It uses Cython for some hot loops, otherwise it's fairly simple. It uses a
flattened COO-like format that's pretty efficient for common ndarray use
cases. I was planning to add other backends, but then I finished my degree
and got a Real Job.

Around the same time, Evgeni was experimenting with
https://github.com/ev-br/sparr, which uses a generalized DOK format for
sparse ndarrays.

Both of these haven't seen development or use in a while, but if you're
looking to put a sparse ndarray together they might be useful starting
points.

On Thu, Jun 3, 2021 at 4:03 PM Stefan van der Walt <stefanv at berkeley.edu>
wrote:

> Hi Evgeni,
>
> On Thu, Jun 3, 2021, at 12:34, Evgeni Burovski wrote:
> > You're thinking something like a lightweight `csr_array` which is
> > almost identical to the existing csr_matrix, but has  a np.array-like
> > multiplication?
>
> Exactly, and that also conforms with indexing expectations.
>
> > 1. What's the minimal subset of formats? Would CSR be enough, or some
> > others are needed from the start?
>
> I suspect we will have to cover most of them in order for this to be
> useful.  I just looked at NetworkX, for example, and they use CSR, LIL, and
> COO at least.
>
> > 2. Should this live in scipy.sparse or keep it in a separate repo in
> > the scipy org for a while?
>
> Most of it we can probably build separately, but it may require a few
> tweaks to the existing classes.  Those changes could simply be PRs (once
> there is rough consensus that we want to go this route).
>
> > 3.What about d != 2 ? E.g., does array[0, :] behave as a 1D numpy
> > array. What about array[:, :, None]. Does it broadcast? (This can
> > develop into quite a rabbit hole unless defined very clearly from the
> > start I suspect.)
>
> Indexing is probably the trickiest part to get right.  A first thought
> would be to limit any interactions that produce >2D, and to introduce 1d
> and 0d.  I'd have to double check, but it may be possible to do shape
> calculations, follow that by relying on existing operations to produce a
> result, and then adjust that result to conform to the expected shape.
>
> > 4. The interaction with scipy.sparse.linalg.
>
> We'd need some way to let `scipy.sparse.linalg` know what's going on.  One
> way would be to have `linalg` call `as_sparse_matrix` or `as_sparse_array`
> explicitly, depending on its needs.
>
> Stéfan
> _______________________________________________
> SciPy-Dev mailing list
> SciPy-Dev at python.org
> https://mail.python.org/mailman/listinfo/scipy-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mail.python.org/pipermail/scipy-dev/attachments/20210604/01469b64/attachment-0001.html>