PyRSB: Python interface to librsb sparse matrices library
Hi.
I'm the author of the high performance multithreaded sparse matrix library `librsb' (mostly C, LGPLv3): http://librsb.sourceforge.net/
I'm *not* a user of SciPy/NumPy/Python, but using Cython I have written a proofofconcept interface to librsb, named `PyRSB': https://github.com/michelemartone/pyrsb
PyRSB is in a prototypal state; e.g. still lacks good error handling. Its interface is trivial, as it mimicks that of SciPy's 'csr_matrix'. Advantages over csr_matrix are in fast multithreaded multiplication of huge sparse matrices. Intended application area is iterative solution of linear systems; particularly fast if with symmetric matrices and many rhs.
With this email I am looking for prospective:  users/testers  developers (any interest to collaborate/adopt/include the project?)
Looking forward for your feedback, Michele
Hi Michele,
This is really interesting. I am a coauthor of the xtensor project and one thing that could be interesting is to wrap the various sparse matrix data structures in the form of xtensor expressions. A byproduct of doing so is that it would simplify creating bindings for multiple scientific computing languages (Python, Julia, R, and more coming). You can see the blog post http://quantstack.net/c++/2017/05/30/polyglotscientificcomputingwith xtensor.html for reference...
Also, one quick question: is the LGPL license a deliberate choice or is it not important to you? Most projects in the Python scientific stack are BSD licensed. So the LGPL choice makes it unlikely that a higherlevel project adopts it as a dependency. If you are the only copyright holder, you would still have the possibility to license it under a more permissive license such as BSD or MIT...
Congratulations on the release!
Sylvain
On Wed, Jun 21, 2017 at 11:51 AM, Michele Martone <michelemartone@users. sourceforge.net> wrote:
Hi.
I'm the author of the high performance multithreaded sparse matrix library `librsb' (mostly C, LGPLv3): http://librsb.sourceforge.net/
I'm *not* a user of SciPy/NumPy/Python, but using Cython I have written a proofofconcept interface to librsb, named `PyRSB': https://github.com/michelemartone/pyrsb
PyRSB is in a prototypal state; e.g. still lacks good error handling. Its interface is trivial, as it mimicks that of SciPy's 'csr_matrix'. Advantages over csr_matrix are in fast multithreaded multiplication of huge sparse matrices. Intended application area is iterative solution of linear systems; particularly fast if with symmetric matrices and many rhs.
With this email I am looking for prospective:
 users/testers
 developers (any interest to collaborate/adopt/include the project?)
Looking forward for your feedback, Michele
SciPyDev mailing list SciPyDev@python.org https://mail.python.org/mailman/listinfo/scipydev
On Jun 24, 2017 7:29 AM, "Sylvain Corlay" sylvain.corlay@gmail.com wrote:
Also, one quick question: is the LGPL license a deliberate choice or is it not important to you? Most projects in the Python scientific stack are BSD licensed. So the LGPL choice makes it unlikely that a higherlevel project adopts it as a dependency. If you are the only copyright holder, you would still have the possibility to license it under a more permissive license such as BSD or MIT...
Why would LGPL be a problem in a dependency? That doesn't stop you making your code BSD, and it's less restrictive licensewise than depending on MKL or the windows C runtime...
n
On Sat, Jun 24, 2017 at 3:16 PM, Nathaniel Smith njs@pobox.com wrote:
On Jun 24, 2017 7:29 AM, "Sylvain Corlay" sylvain.corlay@gmail.com wrote:
Also, one quick question: is the LGPL license a deliberate choice or is it not important to you? Most projects in the Python scientific stack are BSD licensed. So the LGPL choice makes it unlikely that a higherlevel project adopts it as a dependency. If you are the only copyright holder, you would still have the possibility to license it under a more permissive license such as BSD or MIT...
Why would LGPL be a problem in a dependency? That doesn't stop you making your code BSD, and it's less restrictive licensewise than depending on MKL or the windows C runtime...
Is scipy still including any LGPL code, I thought not. There might still be some optional dependencies that not many users are using by default. ? Julia packages are mostly MIT, AFAIK. (except for the GPL parts because of cholmod, which we (?) avoid)
Josef
n
NumPyDiscussion mailing list NumPyDiscussion@python.org https://mail.python.org/mailman/listinfo/numpydiscussion
On Sat, 20170624 at 15:47 0400, josef.pktd@gmail.com wrote:
On Sat, Jun 24, 2017 at 3:16 PM, Nathaniel Smith njs@pobox.com wrote:
On Jun 24, 2017 7:29 AM, "Sylvain Corlay" <sylvain.corlay@gmail.com
wrote:
Also, one quick question: is the LGPL license a deliberate choice or is it not important to you? Most projects in the Python scientific stack are BSD licensed. So the LGPL choice makes it unlikely that a higherlevel project adopts it as a dependency. If you are the only copyright holder, you would still have the possibility to license it under a more permissive license such as BSD or MIT...
Why would LGPL be a problem in a dependency? That doesn't stop you making your code BSD, and it's less restrictive licensewise than depending on MKL or the windows C runtime...
Is scipy still including any LGPL code, I thought not. There might still be some optional dependencies that not many users are using by default. ? Julia packages are mostly MIT, AFAIK. (except for the GPL parts because of cholmod, which we (?) avoid)
Well, I don't think scipy has many dependencies (but I would not be surprised if those are LGPL). Not a specialist, but as a dependency it should be fine (that is the point of the L in LGPL after all as far as I understand, it is much less viral). If you package it with your own stuff, you have to make sure to point out that parts are LGPL of course (just like there is a reason you get the GPL printed out with some devices) and if you modify it provide these modifications, etc.
Of course you cannot include it into the scipy codebase itself, but there is probably no aim of doing so here, so without a specific reason I would think that LGPL is a great license.
 Sebastian
Josef
n
NumPyDiscussion mailing list NumPyDiscussion@python.org https://mail.python.org/mailman/listinfo/numpydiscussion
NumPyDiscussion mailing list NumPyDiscussion@python.org https://mail.python.org/mailman/listinfo/numpydiscussion
Does this still apply: https://scipy.github.io/oldwiki/pages/License_Compatibility.html
Carl
20170624 22:07 GMT+02:00 Sebastian Berg sebastian@sipsolutions.net:
On Sat, 20170624 at 15:47 0400, josef.pktd@gmail.com wrote:
On Sat, Jun 24, 2017 at 3:16 PM, Nathaniel Smith njs@pobox.com wrote:
On Jun 24, 2017 7:29 AM, "Sylvain Corlay" <sylvain.corlay@gmail.com
wrote:
Also, one quick question: is the LGPL license a deliberate choice or is it not important to you? Most projects in the Python scientific stack are BSD licensed. So the LGPL choice makes it unlikely that a higherlevel project adopts it as a dependency. If you are the only copyright holder, you would still have the possibility to license it under a more permissive license such as BSD or MIT...
Why would LGPL be a problem in a dependency? That doesn't stop you making your code BSD, and it's less restrictive licensewise than depending on MKL or the windows C runtime...
Is scipy still including any LGPL code, I thought not. There might still be some optional dependencies that not many users are using by default. ? Julia packages are mostly MIT, AFAIK. (except for the GPL parts because of cholmod, which we (?) avoid)
Well, I don't think scipy has many dependencies (but I would not be surprised if those are LGPL). Not a specialist, but as a dependency it should be fine (that is the point of the L in LGPL after all as far as I understand, it is much less viral). If you package it with your own stuff, you have to make sure to point out that parts are LGPL of course (just like there is a reason you get the GPL printed out with some devices) and if you modify it provide these modifications, etc.
Of course you cannot include it into the scipy codebase itself, but there is probably no aim of doing so here, so without a specific reason I would think that LGPL is a great license.
 Sebastian
Josef
n
NumPyDiscussion mailing list NumPyDiscussion@python.org https://mail.python.org/mailman/listinfo/numpydiscussion
NumPyDiscussion mailing list NumPyDiscussion@python.org https://mail.python.org/mailman/listinfo/numpydiscussion
NumPyDiscussion mailing list NumPyDiscussion@python.org https://mail.python.org/mailman/listinfo/numpydiscussion
On Sat, 20170624 at 22:58 +0200, Carl Kleffner wrote:
Does this still apply: https://scipy.github.io/oldwiki/pages/License _Compatibility.html
Of course, but it talks about putting it into the code base of scipy not about being able to use the package in any way in a dependency (i.e. `import package`).
 Sebastian
Carl
20170624 22:07 GMT+02:00 Sebastian Berg <sebastian@sipsolutions.net
: On Sat, 20170624 at 15:47 0400, josef.pktd@gmail.com wrote:
On Sat, Jun 24, 2017 at 3:16 PM, Nathaniel Smith njs@pobox.com wrote:
On Jun 24, 2017 7:29 AM, "Sylvain Corlay" <sylvain.corlay@gmail
.com
wrote:
Also, one quick question: is the LGPL license a deliberate
choice
or is it not important to you? Most projects in the Python scientific stack are BSD licensed. So the LGPL choice makes it unlikely that a higherlevel project adopts it as a dependency.
If
you are the only copyright holder, you would still have the possibility to license it under a more permissive license such
as
BSD or MIT...
Why would LGPL be a problem in a dependency? That doesn't stop
you
making your code BSD, and it's less restrictive licensewise
than
depending on MKL or the windows C runtime...
Is scipy still including any LGPL code, I thought not. There might still be some optional dependencies that not many
users
are using by default. ? Julia packages are mostly MIT, AFAIK. (except for the GPL parts because of cholmod, which we (?) avoid)
Well, I don't think scipy has many dependencies (but I would not be surprised if those are LGPL). Not a specialist, but as a dependency it should be fine (that is the point of the L in LGPL after all as far as I understand, it is much less viral). If you package it with your own stuff, you have to make sure to point out that parts are LGPL of course (just like there is a reason you get the GPL printed out with some devices) and if you modify it provide these modifications, etc.
Of course you cannot include it into the scipy codebase itself, but there is probably no aim of doing so here, so without a specific reason I would think that LGPL is a great license.
 Sebastian
Josef
n
NumPyDiscussion mailing list NumPyDiscussion@python.org https://mail.python.org/mailman/listinfo/numpydiscussion
NumPyDiscussion mailing list NumPyDiscussion@python.org https://mail.python.org/mailman/listinfo/numpydiscussion
NumPyDiscussion mailing list NumPyDiscussion@python.org https://mail.python.org/mailman/listinfo/numpydiscussion
NumPyDiscussion mailing list NumPyDiscussion@python.org https://mail.python.org/mailman/listinfo/numpydiscussion
On Sat, Jun 24, 2017 at 5:26 PM, Sebastian Berg sebastian@sipsolutions.net wrote:
On Sat, 20170624 at 22:58 +0200, Carl Kleffner wrote:
Does this still apply: https://scipy.github.io/oldwiki/pages/License _Compatibility.html
Of course, but it talks about putting it into the code base of scipy not about being able to use the package in any way in a dependency (i.e. `import package`).
But scipy does bundle a lot of external packages and then the license is relevant. I have no idea if this would apply here, but I find Sylvain's question relevant if closer integration and usage within the scientific python, and maybe Julia, community is desired.
LGPL is not as bad as GPL but still adds another hurdle, as far as I know the scipy and related history.
Josef
 Sebastian
Carl
20170624 22:07 GMT+02:00 Sebastian Berg <sebastian@sipsolutions.net
: On Sat, 20170624 at 15:47 0400, josef.pktd@gmail.com wrote:
On Sat, Jun 24, 2017 at 3:16 PM, Nathaniel Smith njs@pobox.com wrote:
On Jun 24, 2017 7:29 AM, "Sylvain Corlay" <sylvain.corlay@gmail
.com
wrote:
Also, one quick question: is the LGPL license a deliberate
choice
or is it not important to you? Most projects in the Python scientific stack are BSD licensed. So the LGPL choice makes it unlikely that a higherlevel project adopts it as a dependency.
If
you are the only copyright holder, you would still have the possibility to license it under a more permissive license such
as
BSD or MIT...
Why would LGPL be a problem in a dependency? That doesn't stop
you
making your code BSD, and it's less restrictive licensewise
than
depending on MKL or the windows C runtime...
Is scipy still including any LGPL code, I thought not. There might still be some optional dependencies that not many
users
are using by default. ? Julia packages are mostly MIT, AFAIK. (except for the GPL parts because of cholmod, which we (?) avoid)
Well, I don't think scipy has many dependencies (but I would not be surprised if those are LGPL). Not a specialist, but as a dependency it should be fine (that is the point of the L in LGPL after all as far as I understand, it is much less viral). If you package it with your own stuff, you have to make sure to point out that parts are LGPL of course (just like there is a reason you get the GPL printed out with some devices) and if you modify it provide these modifications, etc.
Of course you cannot include it into the scipy codebase itself, but there is probably no aim of doing so here, so without a specific reason I would think that LGPL is a great license.
 Sebastian
Josef
n
NumPyDiscussion mailing list NumPyDiscussion@python.org https://mail.python.org/mailman/listinfo/numpydiscussion
NumPyDiscussion mailing list NumPyDiscussion@python.org https://mail.python.org/mailman/listinfo/numpydiscussion
NumPyDiscussion mailing list NumPyDiscussion@python.org https://mail.python.org/mailman/listinfo/numpydiscussion
NumPyDiscussion mailing list NumPyDiscussion@python.org https://mail.python.org/mailman/listinfo/numpydiscussion
NumPyDiscussion mailing list NumPyDiscussion@python.org https://mail.python.org/mailman/listinfo/numpydiscussion
On 20170624@16:29, Sylvain Corlay wrote:
Hi Michele,
Hi Sylvain,
This is really interesting. I am a coauthor of the xtensor project and one thing that could be interesting is to wrap the various sparse matrix data structures in the form of xtensor expressions. A byproduct of doing so is that it would simplify creating bindings for multiple scientific computing languages (Python, Julia, R, and more coming). You can see the blog post http://quantstack.net/c++/2017/05/30/polyglotscientificcomputingwith xtensor.html for reference...
This article exemplifies manipulation of numerical arrays. Now I ask you: Given an interactive language $L of the one you cite above, can xtensor provide objects with custom type and operators for manipulation in *that* language, like in e.g. the pyrsb case: a=rsb_matrix((4,4)) print(a+a) # + operator and 'print' interfacing ?
Also, one quick question: is the LGPL license a deliberate choice or is it not important to you? Most projects in the Python scientific stack are BSD licensed. So the LGPL choice makes it unlikely that a higherlevel project adopts it as a dependency. If you are the only copyright holder, you would still have the possibility to license it under a more permissive license such as BSD or MIT...
No important choice. No problems relicensing the PyRSB prototype to BSD or MIT.
Congratulations on the release!
Thanks for the interest and welcome constructive feedback :)
Sylvain
On Wed, Jun 21, 2017 at 11:51 AM, Michele Martone <michelemartone@users. sourceforge.net> wrote:
Hi.
I'm the author of the high performance multithreaded sparse matrix library `librsb' (mostly C, LGPLv3): http://librsb.sourceforge.net/
I'm *not* a user of SciPy/NumPy/Python, but using Cython I have written a proofofconcept interface to librsb, named `PyRSB': https://github.com/michelemartone/pyrsb
PyRSB is in a prototypal state; e.g. still lacks good error handling. Its interface is trivial, as it mimicks that of SciPy's 'csr_matrix'. Advantages over csr_matrix are in fast multithreaded multiplication of huge sparse matrices. Intended application area is iterative solution of linear systems; particularly fast if with symmetric matrices and many rhs.
With this email I am looking for prospective:
 users/testers
 developers (any interest to collaborate/adopt/include the project?)
Looking forward for your feedback, Michele
SciPyDev mailing list SciPyDev@python.org https://mail.python.org/mailman/listinfo/scipydev
SciPyDev mailing list SciPyDev@python.org https://mail.python.org/mailman/listinfo/scipydev
Hi Michele,
Regarding xtensor, the article focused on how we can wrap an existing data structure in an "xtensor expression", i.e. something that you can operate upon with the numpy style API in C++ (see the numpy to xtensor cheat sheet http://xtensor.readthedocs.io/en/latest/numpy.html). Since it is an expression system, you can do "a + b * sin(t)" with numpystyle broadcasting and it returns an unevaluated expression, which is computed only upon access or assignment to a container. "a", "b", and "c" could all be compound expressions, backed by filesystem operations, be an inmemory container, or a wrapper on your sparse matrices...
Now, we are working on automating the "*C++ tensor expressiontype > python wrapper with access and math operators" *generation of code, which would be what you are describing below...
On the subject of the LGPL, regarding the distinction with the GPL that were brought up earlier in the thread, it is indeed not as problematic as the GPL, but still less permissive as BSD and MIT. In general, I think that using a BSD or MIT license averts these concerns in the Python ecosystem, and is preferable *if you don't care about copyleft*...
Best,
Sylvain
On Sun, Jun 25, 2017 at 12:01 PM, Michele Martone < michelemartone@users.sourceforge.net> wrote:
On 20170624@16:29, Sylvain Corlay wrote:
Hi Michele,
Hi Sylvain,
This is really interesting. I am a coauthor of the xtensor project and
one
thing that could be interesting is to wrap the various sparse matrix data structures in the form of xtensor expressions. A byproduct of doing so is that it would simplify creating bindings for multiple scientific
computing
languages (Python, Julia, R, and more coming). You can see the blog post http://quantstack.net/c++/2017/05/30/polyglotscientificcomputingwith xtensor.html for reference...
This article exemplifies manipulation of numerical arrays. Now I ask you: Given an interactive language $L of the one you cite above, can xtensor provide objects with custom type and operators for manipulation in *that* language, like in e.g. the pyrsb case: a=rsb_matrix((4,4)) print(a+a) # + operator and 'print' interfacing ?
Also, one quick question: is the LGPL license a deliberate choice or is
it
not important to you? Most projects in the Python scientific stack are
BSD
licensed. So the LGPL choice makes it unlikely that a higherlevel
project
adopts it as a dependency. If you are the only copyright holder, you
would
still have the possibility to license it under a more permissive license such as BSD or MIT...
No important choice. No problems relicensing the PyRSB prototype to BSD or MIT.
Congratulations on the release!
Thanks for the interest and welcome constructive feedback :)
Sylvain
On Wed, Jun 21, 2017 at 11:51 AM, Michele Martone <michelemartone@users. sourceforge.net> wrote:
Hi.
I'm the author of the high performance multithreaded sparse matrix library `librsb' (mostly C, LGPLv3): http://librsb.sourceforge.net/
I'm *not* a user of SciPy/NumPy/Python, but using Cython I have written a proofofconcept interface to librsb, named `PyRSB': https://github.com/michelemartone/pyrsb
PyRSB is in a prototypal state; e.g. still lacks good error handling. Its interface is trivial, as it mimicks that of SciPy's 'csr_matrix'. Advantages over csr_matrix are in fast multithreaded multiplication of huge sparse matrices. Intended application area is iterative solution of linear systems; particularly fast if with symmetric matrices and many rhs.
With this email I am looking for prospective:
 users/testers
 developers (any interest to collaborate/adopt/include the project?)
Looking forward for your feedback, Michele
SciPyDev mailing list SciPyDev@python.org https://mail.python.org/mailman/listinfo/scipydev
SciPyDev mailing list SciPyDev@python.org https://mail.python.org/mailman/listinfo/scipydev
On 20170624@16:29, Sylvain Corlay wrote:
Hi Michele,
Hi Sylvain,
This is really interesting. I am a coauthor of the xtensor project and one thing that could be interesting is to wrap the various sparse matrix data structures in the form of xtensor expressions. A byproduct of doing so is that it would simplify creating bindings for multiple scientific computing languages (Python, Julia, R, and more coming). You can see the blog post http://quantstack.net/c++/2017/05/30/polyglotscientificcomputingwith xtensor.html for reference...
This article exemplifies manipulation of numerical arrays. Now I ask you: Given an interactive language $L of the one you cite above, can xtensor provide objects with custom type and operators for manipulation in *that* language, like in e.g. the pyrsb case: a=rsb_matrix((4,4)) print(a+a) # + operator and 'print' interfacing ?
Also, one quick question: is the LGPL license a deliberate choice or is it not important to you? Most projects in the Python scientific stack are BSD licensed. So the LGPL choice makes it unlikely that a higherlevel project adopts it as a dependency. If you are the only copyright holder, you would still have the possibility to license it under a more permissive license such as BSD or MIT...
No important choice. No problems relicensing the PyRSB prototype to BSD or MIT.
Congratulations on the release!
Thanks for the interest and welcome constructive feedback :)
Sylvain
On Wed, Jun 21, 2017 at 11:51 AM, Michele Martone <michelemartone@users. sourceforge.net> wrote:
Hi.
I'm the author of the high performance multithreaded sparse matrix library `librsb' (mostly C, LGPLv3): http://librsb.sourceforge.net/
I'm *not* a user of SciPy/NumPy/Python, but using Cython I have written a proofofconcept interface to librsb, named `PyRSB': https://github.com/michelemartone/pyrsb
PyRSB is in a prototypal state; e.g. still lacks good error handling. Its interface is trivial, as it mimicks that of SciPy's 'csr_matrix'. Advantages over csr_matrix are in fast multithreaded multiplication of huge sparse matrices. Intended application area is iterative solution of linear systems; particularly fast if with symmetric matrices and many rhs.
With this email I am looking for prospective:
 users/testers
 developers (any interest to collaborate/adopt/include the project?)
Looking forward for your feedback, Michele
SciPyDev mailing list SciPyDev@python.org https://mail.python.org/mailman/listinfo/scipydev
SciPyDev mailing list SciPyDev@python.org https://mail.python.org/mailman/listinfo/scipydev
participants (6)

Carl Kleffner

josef.pktd＠gmail.com

Michele Martone

Nathaniel Smith

Sebastian Berg

Sylvain Corlay