Great! Ralf Gommers wrote:
Hi, I'm the author of fast_matrix_market, a package that reads and writes Matrix Market files: https://github.com/alugowski/fast_matrix_market/tree/main/python The Python binding is API compatible with scipy.io.mmio, but about 25x faster. A few other handy improvements include 64-bit indices (only if necessary), direct csc/csr writes with no coo intermediary, longdouble. It passes the SciPy mmio test suite. Would there be any interest in somehow integrating this library into SciPy? The speed difference really does make a big difference for large files. The package is written in C++17 and pybind11. Threading is with a simple thread pool based on c++11 threads, but can be changed. A significant part of the speed increase is parsing with std::from_chars instead of strto*, which alone contributes a massive improvement. Thanks for this package and proposal Adam! I had a quick look at your
On Wed, Apr 12, 2023 at 9:57 AM alugowski@gmail.com wrote: package, and it looks good to me. This kind of upstreaming of code, when there's a clear performance benefit and the code is maintained, seems nice to me. Just to make sure: are you planning to continue maintaining this code? Either within SciPy only, or also as a separate package and keeping the two in sync?
Yes, I'm seeing increasing interest in the package so I'm going to keep maintaining it. I do have a lot of use of the C++ bindings, so it makes sense to keep it as a separate package in addition to within SciPy.
The C++17 should be fine as far as I can tell - our http://scipy.github.io/devdocs/dev/toolchain.html has an upgrade to C++17 marked for 2022 already, so as long as things build with what MSVC provides (the usual limiting factor), that should be good. Cheers, Ralf
I've gone through the MSVC pains already to build Windows wheels with cibuildwheel, so that shouldn't be an issue. -Adam