Re: [SciPy-Dev] Boost for stats
Hello, I’ve been working on a spreadsheet that compares Boost and SciPy. I looked at statistical distributions, special functions, and ODE solvers. Here’s the google sheets link: https://docs.google.com/spreadsheets/d/1zVaau6k1_0yQNW107D81RVCirWEN8sXwcYaW... I’ve left it on suggestion mode with that sharing link, so if anyone has any thoughts please feel free to leave a comment. It looks like Boost may have a lot to add! Regards, Sam
On Feb 15, 2021, at 4:48 AM, scipy-dev-request@python.org wrote:
Send SciPy-Dev mailing list submissions to scipy-dev@python.org
To subscribe or unsubscribe via the World Wide Web, visit https://mail.python.org/mailman/listinfo/scipy-dev or, via email, send a message with subject or body 'help' to scipy-dev-request@python.org
You can reach the person managing the list at scipy-dev-owner@python.org
When replying, please edit your Subject line so it is more specific than "Re: Contents of SciPy-Dev digest..."
Today's Topics:
1. Re: Boost for stats (Neal Becker) 2. Re: Boost for stats (Hans Dembinski) 3. Re: Boost for stats (Ralf Gommers) 4. Re: Boost for stats (Neal Becker)
----------------------------------------------------------------------
Message: 1 Date: Mon, 15 Feb 2021 07:23:38 -0500 From: Neal Becker <ndbecker2@gmail.com> To: SciPy Developers List <scipy-dev@python.org> Subject: Re: [SciPy-Dev] Boost for stats Message-ID: <CAG3t+pHTnLa+EHL5G=_Esvi1unvYO0+DNnv8RxGKryuTS+jBUg@mail.gmail.com> Content-Type: text/plain; charset="UTF-8"
I have been using (and it's predecessor before it, boost::python) to package c++ code for python use for many years, including some of boost libraries. pybind11 is easy to use and is much better than e.g., cython for packaging c++ code. pybind11 is also header-only.
I would also like to call attention for anyone interested in scientific software and c++ to a wonderful library (header-only), xtensor https://xtensor.readthedocs.io/en/latest/
On Mon, Feb 15, 2021 at 7:15 AM Hans Dembinski <hans.dembinski@gmail.com> wrote:
On 15. Feb 2021, at 08:26, Andrew Nelson <andyfaff@gmail.com> wrote:
My questions would be:
- how portable is the boost code in general?
It is very portable. The core goal of Boost is to offer implementations with quality and portability on par with the C++ standard library implementations. Non-portable extensions are sometimes used to speed up things, but there is always a standard compliant vanilla version. In practice, maintainers test portability with CI on Windows, OSX, Linux, using various versions of gcc, clang, msvc, intel, see e.g.
https://github.com/boostorg/math/blob/develop/.github/workflows/ci.yml
and the Boost build farm from the days before free CI for OSS was easily available,
https://www.boost.org/development/tests/master/developer/move.html
Not all compilers/platforms are fully compliant, of course. Boost uses workarounds to combat that and submits bug reports on the compiler bug trackers.
- how easy is it to install the library.
As Nicholas mentioned, Boost.Math (and Boost.Histogram) is header-only, so it is sufficient to include the headers.
Best regards, Hans _______________________________________________ SciPy-Dev mailing list SciPy-Dev@python.org https://mail.python.org/mailman/listinfo/scipy-dev
-- Those who don't understand recursion are doomed to repeat it
------------------------------
Message: 2 Date: Mon, 15 Feb 2021 13:35:25 +0100 From: Hans Dembinski <hans.dembinski@gmail.com> To: SciPy Developers List <scipy-dev@python.org> Subject: Re: [SciPy-Dev] Boost for stats Message-ID: <13411649-82A4-4EC1-A58C-FAA3DDFF11D1@gmail.com> Content-Type: text/plain; charset=us-ascii
On 15. Feb 2021, at 01:47, Warren Weckesser <warren.weckesser@gmail.com> wrote:
* The Boost histogram library might provide some benefits over the existing NumPy and SciPy options. (Hans Dembinski, the author of the histrogram library, has already commented in this email thread.)
I would happily support this. We currently offer a Python front-end to Boost.Histogram https://github.com/scikit-hep/boost-histogram which includes a numpy.histogram compatible interface.
Switching to Boost.Histogram may offer performance benefits, see https://boost-histogram.readthedocs.io/en/latest/notebooks/PerformanceCompar...
Compared to np.histogram we saw a 1.7 times increase - single threaded, more if multiple threads are used. Compared to np.histogram2d we saw a 11 times increase. These numbers should probably be checked more carefully before decisions are made.
Boost.Histogram offers generalised histograms with arbitrary accumulators per cell, so it could also replace the implementations of https://docs.scipy.org/doc/scipy/reference/generated/scipy.stats.binned_stat... and friends.
Best regards, Hans
------------------------------
Message: 3 Date: Mon, 15 Feb 2021 13:41:51 +0100 From: Ralf Gommers <ralf.gommers@gmail.com> To: SciPy Developers List <scipy-dev@python.org> Subject: Re: [SciPy-Dev] Boost for stats Message-ID: <CABL7CQjYZh0CyA6Kx5FULw2KaYMmdrLbm0Jecztc5+4z+r8OJg@mail.gmail.com> Content-Type: text/plain; charset="utf-8"
On Mon, Feb 15, 2021 at 1:35 PM Hans Dembinski <hans.dembinski@gmail.com> wrote:
On 15. Feb 2021, at 01:47, Warren Weckesser <warren.weckesser@gmail.com> wrote:
* The Boost histogram library might provide some benefits over the existing NumPy and SciPy options. (Hans Dembinski, the author of the histrogram library, has already commented in this email thread.)
I would happily support this. We currently offer a Python front-end to Boost.Histogram https://github.com/scikit-hep/boost-histogram which includes a numpy.histogram compatible interface.
Switching to Boost.Histogram may offer performance benefits, see
https://boost-histogram.readthedocs.io/en/latest/notebooks/PerformanceCompar...
Compared to np.histogram we saw a 1.7 times increase - single threaded, more if multiple threads are used. Compared to np.histogram2d we saw a 11 times increase. These numbers should probably be checked more carefully before decisions are made.
Boost.Histogram offers generalised histograms with arbitrary accumulators per cell, so it could also replace the implementations of https://docs.scipy.org/doc/scipy/reference/generated/scipy.stats.binned_stat... and friends.
That would be really nice. binned_statistic is currently pure Python, and can be a performance hotspot (I've seen multiple cases of that in dealing with image and geospatial data).
Cheers, Ralf
Hi all, Responding to some comments I've seen fly by on this thread in no particular order:
However, playing devil's advocate somewhat: - does the scipy PR need the whole Boost.Math? If it only needs a select subset (e.g., do we need root-finding etc?), then maybe the size can be reduced.
As Hans mentioned, the Boost.Math depends on the whole of Boost, so not without a lot of pain of detangling code and losing the ability to easily bring in upstream updates.
- do we need the whole thing? e.g. ufunc loops only need a select subset of types.
- if we do go this route of taking parts / applying scipy specific
Virtually all Boost functions (and certainly the ones we're dealing with in the stats distributions) are templated. The ufunc generators I've written specialize the templates to create all the types we need for the ufuncs (single, double, and long double precision, specifically). float16 could be done in principle by unpacking to floats in the ufunc loop function, but no other distribution considers float16 so I didn't either. patches, what is easier to do or better maintenance-wise: vendor original code + patches, or do the work once by porting relevant parts to standalone C or C++ subset? It sounds like the prefered option (taking the discussion here and in the PR) is to include Boost as a submodule (which precludes SciPy specific patches, incidentally) and track specific tagged commits or commits with bug fixes as necessary. The problem with porting to C is that we lose the typing extensibility and easy upstream pulling of upstream bug fixes. Existing C ports of Boost functions could (should?) be moved to use Boost-proper to reduce maintenance burden.
pybind11
The only reason I didn't consider pybind11 is because I've never used it before and could get it done with Cython. The only troublesome C++ features I ran into were non-type template parameters, but there are easy workarounds for this. If anyone would like to patch my PR to use pybind11, please do!
Probably good to make sure that aarch64 build times remain relatively stable
Good point! Can this be checked via a PR to scipy-wheels? Best, Nicholas On Wed, Feb 17, 2021 at 7:09 PM Sam Wallan <samwallan@icloud.com> wrote:
Hello,
I’ve been working on a spreadsheet that compares Boost and SciPy. I looked at statistical distributions, special functions, and ODE solvers. Here’s the google sheets link:
https://docs.google.com/spreadsheets/d/1zVaau6k1_0yQNW107D81RVCirWEN8sXwcYaW...
I’ve left it on suggestion mode with that sharing link, so if anyone has any thoughts please feel free to leave a comment. It looks like Boost may have a lot to add!
Regards,
Sam
On Feb 15, 2021, at 4:48 AM, scipy-dev-request@python.org wrote:
Send SciPy-Dev mailing list submissions to scipy-dev@python.org
To subscribe or unsubscribe via the World Wide Web, visit https://mail.python.org/mailman/listinfo/scipy-dev or, via email, send a message with subject or body 'help' to scipy-dev-request@python.org
You can reach the person managing the list at scipy-dev-owner@python.org
When replying, please edit your Subject line so it is more specific than "Re: Contents of SciPy-Dev digest..."
Today's Topics:
1. Re: Boost for stats (Neal Becker) 2. Re: Boost for stats (Hans Dembinski) 3. Re: Boost for stats (Ralf Gommers) 4. Re: Boost for stats (Neal Becker)
----------------------------------------------------------------------
Message: 1 Date: Mon, 15 Feb 2021 07:23:38 -0500 From: Neal Becker <ndbecker2@gmail.com> To: SciPy Developers List <scipy-dev@python.org> Subject: Re: [SciPy-Dev] Boost for stats Message-ID: <CAG3t+pHTnLa+EHL5G=_ Esvi1unvYO0+DNnv8RxGKryuTS+jBUg@mail.gmail.com> Content-Type: text/plain; charset="UTF-8"
I have been using (and it's predecessor before it, boost::python) to package c++ code for python use for many years, including some of boost libraries. pybind11 is easy to use and is much better than e.g., cython for packaging c++ code. pybind11 is also header-only.
I would also like to call attention for anyone interested in scientific software and c++ to a wonderful library (header-only), xtensor https://xtensor.readthedocs.io/en/latest/
On Mon, Feb 15, 2021 at 7:15 AM Hans Dembinski <hans.dembinski@gmail.com> wrote:
On 15. Feb 2021, at 08:26, Andrew Nelson <andyfaff@gmail.com> wrote:
My questions would be:
- how portable is the boost code in general?
It is very portable. The core goal of Boost is to offer implementations
with quality and portability on par with the C++ standard library implementations. Non-portable extensions are sometimes used to speed up things, but there is always a standard compliant vanilla version. In practice, maintainers test portability with CI on Windows, OSX, Linux, using various versions of gcc, clang, msvc, intel, see e.g.
https://github.com/boostorg/math/blob/develop/.github/workflows/ci.yml
and the Boost build farm from the days before free CI for OSS was
easily available,
https://www.boost.org/development/tests/master/developer/move.html
Not all compilers/platforms are fully compliant, of course. Boost uses
workarounds to combat that and submits bug reports on the compiler bug trackers.
- how easy is it to install the library.
As Nicholas mentioned, Boost.Math (and Boost.Histogram) is header-only,
so it is sufficient to include the headers.
Best regards, Hans _______________________________________________ SciPy-Dev mailing list SciPy-Dev@python.org https://mail.python.org/mailman/listinfo/scipy-dev
-- Those who don't understand recursion are doomed to repeat it
------------------------------
Message: 2 Date: Mon, 15 Feb 2021 13:35:25 +0100 From: Hans Dembinski <hans.dembinski@gmail.com> To: SciPy Developers List <scipy-dev@python.org> Subject: Re: [SciPy-Dev] Boost for stats Message-ID: <13411649-82A4-4EC1-A58C-FAA3DDFF11D1@gmail.com> Content-Type: text/plain; charset=us-ascii
On 15. Feb 2021, at 01:47, Warren Weckesser <warren.weckesser@gmail.com> wrote:
* The Boost histogram library might provide some benefits over the existing NumPy and SciPy options. (Hans Dembinski, the author of the histrogram library, has already commented in this email thread.)
I would happily support this. We currently offer a Python front-end to Boost.Histogram https://github.com/scikit-hep/boost-histogram which includes a numpy.histogram compatible interface.
Switching to Boost.Histogram may offer performance benefits, see
https://boost-histogram.readthedocs.io/en/latest/notebooks/PerformanceCompar...
Compared to np.histogram we saw a 1.7 times increase - single threaded,
more if multiple threads are used. Compared to np.histogram2d we saw a 11 times increase. These numbers should probably be checked more carefully before decisions are made.
Boost.Histogram offers generalised histograms with arbitrary
accumulators per cell, so it could also replace the implementations of https://docs.scipy.org/doc/scipy/reference/generated/scipy.stats.binned_stat... and friends.
Best regards, Hans
------------------------------
Message: 3 Date: Mon, 15 Feb 2021 13:41:51 +0100 From: Ralf Gommers <ralf.gommers@gmail.com> To: SciPy Developers List <scipy-dev@python.org> Subject: Re: [SciPy-Dev] Boost for stats Message-ID: <
CABL7CQjYZh0CyA6Kx5FULw2KaYMmdrLbm0Jecztc5+4z+r8OJg@mail.gmail.com>
Content-Type: text/plain; charset="utf-8"
On Mon, Feb 15, 2021 at 1:35 PM Hans Dembinski <hans.dembinski@gmail.com
wrote:
On 15. Feb 2021, at 01:47, Warren Weckesser <
warren.weckesser@gmail.com>
wrote:
* The Boost histogram library might provide some benefits over the existing NumPy and SciPy options. (Hans Dembinski, the author of the histrogram library, has already commented in this email thread.)
I would happily support this. We currently offer a Python front-end to Boost.Histogram https://github.com/scikit-hep/boost-histogram which includes a numpy.histogram compatible interface.
Switching to Boost.Histogram may offer performance benefits, see
https://boost-histogram.readthedocs.io/en/latest/notebooks/PerformanceCompar...
Compared to np.histogram we saw a 1.7 times increase - single threaded, more if multiple threads are used. Compared to np.histogram2d we saw a
11
times increase. These numbers should probably be checked more carefully before decisions are made.
Boost.Histogram offers generalised histograms with arbitrary accumulators per cell, so it could also replace the implementations of
https://docs.scipy.org/doc/scipy/reference/generated/scipy.stats.binned_stat...
and friends.
That would be really nice. binned_statistic is currently pure Python, and can be a performance hotspot (I've seen multiple cases of that in dealing with image and geospatial data).
Cheers, Ralf
On Thu, Feb 18, 2021 at 3:45 AM Nicholas McKibben <nicholas.bgp@gmail.com> wrote:
Probably good to make sure that aarch64 build times remain relatively stable
Good point! Can this be checked via a PR to scipy-wheels?
Don't worry about this one, if compile time increase on other platforms is minor, it'll be fine for aarch64 too. We have limited TravisCI credits (actual status of that is a little unclear), so no need to burn them for this. We anyway should be moving CI providers for aarch64 at some point, probably to https://www.drone.io/ Cheers, Ralf
Local testing of inclusion of Boost as a submodule has revealed some undesirable side effects: - all sources, documentation, etc. regardless of relevance to SciPy must be fetched - recursive submodule initialization can take quite a while (~10 minutes on my machine and internet connection) - lots of churn when running commands like `git status` Of course we will also need to see how this impacts the CI pipelines. This extra overhead may initially cause some timeouts. Another option that will alleviate some of these pains is to create a header only repo similar to this one: https://github.com/povilasb/boost-header-only. It could live in the SciPy github account and would be easy to update -- simply download the Boost tarball release and copy over the include directory only (or build a specific commit locally and do the same thing). It is more maintenance than simply checking out the latest tagged release of Boost and updating the submodules (adds an extra step of updating the header only repo), but it minimizes space and bandwidth usage. Thoughts? On Thu, Feb 18, 2021 at 3:01 AM Ralf Gommers <ralf.gommers@gmail.com> wrote:
On Thu, Feb 18, 2021 at 3:45 AM Nicholas McKibben <nicholas.bgp@gmail.com> wrote:
Probably good to make sure that aarch64 build times remain relatively stable
Good point! Can this be checked via a PR to scipy-wheels?
Don't worry about this one, if compile time increase on other platforms is minor, it'll be fine for aarch64 too. We have limited TravisCI credits (actual status of that is a little unclear), so no need to burn them for this. We anyway should be moving CI providers for aarch64 at some point, probably to https://www.drone.io/
Cheers, Ralf
_______________________________________________ SciPy-Dev mailing list SciPy-Dev@python.org https://mail.python.org/mailman/listinfo/scipy-dev
The header-only repository sounds like the better option to me. That level of git churn for the submodule would be a noticeable burden.
On Mon, Feb 22, 2021 at 5:36 AM Nicholas McKibben <nicholas.bgp@gmail.com> wrote:
Local testing of inclusion of Boost as a submodule has revealed some undesirable side effects: - all sources, documentation, etc. regardless of relevance to SciPy must be fetched - recursive submodule initialization can take quite a while (~10 minutes on my machine and internet connection) - lots of churn when running commands like `git status`
Ouch, that's a lot slower than I expected. I'm not sure I understand it though, there should be no `git status` churn at all (unless the build process messes with files in-place?) and it's faster than cloning our own repo: $ time git clone git@github.com:boostorg/boost.git Cloning into 'boost'... remote: Enumerating objects: 15, done. remote: Counting objects: 100% (15/15), done. remote: Compressing objects: 100% (11/11), done. remote: Total 254626 (delta 8), reused 11 (delta 4), pack-reused 254611 Receiving objects: 100% (254626/254626), 62.02 MiB | 7.47 MiB/s, done. Resolving deltas: 100% (163071/163071), done. real 0m12.221s user 0m5.959s sys 0m2.725s $ time git clone git@github.com:scipy/scipy.git Cloning into 'scipy'... remote: Enumerating objects: 178585, done. remote: Total 178585 (delta 0), reused 0 (delta 0), pack-reused 178585 Receiving objects: 100% (178585/178585), 104.61 MiB | 6.56 MiB/s, done. Resolving deltas: 100% (137836/137836), done. real 0m21.492s user 0m9.620s sys 0m3.231s What should I test to reproduce the problem? Cheers, Ralf
Of course we will also need to see how this impacts the CI pipelines. This extra overhead may initially cause some timeouts. Another option that will alleviate some of these pains is to create a header only repo similar to this one: https://github.com/povilasb/boost-header-only. It could live in the SciPy github account and would be easy to update -- simply download the Boost tarball release and copy over the include directory only (or build a specific commit locally and do the same thing). It is more maintenance than simply checking out the latest tagged release of Boost and updating the submodules (adds an extra step of updating the header only repo), but it minimizes space and bandwidth usage. Thoughts?
On Thu, Feb 18, 2021 at 3:01 AM Ralf Gommers <ralf.gommers@gmail.com> wrote:
On Thu, Feb 18, 2021 at 3:45 AM Nicholas McKibben <nicholas.bgp@gmail.com> wrote:
Probably good to make sure that aarch64 build times remain relatively stable
Good point! Can this be checked via a PR to scipy-wheels?
Don't worry about this one, if compile time increase on other platforms is minor, it'll be fine for aarch64 too. We have limited TravisCI credits (actual status of that is a little unclear), so no need to burn them for this. We anyway should be moving CI providers for aarch64 at some point, probably to https://www.drone.io/
Cheers, Ralf
_______________________________________________ SciPy-Dev mailing list SciPy-Dev@python.org https://mail.python.org/mailman/listinfo/scipy-dev
_______________________________________________ SciPy-Dev mailing list SciPy-Dev@python.org https://mail.python.org/mailman/listinfo/scipy-dev
Hi,
On 22. Feb 2021, at 10:35, Ralf Gommers <ralf.gommers@gmail.com> wrote:
Ouch, that's a lot slower than I expected. I'm not sure I understand it though, there should be no `git status` churn at all (unless the build process messes with files in-place?) and it's faster than cloning our own repo:
$ time git clone git@github.com:boostorg/boost.git Cloning into 'boost'... remote: Enumerating objects: 15, done. remote: Counting objects: 100% (15/15), done. remote: Compressing objects: 100% (11/11), done. remote: Total 254626 (delta 8), reused 11 (delta 4), pack-reused 254611 Receiving objects: 100% (254626/254626), 62.02 MiB | 7.47 MiB/s, done. Resolving deltas: 100% (163071/163071), done.
real 0m12.221s user 0m5.959s sys 0m2.725s
because of the a long-term goal to make boost more modular, cloning boostorg/boost like this only clones the so called superproject of Boost, which indeed very small. It consists itself of many submodules with the individual Boost libraries, like Boost.Math etc, which live in separate repositories. If you do git clone --recurse-submodules git@github.com:boostorg/boost.git instead, you will see the long delay. Fetching all the submodules indeed takes a lot of time, unfortunately. The main Boost repo includes 157 submodules. Best regards, Hans
Adding the following options to the .gitmodules file was also useful for speeding up routine git commands: active = false ignore = true shallow = true I am not sure if all of them are necessary - I don't profess to be a git wizard. I still had commands such as 'git add -u' hang. Indeed --recurse-submodules is necessary (and might be why the CI is currently failing for the the PR). Thanks, Nicholas On Mon, Feb 22, 2021, 04:23 Hans Dembinski <hans.dembinski@gmail.com> wrote:
Hi,
On 22. Feb 2021, at 10:35, Ralf Gommers <ralf.gommers@gmail.com> wrote:
Ouch, that's a lot slower than I expected. I'm not sure I understand it though, there should be no `git status` churn at all (unless the build process messes with files in-place?) and it's faster than cloning our own repo:
$ time git clone git@github.com:boostorg/boost.git Cloning into 'boost'... remote: Enumerating objects: 15, done. remote: Counting objects: 100% (15/15), done. remote: Compressing objects: 100% (11/11), done. remote: Total 254626 (delta 8), reused 11 (delta 4), pack-reused 254611 Receiving objects: 100% (254626/254626), 62.02 MiB | 7.47 MiB/s, done. Resolving deltas: 100% (163071/163071), done.
real 0m12.221s user 0m5.959s sys 0m2.725s
because of the a long-term goal to make boost more modular, cloning boostorg/boost like this only clones the so called superproject of Boost, which indeed very small. It consists itself of many submodules with the individual Boost libraries, like Boost.Math etc, which live in separate repositories. If you do
git clone --recurse-submodules git@github.com:boostorg/boost.git
instead, you will see the long delay. Fetching all the submodules indeed takes a lot of time, unfortunately. The main Boost repo includes 157 submodules.
Best regards, Hans _______________________________________________ SciPy-Dev mailing list SciPy-Dev@python.org https://mail.python.org/mailman/listinfo/scipy-dev
participants (5)
-
Hans Dembinski -
Matthew Brett -
Nicholas McKibben -
Ralf Gommers -
Sam Wallan