Hi, I'd like to consider replacing some of the function implementations in scipy.special with versions from the C++ Boost library, cf. http://www.boost.org/doc/libs/1_51_0/libs/math/doc/sf_and_dist/html/index.ht... The reason is that these implementations appear to be made with more care than what we use currently, and behave more correctly in corner cases (cf. e.g. ticket #1740). To minimize work, it would be useful just to use the Boost functions direcly, rather than doing manual C transcriptions. The drawback here is that the portion of Boost library required weights about 8 MB of source code, and we would most likely like to bundle it, as it is not really a standard part of many installations. This does not reflect much on the compiled binary size, however. I'm not 100 % certain about the compiler support. Perhaps C++ is already mature enough to work across the platforms we care about. I'm not aware of many good BSD-compatible floating-point special function libraries, so if you know others, or would be opposed to bundling Boost, please chime up! -- Pauli Virtanen
On Sun, Oct 7, 2012 at 9:40 AM, Pauli Virtanen <pav@iki.fi> wrote:
Hi,
I'd like to consider replacing some of the function implementations in scipy.special with versions from the C++ Boost library, cf.
http://www.boost.org/doc/libs/1_51_0/libs/math/doc/sf_and_dist/html/index.ht...
The reason is that these implementations appear to be made with more care than what we use currently, and behave more correctly in corner cases (cf. e.g. ticket #1740).
To minimize work, it would be useful just to use the Boost functions direcly, rather than doing manual C transcriptions. The drawback here is that the portion of Boost library required weights about 8 MB of source code, and we would most likely like to bundle it, as it is not really a standard part of many installations. This does not reflect much on the compiled binary size, however.
I'm not 100 % certain about the compiler support. Perhaps C++ is already mature enough to work across the platforms we care about.
I'm not aware of many good BSD-compatible floating-point special function libraries, so if you know others, or would be opposed to bundling Boost, please chime up!
I think using the boost library is a good idea. It is well tested and looks to support quad precision, something we will probably want at some point. It also looks to be highly templated and tightly integrated, so I suspect getting it properly interfaced might be non-trivial. The same holds for the distributions, but we have done much the same. It might be worth looking over the boost classes for some ideas. As to the size of the code, the current scipy/special library is ~40MB and I expect we can get rid of some of that. We should check for LLVM compatibility to make sure Apple isn't a problem, but it looks like most other C++ compilers will work, Boost does try hard for universality. Compile times will probably increase if we keep all the templates. Chuck
07.10.2012 19:21, Charles R Harris kirjoitti: [clip]
I think using the boost library is a good idea. It is well tested and looks to support quad precision, something we will probably want at some point. It also looks to be highly templated and tightly integrated, so I suspect getting it properly interfaced might be non-trivial. The same holds for the distributions, but we have done much the same. It might be worth looking over the boost classes for some ideas.
As to the size of the code, the current scipy/special library is ~40MB and I expect we can get rid of some of that. We should check for LLVM compatibility to make sure Apple isn't a problem, but it looks like most other C++ compilers will work, Boost does try hard for universality.
Compile times will probably increase if we keep all the templates.
Integrating it is actually not so hard, it's here if someone wants to try (e.g. if it works at all on OSX): https://github.com/pv/scipy-work/commits/special-boost and you get `scipy.special._ufuncs_cxx.jv`. -- Pauli Virtanen
On Sun, Oct 7, 2012 at 10:48 AM, Pauli Virtanen <pav@iki.fi> wrote:
07.10.2012 19:21, Charles R Harris kirjoitti: [clip]
I think using the boost library is a good idea. It is well tested and looks to support quad precision, something we will probably want at some point. It also looks to be highly templated and tightly integrated, so I suspect getting it properly interfaced might be non-trivial. The same holds for the distributions, but we have done much the same. It might be worth looking over the boost classes for some ideas.
As to the size of the code, the current scipy/special library is ~40MB and I expect we can get rid of some of that. We should check for LLVM compatibility to make sure Apple isn't a problem, but it looks like most other C++ compilers will work, Boost does try hard for universality.
Compile times will probably increase if we keep all the templates.
Integrating it is actually not so hard, it's here if someone wants to try (e.g. if it works at all on OSX):
https://github.com/pv/scipy-work/commits/special-boost
and you get `scipy.special._ufuncs_cxx.jv`.
A few questions. Are you using the C compatibility option for boost? How are you dealing with errors? Any support for complex numbers? I think it is a plus that Boost supports floats, doubles, and long doubles in the C compatibility mode. I don't know if we will also need complex for some of the functions. Chuck
07.10.2012 20:18, Charles R Harris kirjoitti: [clip]
A few questions.
Are you using the C compatibility option for boost?
No, just write a extern C wrapper function that calls it.
How are you dealing with errors?
I'm not --- I suppose exceptions could be caught, however.
Any support for complex numbers?
AFAIK, boost doesn't support complex-valued functions. Pauli
On Sun, Oct 7, 2012 at 12:04 PM, Pauli Virtanen <pav@iki.fi> wrote:
07.10.2012 20:18, Charles R Harris kirjoitti: [clip]
A few questions.
Are you using the C compatibility option for boost?
No, just write a extern C wrapper function that calls it.
How are you dealing with errors?
I'm not --- I suppose exceptions could be caught, however.
Any support for complex numbers?
AFAIK, boost doesn't support complex-valued functions.
One could always hope ;) Grepping shows complex returns for the inverse trig functions in boost/math/complex, I think that is for C++ standards compatibility. They aren't among the special_functions, however. The math library also include quaternions and octonions, are you planning on including the whole math library or just the special_functions? Just to be clear, I like this effort, we have already borrowed some tests from Boost, we might as well take the functions that go with them. Chuck
On Sun, Oct 7, 2012 at 7:04 PM, Pauli Virtanen <pav@iki.fi> wrote:
07.10.2012 20:18, Charles R Harris kirjoitti: [clip]
A few questions.
Are you using the C compatibility option for boost?
No, just write a extern C wrapper function that calls it.
How are you dealing with errors?
I'm not --- I suppose exceptions could be caught, however.
Is it possible to set up the boost mode so that ERRNO is used instead of exceptions ? I would rather be as close as possible to C semantics as possible, David
On Oct 7, 2012, at 12:48 PM, Pauli Virtanen <pav@iki.fi> wrote:
07.10.2012 19:21, Charles R Harris kirjoitti: [clip]
I think using the boost library is a good idea. It is well tested and looks to support quad precision, something we will probably want at some point. It also looks to be highly templated and tightly integrated, so I suspect getting it properly interfaced might be non-trivial. The same holds for the distributions, but we have done much the same. It might be worth looking over the boost classes for some ideas.
As to the size of the code, the current scipy/special library is ~40MB and I expect we can get rid of some of that. We should check for LLVM compatibility to make sure Apple isn't a problem, but it looks like most other C++ compilers will work, Boost does try hard for universality.
Compile times will probably increase if we keep all the templates.
Integrating it is actually not so hard, it's here if someone wants to try (e.g. if it works at all on OSX):
https://github.com/pv/scipy-work/commits/special-boost
and you get `scipy.special._ufuncs_cxx.jv`.
-- Pauli Virtanen
I was able to build it with clang 4.1 under OS X 10.8.2. I don't see any *_ufuncs_cxx.jv files, but I do see these _ufunc files in ./build ./build/lib.macosx-10.7-intel-2.7/scipy/special/_ufuncs.so ./build/lib.macosx-10.7-intel-2.7/scipy/special/_ufuncs_cxx.so ./build/lib.macosx-10.7-intel-2.7/scipy/special/generate_ufuncs.py ./build/temp.macosx-10.7-intel-2.7/scipy/special/_ufuncs.o ./build/temp.macosx-10.7-intel-2.7/scipy/special/_ufuncs_cxx.o I see scipy.special._ufuncs which is not in scipy-0.12.0.dev_941351d from github In [2]: print dir(scipy.special._ufuncs) ['__builtins__', '__doc__', '__file__', '__name__', '__package__', '__test__', '_eval_chebyt', '_lambertw', 'airy', 'airye', 'bdtr', 'bdtrc', 'bdtri', 'bdtrik', 'bdtrin', 'bei', 'beip', 'ber', 'berp', 'besselpoly', 'beta', 'betainc', 'betaincinv', 'betaln', 'btdtr', 'btdtri', 'btdtria', 'btdtrib', 'cbrt', 'chdtr', 'chdtrc', 'chdtri', 'chdtriv', 'chndtr', 'chndtridf', 'chndtrinc', 'chndtrix', 'cosdg', 'cosm1', 'cotdg', 'dawsn', 'ellipe', 'ellipeinc', 'ellipj', 'ellipkinc', 'ellipkm1', 'erf', 'erfc', 'errprint', 'exp1', 'exp10', 'exp2', 'expi', 'expit', 'expm1', 'expn', 'fdtr', 'fdtrc', 'fdtri', 'fdtridfd', 'fresnel', 'gamma', 'gammainc', 'gammaincc', 'gammainccinv', 'gammaincinv', 'gammaln', 'gdtr', 'gdtrc', 'gdtria', 'gdtrib', 'gdtrix', 'hankel1', 'hankel1e', 'hankel2', 'hankel2e', 'hyp1f1', 'hyp1f2', 'hyp2f0', 'hyp2f1', 'hyp3f0', 'hyperu', 'i0', 'i0e', 'i1', 'i1e', 'it2i0k0', 'it2j0y0', 'it2struve0', 'itairy', 'iti0k0', 'itj0y0', 'itmodstruve0', 'itstruve0', 'iv', 'ive', 'j0', 'j1', 'jn', 'jv', 'jve', 'k0', 'k0e', 'k1', 'k1e', 'kei', 'keip', 'kelvin', 'ker', 'kerp', 'kn', 'kolmogi', 'kolmogorov', 'kv', 'kve', 'log1p', 'log_ndtr', 'logit', 'lpmv', 'mathieu_a', 'mathieu_b', 'mathieu_cem', 'mathieu_modcem1', 'mathieu_modcem2', 'mathieu_modsem1', 'mathieu_modsem2', 'mathieu_sem', 'modfresnelm', 'modfresnelp', 'modstruve', 'nbdtr', 'nbdtrc', 'nbdtri', 'nbdtrik', 'nbdtrin', 'ncfdtr', 'ncfdtri', 'ncfdtridfd', 'ncfdtridfn', 'ncfdtrinc', 'nctdtr', 'nctdtridf', 'nctdtrinc', 'nctdtrit', 'ndtr', 'ndtri', 'nrdtrimn', 'nrdtrisd', 'obl_ang1', 'obl_ang1_cv', 'obl_cv', 'obl_rad1', 'obl_rad1_cv', 'obl_rad2', 'obl_rad2_cv', 'pbdv', 'pbvv', 'pbwa', 'pdtr', 'pdtrc', 'pdtri', 'pdtrik', 'pro_ang1', 'pro_ang1_cv', 'pro_cv', 'pro_rad1', 'pro_rad1_cv', 'pro_rad2', 'pro_rad2_cv', 'psi', 'radian', 'rgamma', 'round', 'shichi', 'sici', 'sindg', 'smirnov', 'smirnovi', 'spence', 'stdtr', 'stdtridf', 'stdtrit', 'struve', 'tandg', 'tklmbda', 'wofz', 'y0', 'y1', 'yn', 'yv', 'yve', 'zeta', 'zetac'] In [3]: scipy.special.test('full') Running unit tests for scipy.special NumPy version 1.8.0.dev-3f10c36 NumPy is installed in /Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/numpy-1.8.0.dev_3f10c36-py2.7-macosx-10.7-intel.egg/numpy SciPy version 0.12.0.dev-ccbfd79 SciPy is installed in /Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/scipy-0.12.0.dev_ccbfd79-py2.7-macosx-10.7-intel.egg/scipy Python version 2.7.3 (default, Jun 24 2012, 10:45:32) [GCC 4.2.1 Compatible Apple Clang 4.0 ((tags/Apple/clang-421.10.42))] nose version 1.1.2 ..................E...................................................K.K.............................................................................................................................................................................................................................................................................................................................................................................................K........K..............SSSSSSSS............................ ====================================================================== ERROR: test_iv_cephes_vs_amos_mass_test (test_basic.TestBessel) ---------------------------------------------------------------------- Traceback (most recent call last): File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/scipy-0.12.0.dev_ccbfd79-py2.7-macosx-10.7-intel.egg/scipy/special/tests/test_basic.py", line 1643, in test_iv_cephes_vs_amos_mass_test v[imsk] = v.astype(int) ValueError: NumPy boolean array indexing assignment cannot assign 1000000 input values to the 125035 output values where the mask is true ---------------------------------------------------------------------- Ran 514 tests in 1.993s FAILED (KNOWNFAIL=4, SKIP=8, errors=1) Out[3]: <nose.result.TextTestResult run=514 errors=1 failures=0> The same test fails in scipy-0.12.0.dev_941351d
On Sun, Oct 7, 2012 at 6:21 PM, Charles R Harris <charlesr.harris@gmail.com>wrote:
On Sun, Oct 7, 2012 at 9:40 AM, Pauli Virtanen <pav@iki.fi> wrote:
Hi,
I'd like to consider replacing some of the function implementations in scipy.special with versions from the C++ Boost library, cf.
http://www.boost.org/doc/libs/1_51_0/libs/math/doc/sf_and_dist/html/index.ht...
The reason is that these implementations appear to be made with more care than what we use currently, and behave more correctly in corner cases (cf. e.g. ticket #1740).
To minimize work, it would be useful just to use the Boost functions direcly, rather than doing manual C transcriptions. The drawback here is that the portion of Boost library required weights about 8 MB of source code, and we would most likely like to bundle it, as it is not really a standard part of many installations. This does not reflect much on the compiled binary size, however.
I'm not 100 % certain about the compiler support. Perhaps C++ is already mature enough to work across the platforms we care about.
An obvious worry is Windows. MinGW is explicitly not supported, and "may or may not work": http://www.boost.org/doc/libs/1_51_0/more/getting_started/windows.html
I'm not aware of many good BSD-compatible floating-point special function libraries, so if you know others, or would be opposed to bundling Boost, please chime up!
I think using the boost library is a good idea. It is well tested and looks to support quad precision, something we will probably want at some point. It also looks to be highly templated and tightly integrated, so I suspect getting it properly interfaced might be non-trivial. The same holds for the distributions, but we have done much the same. It might be worth looking over the boost classes for some ideas.
As to the size of the code, the current scipy/special library is ~40MB and I expect we can get rid of some of that.
It's ~11Mb for me. We should check for LLVM compatibility to make sure Apple isn't a problem,
but it looks like most other C++ compilers will work, Boost does try hard for universality.
Compile times will probably increase if we keep all the templates.
That's a much bigger worry than how many Mb of source code it is. Does anyone have an idea of how much compile time would increase and how much RAM it would use? Ralf
On Sun, Oct 7, 2012 at 3:52 PM, Ralf Gommers <ralf.gommers@gmail.com> wrote:
On Sun, Oct 7, 2012 at 6:21 PM, Charles R Harris <charlesr.harris@gmail.com> wrote:
On Sun, Oct 7, 2012 at 9:40 AM, Pauli Virtanen <pav@iki.fi> wrote:
Hi,
I'd like to consider replacing some of the function implementations in scipy.special with versions from the C++ Boost library, cf.
http://www.boost.org/doc/libs/1_51_0/libs/math/doc/sf_and_dist/html/index.ht...
The reason is that these implementations appear to be made with more care than what we use currently, and behave more correctly in corner cases (cf. e.g. ticket #1740).
To minimize work, it would be useful just to use the Boost functions direcly, rather than doing manual C transcriptions. The drawback here is that the portion of Boost library required weights about 8 MB of source code, and we would most likely like to bundle it, as it is not really a standard part of many installations. This does not reflect much on the compiled binary size, however.
I'm not 100 % certain about the compiler support. Perhaps C++ is already mature enough to work across the platforms we care about.
An obvious worry is Windows. MinGW is explicitly not supported, and "may or may not work": http://www.boost.org/doc/libs/1_51_0/more/getting_started/windows.html
I think that line refers only to MSYS shell, not to MinGW compiler. A few years ago I managed to get quantlib with python bindings build on top of boost with MinGW without too much problems, given that I had not much idea of what I was doing. (I'm not sure I didn't get boost binaries and only build quantlib) It looks like there are lots of instructions and examples building boost with mingw on the web. Josef
I'm not aware of many good BSD-compatible floating-point special function libraries, so if you know others, or would be opposed to bundling Boost, please chime up!
I think using the boost library is a good idea. It is well tested and looks to support quad precision, something we will probably want at some point. It also looks to be highly templated and tightly integrated, so I suspect getting it properly interfaced might be non-trivial. The same holds for the distributions, but we have done much the same. It might be worth looking over the boost classes for some ideas.
As to the size of the code, the current scipy/special library is ~40MB and I expect we can get rid of some of that.
It's ~11Mb for me.
We should check for LLVM compatibility to make sure Apple isn't a problem, but it looks like most other C++ compilers will work, Boost does try hard for universality.
Compile times will probably increase if we keep all the templates.
That's a much bigger worry than how many Mb of source code it is. Does anyone have an idea of how much compile time would increase and how much RAM it would use?
Ralf
_______________________________________________ SciPy-Dev mailing list SciPy-Dev@scipy.org http://mail.scipy.org/mailman/listinfo/scipy-dev
On Sun, Oct 7, 2012 at 8:40 AM, Pauli Virtanen <pav@iki.fi> wrote:
I'm not 100 % certain about the compiler support. Perhaps C++ is already mature enough to work across the platforms we care about.
It's worth having a good test of the compiler situation first. I have horrible memories a few years ago (not that long, though) of trying to use the Boost Graph Library for a project with python wrappers, and the Boost version this used wouldn't even compile with the newest gcc of the current ubuntu at the time. My only solution was to build gcc itself from source. So before we end up foisting upon RedHat users or similar the requirement that to build scipy they need to rebuild their compilers from source, which I suspect wouldn't be a very popular move, let's find out what is the oldest version of gcc this particular part of boost will require. Boost is great, but it's also famous for pushing compilers very, very far beyond their comfort zone. So this step should not be taken lightly. It would also be good to know: - does it compile with MS compilers? If so, what's the oldest version that works? - and what about the Intel ones? Not trying to rain on the parade, but over the last 10 years I've tried to use boost a few times, and every occasion has led to compiler pain. So I'd be cautious with putting it in as a scipy dependency. If it turns out that this part of Boost is less sensitive to compiler details, then great! I'd love to be proven wrong in my paranoia here... Cheers, f
On Sun, Oct 7, 2012 at 7:58 PM, Fernando Perez <fperez.net@gmail.com> wrote:
On Sun, Oct 7, 2012 at 8:40 AM, Pauli Virtanen <pav@iki.fi> wrote:
I'm not 100 % certain about the compiler support. Perhaps C++ is already mature enough to work across the platforms we care about.
It's worth having a good test of the compiler situation first. I have horrible memories a few years ago (not that long, though) of trying to use the Boost Graph Library for a project with python wrappers, and the Boost version this used wouldn't even compile with the newest gcc of the current ubuntu at the time. My only solution was to build gcc itself from source.
So before we end up foisting upon RedHat users or similar the requirement that to build scipy they need to rebuild their compilers from source, which I suspect wouldn't be a very popular move, let's find out what is the oldest version of gcc this particular part of boost will require.
Boost is great, but it's also famous for pushing compilers very, very far beyond their comfort zone. So this step should not be taken lightly. It would also be good to know:
- does it compile with MS compilers? If so, what's the oldest version that works?
the boost doc page that Ralf linked to has Visual Studio 2003 and 2005 as example case for compilers and IDEs Josef
- and what about the Intel ones?
Not trying to rain on the parade, but over the last 10 years I've tried to use boost a few times, and every occasion has led to compiler pain. So I'd be cautious with putting it in as a scipy dependency.
If it turns out that this part of Boost is less sensitive to compiler details, then great! I'd love to be proven wrong in my paranoia here...
Cheers,
f _______________________________________________ SciPy-Dev mailing list SciPy-Dev@scipy.org http://mail.scipy.org/mailman/listinfo/scipy-dev
On Mon, Oct 8, 2012 at 12:58 AM, Fernando Perez <fperez.net@gmail.com> wrote:
On Sun, Oct 7, 2012 at 8:40 AM, Pauli Virtanen <pav@iki.fi> wrote:
I'm not 100 % certain about the compiler support. Perhaps C++ is already mature enough to work across the platforms we care about.
It's worth having a good test of the compiler situation first. I have horrible memories a few years ago (not that long, though) of trying to use the Boost Graph Library for a project with python wrappers, and the Boost version this used wouldn't even compile with the newest gcc of the current ubuntu at the time. My only solution was to build gcc itself from source.
So before we end up foisting upon RedHat users or similar the requirement that to build scipy they need to rebuild their compilers from source, which I suspect wouldn't be a very popular move, let's find out what is the oldest version of gcc this particular part of boost will require.
Boost is great, but it's also famous for pushing compilers very, very far beyond their comfort zone. So this step should not be taken lightly. It would also be good to know:
- does it compile with MS compilers? If so, what's the oldest version that works? - and what about the Intel ones?
Not trying to rain on the parade, but over the last 10 years I've tried to use boost a few times, and every occasion has led to compiler pain. So I'd be cautious with putting it in as a scipy dependency.
We don't want to put boost altogether as a dependency, for all the reasons you're giving. In this case, I don't mind so much, because: - the templates for each special function are used in only one compilation unit, so they will be compiled only once - there is a mode when you can force the exceptions do be disabled and use ERRNO instead, so not too many issues there either. David
On Mon, Oct 8, 2012 at 11:41 AM, David Cournapeau <cournape@gmail.com> wrote:
In this case, I don't mind so much, because: - the templates for each special function are used in only one compilation unit, so they will be compiled only once - there is a mode when you can force the exceptions do be disabled and use ERRNO instead, so not too many issues there either.
Wait, literally 'errno'? That seems like a terrible idea, given how tightly wound that API is with low-level system interfaces and compatibility hacks. C wrappers that looked like extern "C" double my_c_j0_wrapper(double x, int * errflags) { try { return boost_j0(x); } except (boost::exception e) { *errflags |= exc_to_flags(e); } } would be just as easy to auto-generate as anything else, and we'd never have to fight with thread-local storage platform compatibility issues or anything... -n
On Mon, Oct 8, 2012 at 12:09 PM, Nathaniel Smith <njs@pobox.com> wrote:
On Mon, Oct 8, 2012 at 11:41 AM, David Cournapeau <cournape@gmail.com> wrote:
In this case, I don't mind so much, because: - the templates for each special function are used in only one compilation unit, so they will be compiled only once - there is a mode when you can force the exceptions do be disabled and use ERRNO instead, so not too many issues there either.
Wait, literally 'errno'? That seems like a terrible idea, given how tightly wound that API is with low-level system interfaces and compatibility hacks.
While I am not a fan of errno either, errno_on_error seems the closest to what libraries using C semantics would expect: http://www.boost.org/doc/libs/1_51_0/libs/math/doc/sf_and_dist/html/math_too... As for compatibility goes, we have been using the C math function which has the same behaviour for quite some time now, David
To comment on compiler support for boost, I have supported Boost on Intel for some time. The problem compilers are pgi and xl, but there is pressure on these vendors to better support metatemplate progamming. -- Andy
On Mon, Oct 8, 2012 at 10:22 AM, Andy Ray Terrel <andy.terrel@gmail.com>wrote:
To comment on compiler support for boost, I have supported Boost on Intel for some time. The problem compilers are pgi and xl, but there is pressure on these vendors to better support metatemplate progamming.
I second Andy's comments about pgi and xl. Otherwise, I am generally in favor of adding Boost. Be Well Anthony
-- Andy _______________________________________________ SciPy-Dev mailing list SciPy-Dev@scipy.org http://mail.scipy.org/mailman/listinfo/scipy-dev
On Mon, Oct 8, 2012 at 9:41 AM, Anthony Scopatz <scopatz@gmail.com> wrote:
On Mon, Oct 8, 2012 at 10:22 AM, Andy Ray Terrel <andy.terrel@gmail.com>wrote:
To comment on compiler support for boost, I have supported Boost on Intel for some time. The problem compilers are pgi and xl, but there is pressure on these vendors to better support metatemplate progamming.
I second Andy's comments about pgi and xl. Otherwise, I am generally in favor of adding Boost.
I don't think the special_functions make use of metatemplate techniques. The ublas templates in boost/math do, but that is a different application. Chuck
On Mon, Oct 8, 2012 at 11:25 AM, Charles R Harris <charlesr.harris@gmail.com> wrote:
On Mon, Oct 8, 2012 at 9:41 AM, Anthony Scopatz <scopatz@gmail.com> wrote:
On Mon, Oct 8, 2012 at 10:22 AM, Andy Ray Terrel <andy.terrel@gmail.com> wrote:
To comment on compiler support for boost, I have supported Boost on Intel for some time. The problem compilers are pgi and xl, but there is pressure on these vendors to better support metatemplate progamming.
I second Andy's comments about pgi and xl. Otherwise, I am generally in favor of adding Boost.
I don't think the special_functions make use of metatemplate techniques. The ublas templates in boost/math do, but that is a different application.
Chuck
Well pgi is still doing really bad. http://www.boost.org/development/tests/trunk/developer/math.html
_______________________________________________ SciPy-Dev mailing list SciPy-Dev@scipy.org http://mail.scipy.org/mailman/listinfo/scipy-dev
Andy, I agree with your point but why are you linking to boost trunk as opposed to tests against a recent release? My understanding is that the IBM compiler team, for example, always makes sure that some version of boost builds against their C++ compilers, even if it is not bleeding-edge. A On Mon, Oct 8, 2012 at 8:22 PM, Andy Ray Terrel <andy.terrel@gmail.com>wrote:
On Mon, Oct 8, 2012 at 11:25 AM, Charles R Harris <charlesr.harris@gmail.com> wrote:
On Mon, Oct 8, 2012 at 9:41 AM, Anthony Scopatz <scopatz@gmail.com>
wrote:
On Mon, Oct 8, 2012 at 10:22 AM, Andy Ray Terrel <andy.terrel@gmail.com
wrote:
To comment on compiler support for boost, I have supported Boost on Intel for some time. The problem compilers are pgi and xl, but there is pressure on these vendors to better support metatemplate progamming.
I second Andy's comments about pgi and xl. Otherwise, I am generally in favor of adding Boost.
I don't think the special_functions make use of metatemplate techniques. The ublas templates in boost/math do, but that is a different application.
Chuck
Well pgi is still doing really bad.
http://www.boost.org/development/tests/trunk/developer/math.html
_______________________________________________ SciPy-Dev mailing list SciPy-Dev@scipy.org http://mail.scipy.org/mailman/listinfo/scipy-dev
_______________________________________________ SciPy-Dev mailing list SciPy-Dev@scipy.org http://mail.scipy.org/mailman/listinfo/scipy-dev
On Mon, Oct 8, 2012 at 11:27 AM, Aron Ahmadia <aron@ahmadia.net> wrote:
Andy, I agree with your point but why are you linking to boost trunk as opposed to tests against a recent release? My understanding is that the IBM compiler team, for example, always makes sure that some version of boost builds against their C++ compilers, even if it is not bleeding-edge.
The released version<http://www.boost.org/development/tests/release/developer/math.html>looks better. Chuck
On Mon, Oct 8, 2012 at 1:10 PM, Charles R Harris <charlesr.harris@gmail.com> wrote:
On Mon, Oct 8, 2012 at 11:27 AM, Aron Ahmadia <aron@ahmadia.net> wrote:
Andy, I agree with your point but why are you linking to boost trunk as opposed to tests against a recent release? My understanding is that the IBM compiler team, for example, always makes sure that some version of boost builds against their C++ compilers, even if it is not bleeding-edge.
The released version looks better.
Chuck
Aron, I linked to that one because it listed pgi, it didn't list xl. I don't have access to a machine with xl *nudge**nudge* but since you do for at least another week, you might provide some insight. Chuck, unfortunately that one is better because it doesn't list pgi. Personally, I use gcc, intel, and/or clang for almost everything so I'm not pushing for keeping progress back for pgi and xl, rather just commenting on the state of compilers since Fernando brought it up. -- Andy
_______________________________________________ SciPy-Dev mailing list SciPy-Dev@scipy.org http://mail.scipy.org/mailman/listinfo/scipy-dev
08.10.2012 02:58, Fernando Perez kirjoitti: [clip]
So before we end up foisting upon RedHat users or similar the requirement that to build scipy they need to rebuild their compilers from source, which I suspect wouldn't be a very popular move, let's find out what is the oldest version of gcc this particular part of boost will require.
The venerable gcc (GCC) 4.1.2 20080704 (Red Hat 4.1.2-48) manages to compile this subset of Boost without problems, so I wouldn't be too worried about Linux platforms. -- Pauli Virtanen
On Mon, Oct 8, 2012 at 11:19 PM, Pauli Virtanen <pav@iki.fi> wrote:
08.10.2012 02:58, Fernando Perez kirjoitti: [clip]
So before we end up foisting upon RedHat users or similar the requirement that to build scipy they need to rebuild their compilers from source, which I suspect wouldn't be a very popular move, let's find out what is the oldest version of gcc this particular part of boost will require.
The venerable gcc (GCC) 4.1.2 20080704 (Red Hat 4.1.2-48) manages to compile this subset of Boost without problems, so I wouldn't be too worried about Linux platforms.
numpy currently requires a specific rather-old version of mingw to build on win32, might be a good idea to double-check that too. -n
On Tue, Oct 9, 2012 at 12:28 AM, Nathaniel Smith <njs@pobox.com> wrote:
On Mon, Oct 8, 2012 at 11:19 PM, Pauli Virtanen <pav@iki.fi> wrote:
08.10.2012 02:58, Fernando Perez kirjoitti: [clip]
So before we end up foisting upon RedHat users or similar the requirement that to build scipy they need to rebuild their compilers from source, which I suspect wouldn't be a very popular move, let's find out what is the oldest version of gcc this particular part of boost will require.
The venerable gcc (GCC) 4.1.2 20080704 (Red Hat 4.1.2-48) manages to compile this subset of Boost without problems, so I wouldn't be too worried about Linux platforms.
numpy currently requires a specific rather-old version of mingw to build on win32, might be a good idea to double-check that too.
Pauli's branch builds for me with MinGW. Ralf
On Sun, Oct 7, 2012 at 11:40 AM, Pauli Virtanen <pav@iki.fi> wrote:
Hi,
I'd like to consider replacing some of the function implementations in scipy.special with versions from the C++ Boost library, cf. http://www.boost.org/doc/libs/1_51_0/libs/math/doc/sf_and_dist/html/index.ht...
The reason is that these implementations appear to be made with more care than what we use currently, and behave more correctly in corner cases (cf. e.g. ticket #1740).
To minimize work, it would be useful just to use the Boost functions direcly, rather than doing manual C transcriptions. The drawback here is that the portion of Boost library required weights about 8 MB of source code, and we would most likely like to bundle it, as it is not really a standard part of many installations. This does not reflect much on the compiled binary size, however.
I'm not 100 % certain about the compiler support. Perhaps C++ is already mature enough to work across the platforms we care about.
I'm not aware of many good BSD-compatible floating-point special function libraries, so if you know others, or would be opposed to bundling Boost, please chime up!
At SciPy2011 a few of us managed to convince Andreas Klöckner to bundle the necessary bits of boost with PyCUDA so as to remove a dependency and grow the user-base. I recall this causing trouble for some segment of his users that had boost installed on the system, but I don't know the details. Andreas (CCed): Do you remember what problems those were? Fred B. told me about it over lunch one day but I can't recall what exactly went wrong. David
It looks like Andreas' message didn't make it to the list, so here it is forwarded. Looks like there are a few things to learn about bundling boost, but nothing show-stopping. Of course, some of it (compiler hell) is out of our control, but "get a better compiler" is a reasonable response on most platforms (... maybe not Windows). ---------- Forwarded message ---------- From: Andreas Kloeckner <lists@informa.tiker.net> Date: Wed, Oct 10, 2012 at 7:43 PM Subject: Re: [SciPy-Dev] Bundling Boost? To: David Warde-Farley <wardefar@iro.umontreal.ca>, SciPy Developers List <scipy-dev@scipy.org> David Warde-Farley <wardefar@iro.umontreal.ca> writes:
On Sun, Oct 7, 2012 at 11:40 AM, Pauli Virtanen <pav@iki.fi> wrote:
Hi,
I'd like to consider replacing some of the function implementations in scipy.special with versions from the C++ Boost library, cf. http://www.boost.org/doc/libs/1_51_0/libs/math/doc/sf_and_dist/html/index.ht...
The reason is that these implementations appear to be made with more care than what we use currently, and behave more correctly in corner cases (cf. e.g. ticket #1740).
To minimize work, it would be useful just to use the Boost functions direcly, rather than doing manual C transcriptions. The drawback here is that the portion of Boost library required weights about 8 MB of source code, and we would most likely like to bundle it, as it is not really a standard part of many installations. This does not reflect much on the compiled binary size, however.
I'm not 100 % certain about the compiler support. Perhaps C++ is already mature enough to work across the platforms we care about.
I'm not aware of many good BSD-compatible floating-point special function libraries, so if you know others, or would be opposed to bundling Boost, please chime up!
At SciPy2011 a few of us managed to convince Andreas Klöckner to bundle the necessary bits of boost with PyCUDA so as to remove a dependency and grow the user-base. I recall this causing trouble for some segment of his users that had boost installed on the system, but I don't know the details.
Andreas (CCed): Do you remember what problems those were? Fred B. told me about it over lunch one day but I can't recall what exactly went wrong.
Two main failure modes: - User has boost headers in /usr/include/boost. If stuff goes wrong, you'll build the binary parts of boost that you ship (whether you have those depends on which of the libraries you use) against the non-matching headers on the user's system. This usually results in missing symbols, but could conceivably lead to crashes. This doesn't seem to happen too much if you make sure that you have a -I for your shipped headers (i.e. only old/broken compilers). - Two modules using their individual shipped versions of boost::python (if you plan on using that) get loaded. Really weird stuff starts happening. The fix here is to -Dboost=mypackageboost. Strangely, this works without a hitch. That way, the two live in different C++ namespaces. They won't know about each other, but they also won't step on each other's feet. I've found both of those to be pretty benign. See here: https://github.com/inducer/aksetup/blob/master/aksetup_helper.py#L534 and here: https://github.com/inducer/bpl-subset for my work on this. ("bpl" = "boost python library"). do-extract here uses bcp, which refers to this: http://www.boost.org/doc/libs/1_51_0/tools/bcp/doc/html/index.html HTH, Andreas
On Thu, Oct 11, 2012 at 6:11 AM, David Warde-Farley <wardefar@iro.umontreal.ca> wrote:
It looks like Andreas' message didn't make it to the list, so here it is forwarded.
Looks like there are a few things to learn about bundling boost, but nothing show-stopping. Of course, some of it (compiler hell) is out of our control, but "get a better compiler" is a reasonable response on most platforms (... maybe not Windows).
Bundling all of boost is not on the table I think: we are just talking about the math stuff which look mostly like simple templates, in which case most issues are not relevant. Using more boost should be a different discussion. David
On Thu, Oct 11, 2012 at 2:39 AM, David Cournapeau <cournape@gmail.com>wrote:
On Thu, Oct 11, 2012 at 6:11 AM, David Warde-Farley <wardefar@iro.umontreal.ca> wrote:
It looks like Andreas' message didn't make it to the list, so here it is forwarded.
Looks like there are a few things to learn about bundling boost, but nothing show-stopping. Of course, some of it (compiler hell) is out of our control, but "get a better compiler" is a reasonable response on most platforms (... maybe not Windows).
Bundling all of boost is not on the table I think: we are just talking about the math stuff which look mostly like simple templates, in which case most issues are not relevant. Using more boost should be a different discussion.
Well, I did notice more boost dependencies showing up in the include files. C++ libraries tend to have a lot of intertwined parts, making it difficult to tease out just that little bit you really want. Chuck
On Thu, Oct 11, 2012 at 2:42 PM, Charles R Harris <charlesr.harris@gmail.com> wrote:
On Thu, Oct 11, 2012 at 2:39 AM, David Cournapeau <cournape@gmail.com> wrote:
On Thu, Oct 11, 2012 at 6:11 AM, David Warde-Farley <wardefar@iro.umontreal.ca> wrote:
It looks like Andreas' message didn't make it to the list, so here it is forwarded.
Looks like there are a few things to learn about bundling boost, but nothing show-stopping. Of course, some of it (compiler hell) is out of our control, but "get a better compiler" is a reasonable response on most platforms (... maybe not Windows).
Bundling all of boost is not on the table I think: we are just talking about the math stuff which look mostly like simple templates, in which case most issues are not relevant. Using more boost should be a different discussion.
Well, I did notice more boost dependencies showing up in the include files. C++ libraries tend to have a lot of intertwined parts, making it difficult to tease out just that little bit you really want.
I guess that's what 'well designed' mean in c++ land :) More seriously, as long as we don't use the stuff outside math/special functions, it does not matter much (except for sdist size). And surely, smart_ptr and co can be removed in some ways or others. David
On Thu, Oct 11, 2012 at 8:10 AM, David Cournapeau <cournape@gmail.com>wrote:
On Thu, Oct 11, 2012 at 2:42 PM, Charles R Harris <charlesr.harris@gmail.com> wrote:
On Thu, Oct 11, 2012 at 2:39 AM, David Cournapeau <cournape@gmail.com> wrote:
On Thu, Oct 11, 2012 at 6:11 AM, David Warde-Farley <wardefar@iro.umontreal.ca> wrote:
It looks like Andreas' message didn't make it to the list, so here it is forwarded.
Looks like there are a few things to learn about bundling boost, but nothing show-stopping. Of course, some of it (compiler hell) is out of our control, but "get a better compiler" is a reasonable response on most platforms (... maybe not Windows).
Bundling all of boost is not on the table I think: we are just talking about the math stuff which look mostly like simple templates, in which case most issues are not relevant. Using more boost should be a different discussion.
Well, I did notice more boost dependencies showing up in the include
files.
C++ libraries tend to have a lot of intertwined parts, making it difficult to tease out just that little bit you really want.
I guess that's what 'well designed' mean in c++ land :)
I think it is called 'reusability' in c++ land :)
More seriously, as long as we don't use the stuff outside math/special functions, it does not matter much (except for sdist size). And surely, smart_ptr and co can be removed in some ways or others.
Chuck
Perhaps some day things will be easier: https://svn.boost.org/trac/boost/wiki/CMakeModularizeLibrary https://svn.boost.org/trac/boost/wiki/CMakeModularizationStatus
participants (13)
-
Andy Ray Terrel -
Anthony Scopatz -
Aron Ahmadia -
Charles R Harris -
David Cournapeau -
David Warde-Farley -
Fernando Perez -
josef.pktd＠gmail.com -
Nathaniel Smith -
Neal Becker -
Pauli Virtanen -
Ralf Gommers -
Scott Lasley