[Numpy-discussion] Caution about using intrisincs, and other 'advanced' optimizations

[David Cournapeau]

On Tue, Nov 12, 2013 at 6:17 PM, Julian Taylor <
jtaylor.debian at googlemail.com> wrote:

> On 12.11.2013 03:17, David Cournapeau wrote:
> > Hi there,
> >
> > I have noticed more and more subtle and hard to track serious bugs in
> > numpy and scipy, due to the use of advanced optimization features
> > (flags, or gcc intrinsics).
> >
> > I am wondering whether those are worth it: they compile wrongly under
> > quite a few configurations, and it is not always obvious to find the
> > cause (case in point: gcc 4.4 with numpy 1.8.0 causes infinite loop in
> > scipy.stats, which disappear if I disable the intrinsics in
> > numpy/npy_config.h). Maybe they should be disabled by default, and only
> > built in if required ? Do we know for sure they bring significant
> > improvements ?
>
> yes, e.g.
>
> http://yarikoptic.github.io/numpy-vbench/vb_vb_ufunc.html#numpy-add-scalar2-numpy-float32
> http://yarikoptic.github.io/numpy-vbench/vb_vb_ufunc.html#numpy-not-bool
>
> http://yarikoptic.github.io/numpy-vbench/vb_vb_ufunc.html#numpy-isnan-a-10types
> and many more.
>
> this benchmark runs on a pretty old amd, the improvements are greater on
> more modern AMD and Intel cpus.
>
> >
> > While gcc 4.4 is not the most recent compiler, it is not ancient either,
>
> I can't reproduce any issue with gcc 4.4.7
>

So my initial email was wrong, the issue appears with gcc 4.1 and
disappears with 4.4.


>
> Can you narrow it down to a specific intrinsic? they can be enabled and
> disabled in set ./numpy/core/setup_common.py
>

valgrind shows quite a few invalid read in BOOL_ functions when running the
scipy or sklearn test suite. BOOL_logical_or is the one that appears the
most often. I don't have time to track this down now, but I think it would
be good to have at least a system in place to disable the simd intrinsics
when building numpy.

David


_______________________________________________
> NumPy-Discussion mailing list
> NumPy-Discussion at scipy.org
> http://mail.scipy.org/mailman/listinfo/numpy-discussion
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20131113/d4bddce0/attachment.html>