[Numpy-discussion] adding fused multiply and add to numpy

Julian Taylor jtaylor.debian at googlemail.com
Thu Jan 9 10:30:14 EST 2014

On Thu, Jan 9, 2014 at 3:50 PM, Frédéric Bastien <nouiz at nouiz.org> wrote:

> Hi,
> It happen frequently that NumPy isn't compiled with all instruction
> that is available where it run. For example in distro. So if the
> decision is made to use the fast version when we don't use the newer
> instruction, the user need a way to know that. So the library need a
> function/attribute to tell that.

As these instructions are very new runtime cpu feature detection is
required. That way also distribution users get the fast code if their cpu
supports it.

> How hard would it be to provide the choise to the user? We could
> provide 2 functions like: fma_fast() fma_prec() (for precision)? Or
> this could be a parameter or a user configuration option like for the
> overflow/underflow error.

I like Freddie Witherden proposal to name the function madd which does not
guarantee one rounding operation. This leaves the namespace open for a
special fma function with that guarantee. It can use the libc fma function
which is very slow sometimes but platform independent. This is assuming
apple did not again take shortcuts like they did with their libc hypot
implementation, can someone disassemble apple libc to check what they are
doing for C99 fma?
And it leaves users the possibility to use the faster madd function if they
do not need the precision guarantee.

Another option would be a precision context manager which tells numpy which
variant to use. This would also be useful for other code (like
abs/hypot/abs2/sum/reciprocal sqrt) but probably it involves more work.

with numpy.precision_mode('fast'):
  ... # allow no fma, use fast hypot, fast sum, ignore overflow/invalid

with numpy.precision_mode('precise'):
  ... # require fma, use precise hypot, use exact summation (math.fsum) or
at least kahan summation, full overflow/invalid checks etc

> On Thu, Jan 9, 2014 at 9:43 AM, Freddie Witherden <freddie at witherden.org>
> wrote:
> > On 08/01/14 21:39, Julian Taylor wrote:
> >> An issue is software emulation of real fma. This can be enabled in the
> >> test ufunc with npfma.set_type("libc").
> >> This is unfortunately incredibly slow about a factor 300 on my machine
> >> without hardware fma.
> >> This means we either have a function that is fast on some platforms and
> >> slow on others but always gives the same result or we have a fast
> >> function that gives better results on some platforms.
> >> Given that we are not worth that what numpy currently provides I favor
> >> the latter.
> >>
> >> Any opinions on whether this should go into numpy or maybe stay a third
> >> party ufunc?
> >
> > My preference would be to initially add an "madd" intrinsic.  This can
> > be supported on all platforms and can be documented to permit the use of
> > FMA where available.
> >
> > A 'true' FMA intrinsic function should only be provided when hardware
> > FMA support is available.  Many of the more interesting applications of
> > FMA depend on there only being a single rounding step and as such "FMA"
> > should probably mean "a*b + c with only a single rounding".
> >
> > Regards, Freddie.
> >
> >
> > _______________________________________________
> > NumPy-Discussion mailing list
> > NumPy-Discussion at scipy.org
> > http://mail.scipy.org/mailman/listinfo/numpy-discussion
> >
> _______________________________________________
> NumPy-Discussion mailing list
> NumPy-Discussion at scipy.org
> http://mail.scipy.org/mailman/listinfo/numpy-discussion
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20140109/c45a88f8/attachment.html>

More information about the NumPy-Discussion mailing list