[Numpy-discussion] numpy.mean still broken for largefloat32arrays

[Eelco Hoogendoorn]

A context manager makes sense.

I very much appreciate the time constraints and the effort put in this far,
but if we can not make something work uniformly, I wonder if we should
include it in the master at all. I don't have a problem with customizing
algorithms where fp accuracy demands it; I have more of a problem with hard
to predict behavior. If np.ones(bigN).sum() gives different results than
np.ones((bigN,2)).sum(0), aside from the obvious differences, that would be
one hard to catch source of bugs.

Wouldn't per-axis reduction, as a limited form of nested reduction, provide
most of the benefits, without any of the drawbacks?


On Sat, Jul 26, 2014 at 3:53 PM, Julian Taylor <
jtaylor.debian at googlemail.com> wrote:

> On 26.07.2014 15:38, Eelco Hoogendoorn wrote:
> >
> > Why is it not always used?
>
> for 1d reduction the iterator blocks by 8192 elements even when no
> buffering is required. There is a TODO in the source to fix that by
> adding additional checks. Unfortunately nobody knows hat these
> additional tests would need to be and Mark Wiebe who wrote it did not
> reply to a ping yet.
>
> Also along the non-fast axes the iterator optimizes the reduction to
> remove the strided access, see:
> https://github.com/numpy/numpy/pull/4697#issuecomment-42752599
>
>
> Instead of having a keyword argument to mean I would prefer a context
> manager that changes algorithms for different requirements.
> This would easily allow changing the accuracy and performance of third
> party functions using numpy without changing the third party library as
> long as they are using numpy as the base.
> E.g.
> with np.precisionstate(sum="kahan"):
>   scipy.stats.nanmean(d)
>
> We also have case where numpy uses algorithms that are far more precise
> than most people needs them. E.g. np.hypot and the related complex
> absolute value and division.
> These are very slow with glibc as it provides 1ulp accuracy, this is
> hardly ever needed.
> Another case that could use dynamic changing is flushing subnormals to
> zero.
>
> But this api is like Nathaniels parameterizable dtypes just an idea
> floating in my head which needs proper design and implementation written
> down. The issue is as usual ENOTIME.
> _______________________________________________
> NumPy-Discussion mailing list
> NumPy-Discussion at scipy.org
> http://mail.scipy.org/mailman/listinfo/numpy-discussion
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20140726/c0dd683b/attachment.html>