[Numpy-discussion] numpy.mean still broken for largefloat32arrays

[josef.pktd at gmail.com]

On Sat, Jul 26, 2014 at 2:44 PM, Benjamin Root <ben.root at ou.edu> wrote:

> That is one way of doing it, and probably the cleanest way. Or else you
> have to pass in the context object everywhere anyway. But I am not so
> concerned about that (we do that for other things as well). Bigger concerns
> would be nested contexts. For example, what if one of the scikit functions
> use such a context to explicitly state that they need a particular
> reduction algorithm, but the call to that scikit function is buried under a
> few layers of user functions, at the top of which has a context manager
> that states a different reduction op.
>
> Whose context wins? Naively, the scikit's context wins (because that's how
> contexts work). But, does that break with the very broad design goal here?
> To let the user specify the reduction kernel? Practically speaking, we
> could see users naively puting in context managers all over the place in
> their libraries, possibly choosing incorrect algorithms (I am serious here,
> how often have we seen stackoverflow instructions just blindly parrot
> certain arguments "just because")? This gives the user no real mechanism to
> override the library, largely defeating the purpose.
>
> My other concern would be with multi-threaded code (which is where a
> global state would be bad).
>


statsmodels still has avoided anything that smells like a global state that
changes calculation.
(We never even implemented different global warning levels.)

https://groups.google.com/d/msg/pystatsmodels/-J9WXKLjyH4/5xvKu9_mbbEJ


Josef
There be Dragons.



>
> Ben
>
>
>
> On Sat, Jul 26, 2014 at 2:29 PM, <josef.pktd at gmail.com> wrote:
>
>>
>>
>>
>> On Sat, Jul 26, 2014 at 9:57 AM, Benjamin Root <ben.root at ou.edu> wrote:
>>
>>> I could get behind the context manager approach. It would help keep
>>> backwards compatibility, while providing a very easy (and clean) way of
>>> consistently using the same reduction operation. Adding kwargs is just a
>>> road to hell.
>>>
>>
>> Wouldn't a context manager require a global state that changes how
>> everything is calculated ?
>>
>> Josef
>>
>>
>>
>>>
>>> Cheers!
>>> Ben Root
>>>
>>>
>>> On Sat, Jul 26, 2014 at 9:53 AM, Julian Taylor <
>>> jtaylor.debian at googlemail.com> wrote:
>>>
>>>> On 26.07.2014 15:38, Eelco Hoogendoorn wrote:
>>>> >
>>>> > Why is it not always used?
>>>>
>>>> for 1d reduction the iterator blocks by 8192 elements even when no
>>>> buffering is required. There is a TODO in the source to fix that by
>>>> adding additional checks. Unfortunately nobody knows hat these
>>>> additional tests would need to be and Mark Wiebe who wrote it did not
>>>> reply to a ping yet.
>>>>
>>>> Also along the non-fast axes the iterator optimizes the reduction to
>>>> remove the strided access, see:
>>>> https://github.com/numpy/numpy/pull/4697#issuecomment-42752599
>>>>
>>>>
>>>> Instead of having a keyword argument to mean I would prefer a context
>>>> manager that changes algorithms for different requirements.
>>>> This would easily allow changing the accuracy and performance of third
>>>> party functions using numpy without changing the third party library as
>>>> long as they are using numpy as the base.
>>>> E.g.
>>>> with np.precisionstate(sum="kahan"):
>>>>   scipy.stats.nanmean(d)
>>>>
>>>> We also have case where numpy uses algorithms that are far more precise
>>>> than most people needs them. E.g. np.hypot and the related complex
>>>> absolute value and division.
>>>> These are very slow with glibc as it provides 1ulp accuracy, this is
>>>> hardly ever needed.
>>>> Another case that could use dynamic changing is flushing subnormals to
>>>> zero.
>>>>
>>>> But this api is like Nathaniels parameterizable dtypes just an idea
>>>> floating in my head which needs proper design and implementation written
>>>> down. The issue is as usual ENOTIME.
>>>> _______________________________________________
>>>> NumPy-Discussion mailing list
>>>> NumPy-Discussion at scipy.org
>>>> http://mail.scipy.org/mailman/listinfo/numpy-discussion
>>>>
>>>
>>>
>>> _______________________________________________
>>> NumPy-Discussion mailing list
>>> NumPy-Discussion at scipy.org
>>> http://mail.scipy.org/mailman/listinfo/numpy-discussion
>>>
>>>
>>
>> _______________________________________________
>> NumPy-Discussion mailing list
>> NumPy-Discussion at scipy.org
>> http://mail.scipy.org/mailman/listinfo/numpy-discussion
>>
>>
>
> _______________________________________________
> NumPy-Discussion mailing list
> NumPy-Discussion at scipy.org
> http://mail.scipy.org/mailman/listinfo/numpy-discussion
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20140726/82703ecc/attachment.html>