On Wed, Jul 2, 2014 at 10:36 AM, Julian Taylor <jtaylor.debian@googlemail.com> wrote:

we recently fixed a float32/float64 issue in histogram.
https://github.com/numpy/numpy/issues/4799

It's a good idea to keep the edges in the same dtype as the input data, it will make for fewer surprises, but I'm not sure that it's necessarily any more "correct". A value within an eps of a bin could arbitrarily end up on either side -- that's simply the nature of floating point.

 
I think it boils down to the use of round() in histogram which is not so
great in python as its based on decimals not significant figures (so it
does nothing for float32 values > 1e7).

Using decimals rather than sig-figs is a problem regardless of precision, and isn't that the same problem with C libmath round() ?

-CHB




--

Christopher Barker, Ph.D.
Oceanographer

Emergency Response Division
NOAA/NOS/OR&R            (206) 526-6959   voice
7600 Sand Point Way NE   (206) 526-6329   fax
Seattle, WA  98115       (206) 526-6317   main reception

Chris.Barker@noaa.gov