Hello! Function `nextafter(x, y)` returns the next representable value of `x` in the direction of `y`, and if `x` equals to `y`, `y` is returned. [1] It is useful for incrementing/decrementing floating-point number by the smallest amount possible or for testing if two numbers are closest to each other without being the same. The following snippet written by Mark Dickinson emulates the functionality in pure Python [2]: import math import struct def next_up(x): # NaNs and positive infinity map to themselves. if math.isnan(x) or (math.isinf(x) and x > 0): return x # 0.0 and -0.0 both map to the smallest +ve float. if x == 0.0: x = 0.0 n = struct.unpack('<q', struct.pack('<d', x))[0] if n >= 0: n += 1 else: n -= 1 return struct.unpack('<d', struct.pack('<q', n))[0] def next_down(x): return -next_up(-x) def next_after(x, y): # If either argument is a NaN, return that argument. # This matches the implementation in decimal.Decimal if math.isnan(x): return x if math.isnan(y): return y if y == x: return y elif y > x: return next_up(x) else: return next_down(x) Other implementations can be found at [3], [4] or [5], for example. It would be useful to have `math.nextafter` function available in standard library, it also is provided by C99 <math.h>, and is similar in spirit to `math.copysign` or `math.isclose`. As to why to include it by default when the above snippet works just fine, a C implementation is likely to be much faster than using `struct.pack` and `struct.unpack`. Thank you for considering this proposal and any feedback is greatly welcomed! Juraj Sukop [1] http://en.cppreference.com/w/c/numeric/math/nextafter [2] http://stackoverflow.com/a/10426033 [3] http://git.musl-libc.org/cgit/musl/tree/src/math/nextafter.c [4] https://github.com/android/platform_bionic/blob/master/libm/upstream-freebsd... [5] https://github.com/golang/go/blob/master/src/math/nextafter.go
The presence of the function in C99’s math.h isn’t strictly useful unless it is also in the MSVC math.h. MSVC only supports a subset of C99 From: Python-ideas [mailto:python-ideas-bounces+tritium-list=sdamon.com@python.org] On Behalf Of Juraj Sukop Sent: Saturday, February 4, 2017 6:31 AM To: python-ideas@python.org Subject: [Python-ideas] math.nextafter Hello! Function `nextafter(x, y)` returns the next representable value of `x` in the direction of `y`, and if `x` equals to `y`, `y` is returned. [1] It is useful for incrementing/decrementing floating-point number by the smallest amount possible or for testing if two numbers are closest to each other without being the same. The following snippet written by Mark Dickinson emulates the functionality in pure Python [2]: import math import struct def next_up(x): # NaNs and positive infinity map to themselves. if math.isnan(x) or (math.isinf(x) and x > 0): return x # 0.0 and -0.0 both map to the smallest +ve float. if x == 0.0: x = 0.0 n = struct.unpack('<q', struct.pack('<d', x))[0] if n >= 0: n += 1 else: n -= 1 return struct.unpack('<d', struct.pack('<q', n))[0] def next_down(x): return -next_up(-x) def next_after(x, y): # If either argument is a NaN, return that argument. # This matches the implementation in decimal.Decimal if math.isnan(x): return x if math.isnan(y): return y if y == x: return y elif y > x: return next_up(x) else: return next_down(x) Other implementations can be found at [3], [4] or [5], for example. It would be useful to have `math.nextafter` function available in standard library, it also is provided by C99 <math.h>, and is similar in spirit to `math.copysign` or `math.isclose`. As to why to include it by default when the above snippet works just fine, a C implementation is likely to be much faster than using `struct.pack` and `struct.unpack`. Thank you for considering this proposal and any feedback is greatly welcomed! Juraj Sukop [1] http://en.cppreference.com/w/c/numeric/math/nextafter [2] http://stackoverflow.com/a/10426033 [3] http://git.musl-libc.org/cgit/musl/tree/src/math/nextafter.c [4] https://github.com/android/platform_bionic/blob/master/libm/upstream-freebsd... [5] https://github.com/golang/go/blob/master/src/math/nextafter.go
Hi all, Visual C++ 2015 supports this one: https://msdn.microsoft.com/en-us/library/h0dff77w.aspx In any case, this is easy to implement an efficient fallback in C, unlike the fma() function we discussed some time ago. To put this in a bit wider perspective: would it be useful to investigate how much of the C99 math library could be supported in Python in general? Stephan 2017-02-04 12:52 GMT+01:00 <tritium-list@sdamon.com>:
On 04.02.2017 12:59, Stephan Houben wrote:
+1 from me for those features which can be emulated for platforms which don't have the math lib function available and are not too esoteric (though many of those have already been added), e.g. cbt() may be useful. Now, with respect to the one mentioned in the subject, I'm not sure how useful this would be in the stdlib, since it's very much tied to whatever float type Python happens to use on a platform. Juraj: Could you provide some use cases, where such a function would help in Python applications ? (I can see use cases written in C, but due to the low level, find it hard to believe that people would use this at the Python level) Thanks, -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Experts (#1, Feb 06 2017)
::: We implement business ideas - efficiently in both time and costs ::: eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 http://www.egenix.com/company/contact/ http://www.malemburg.com/
On Mon, Feb 6, 2017 at 11:29 AM, M.-A. Lemburg <mal@egenix.com> wrote:
In my case, `nextafter` would be used to check if a number is close to polynomial zero, e.g.: def f(x): return 2.0*x**3 - 3.0*x**2 + 5.0*x - 7.0 # x = 1.4455284586795218 x = 1.445528458679522 # x = 1.4455284586795223 # x = 1.4455284586795225 left = nextafter(x, -float('inf')) right = nextafter(x, float('inf')) print((f(left) < 0.0) != (f(x) < 0.0) or (f(x) < 0.0) != (f(right) < 0.0))
On 06.02.2017 13:22, Juraj Sukop wrote:
Isn't this something you can do with math.isclose() ? This would even give you a predefined error range, not a dynamic one. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Experts (#1, Feb 06 2017)
::: We implement business ideas - efficiently in both time and costs ::: eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 http://www.egenix.com/company/contact/ http://www.malemburg.com/
On Mon, Feb 6, 2017 at 5:42 AM, Juraj Sukop <juraj.sukop@gmail.com> wrote:
isclose is mostly about "relative" closeness, so rel_tol is more-or-less the number of decimal digits you want the same: In [13]: isclose(1.11111111119, 1.11111111118, rel_tol=1e-11) Out[13]: True 11 digits are the same In [14]: isclose(1.11111111119, 1.11111111118, rel_tol=1e-12) Out[14]: False 12 aren't ( I say more or less, because FP values are in binary and we tend to think in decimal) You might think you'd want to know if the two values are as close as possible given the FP representation -- which is what nextafter() would give you -- but it is very, very, rare that a computation accurately preserves ALL the digits -- and if it did, you could use the regular old ==, <, > checks. But NOTHING is relatively close to zero, so if you want to compare to zero you need an absolute tolerance -- what is close to zero in your application: In [19]: isclose(1e-100, 0.0, abs_tol = 1e-101) Out[19]: False In [20]: isclose(1e-100, 0.0, abs_tol = 1e-100) Out[20]: True Again, you probably don't want the smallest possible representable FP value -- cause if your computation was that perfect, why not just check for zero? Figuring out what value makes sense in your use case requires either careful numerical analysis (that few of us are capable of!), or experimentation -- how small a value can you use and get the algorithm to converge? -Chris See PEP 485 for more detail: https://www.python.org/dev/peps/pep-0485/
-- Christopher Barker, Ph.D. Oceanographer Emergency Response Division NOAA/NOS/OR&R (206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception Chris.Barker@noaa.gov
On Fri, Feb 24, 2017 at 5:01 AM, Chris Barker <chris.barker@noaa.gov> wrote:
cause if your computation was that perfect, why not just check for zero?
A polynomial root may simply be not representable in double precision floating-point format. Per the example I posted above, the best one can hope for in such situation is to find (x, nextafter(x, float('inf'))) interval which has opposite function value signs.
On Fri, Feb 24, 2017 at 1:40 AM, Juraj Sukop <juraj.sukop@gmail.com> wrote:
Indeed.
I'm not much of a numerical analyst, but I think the number of applications in which the computation is accurate to the limit of float precision is vanishingly small -- so you need something larger than the smallest representable non-zero value anyway. Also many time when one is looking for a "zero", you are really looking for the difference between two values to be zero -- in which case isclose() is the right tool for the job. Thus this kind of thing is useful primarily for "exploring the behaviour of numerical algorithm implementations" as Nick said. Which doesn't mean it wouldn't' be useful to have in Python. Someone said: " it's very much tied to whatever float type Python happens to use on a platform." Which is the whole point -- if one could assume that Python is using IEEE 754 64 bit floats, then you could write these functions yourself but it's built-in then the implementation can make sure it's correct for the platform/etc that you are currently running on. By the way, it looks like math doesn't have machine epsilon either: https://en.wikipedia.org/wiki/Machine_epsilon which would be handy as well. -CHB -- Christopher Barker, Ph.D. Oceanographer Emergency Response Division NOAA/NOS/OR&R (206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception Chris.Barker@noaa.gov
On Feb 24, 2017, at 10:28 AM, Mahmoud Hashemi <mahmoud@hatnote.com> wrote: By the way, it looks like math doesn't have machine epsilon either:
https://en.wikipedia.org/wiki/Machine_epsilon
Pretty sure machine epsilon is in the sys module's float_info object.
Ahh, thanks! I though I remembered it was somewhere. Or are you saying it would be handy to alias sys.float_info.epsilon over to the math module too? That might be a good idea, yes. Particularly if other related functions are added as being discussed. -CHB
On 07.02.2017 00:46, Steven D'Aprano wrote:
(I'm not sure what you mean with "µPy") By far most implementations will use IEEE 754 64-bit doubles: https://en.wikipedia.org/wiki/Double-precision_floating-point_format MicroPython also supports single precision float configurations or no floats at all (depends on the availability of an FPU). What I wanted to say with the above comment is that Python itself does not make guarantees on how floats are represented internally (in CPython it's a C double, which only has minimum requirements). In theory, it would be possible to have a Python implementation using e.g. binary128 floats or MPFR (http://www.mpfr.org/). You are right in saying that this is a rather unlikely case :-) However, using the internal representation of floats to define a "valid" interval for roots doesn't sound like a good concept. Those intervals will depend on where the roots are. Close to 0, the intervals will be very small, near the outer range limits, very large: https://blogs.msdn.microsoft.com/dwayneneed/2010/05/06/fun-with-floating-poi... I guess there is some use in trying to determine the error intervals of calculations, though. Perhaps that's the main intent of those functions. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Experts (#1, Feb 07 2017)
::: We implement business ideas - efficiently in both time and costs ::: eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 http://www.egenix.com/company/contact/ http://www.malemburg.com/
On 6 February 2017 at 20:29, M.-A. Lemburg <mal@egenix.com> wrote:
Just from an API point of view, a more idiomatically-Pythonic concept seems to me to be a "floatrange()" construct, where the default step was defined by the internal representation (i.e. it would increment by the smallest possible value). I'm not sure of any use cases outside exploring the behaviour of numerical algorithm implementations in the presence of mathematical discontinuities, though. Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia
On 24.02.2017 10:13, Nick Coghlan wrote:
Perhaps closeto() could be extended to address the use case: "Match anything within N number of smallest float representable intervals around float value x" https://www.python.org/dev/peps/pep-0485/ This could then be used to detect cases where it doesn't make sense to run additional rounds of refinement to find roots or local minima, since IEEE floats simply don't provide enough accuracy to dig deeper. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Experts (#1, Feb 24 2017)
::: We implement business ideas - efficiently in both time and costs ::: eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 http://www.egenix.com/company/contact/ http://www.malemburg.com/
Marc-André slightly misspelled the recent-ish addition of math.isclose(), but I agree that it is absolutely where a "nextafter" belongs. The function signature is already relatively complex to cover several different but related use cases. I.e.: is_close(a, b, *, rel_tol=1e-09, abs_tol=0.0) -> bool I think an additional keyword-only argument for `nextafter_tol` (maybe a different spelling) would fit very naturally there. This would allow someone to specify 1 for that degree of closeness, but it would also allow them to specify some integer larger than 1 without needed to write a loop calling `nextafter()` repeatedly. Yours, David... On Fri, Feb 24, 2017 at 5:29 AM, M.-A. Lemburg <mal@egenix.com> wrote:
-- Keeping medicines from the bloodstreams of the sick; food from the bellies of the hungry; books from the hands of the uneducated; technology from the underdeveloped; and putting advocates of freedom in prisons. Intellectual property is to the 21st century what the slave trade was to the 16th.
On Feb 24, 2017 5:29 PM, "David Mertz" <mertz@gnosis.cx> wrote: Marc-André slightly misspelled the recent-ish addition of math.isclose(), but I agree that it is absolutely where a "nextafter" belongs. The function signature is already relatively complex to cover several different but related use cases. I.e.: is_close(a, b, *, rel_tol=1e-09, abs_tol=0.0) -> bool I think an additional keyword-only argument for `nextafter_tol` (maybe a different spelling) would fit very naturally there. This would allow someone to specify 1 for that degree of closeness, but it would also allow them to specify some integer larger than 1 without needed to write a loop calling `nextafter()` repeatedly. My 2c: I disagree -- numerical tolerance based closeness is fundamentally different than floating point representation based closeness (often discussed with the term "ulp" = "unit in the last place". For example, the number that's closest to 1.0 from above is twice as far away in numerical terms as the number that's closest to it from below. Mixing these two concepts in one function is just confusing, and we're not going to run out of names. It's also a little weird to jump from nextafter to isclose, since AFAIK they have pretty much non-overlapping use cases... FWIW, numpy provides all of the following as separate functions: * an isclose equivalent * nextafter * a function for counting the number of ulps between two floats * a function for checking that two floats differ by at most N ulps I'm not enough of an expert on numerical analysis to have an opinion on how useful these would be for Python itself. They certainly are part of a complete IEEE754 implementation, and useful for exploring the details of how floats work, if nothing else. -n
On 25 February 2017 at 12:43, Nathaniel Smith <njs@pobox.com> wrote:
"Does NumPy offer it?" is a useful data point when asking "Is this feature useful for numerical analysis?", so it'd helpful to know that it does have them.
It seems like a reasonable thing to expose to me, rather than asking folks to derive their own equivalents based on the information in sys.float_info. Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia
I do agree that a math.count_ulps() would be very nice to have. Or whatever spelling NumPy users for that concept. I see nextafter as more similar. Yes, it's not a uniform increment, which is the whole point. If you want a convergence that is as good as it can be in float, that's the relevant measure. Likewise for "almost as good" being N ulps. On Feb 24, 2017 9:43 PM, "Nathaniel Smith" <njs@pobox.com> wrote: On Feb 24, 2017 5:29 PM, "David Mertz" <mertz@gnosis.cx> wrote: Marc-André slightly misspelled the recent-ish addition of math.isclose(), but I agree that it is absolutely where a "nextafter" belongs. The function signature is already relatively complex to cover several different but related use cases. I.e.: is_close(a, b, *, rel_tol=1e-09, abs_tol=0.0) -> bool I think an additional keyword-only argument for `nextafter_tol` (maybe a different spelling) would fit very naturally there. This would allow someone to specify 1 for that degree of closeness, but it would also allow them to specify some integer larger than 1 without needed to write a loop calling `nextafter()` repeatedly. My 2c: I disagree -- numerical tolerance based closeness is fundamentally different than floating point representation based closeness (often discussed with the term "ulp" = "unit in the last place". For example, the number that's closest to 1.0 from above is twice as far away in numerical terms as the number that's closest to it from below. Mixing these two concepts in one function is just confusing, and we're not going to run out of names. It's also a little weird to jump from nextafter to isclose, since AFAIK they have pretty much non-overlapping use cases... FWIW, numpy provides all of the following as separate functions: * an isclose equivalent * nextafter * a function for counting the number of ulps between two floats * a function for checking that two floats differ by at most N ulps I'm not enough of an expert on numerical analysis to have an opinion on how useful these would be for Python itself. They certainly are part of a complete IEEE754 implementation, and useful for exploring the details of how floats work, if nothing else. -n
On Fri, Feb 24, 2017 at 6:43 PM, Nathaniel Smith <njs@pobox.com> wrote:
My 2c: I disagree -- numerical tolerance based closeness is fundamentally different than floating point representation based closeness
agreed -- isclose() is a convenience function to provide folks a way to "do a reasonable thing" without delving into floating point representation issues, etc. INdeed, essentially the same approach could be used for Decimal and Fraction, though they aren't supported by math.isclose() due to the math module being written in C, and the rest of math being all about floats. It's also a little weird to jump from nextafter to isclose, since AFAIK
they have pretty much non-overlapping use cases...
Exactly -- and you can tell by this this thread that confusion is easy with this stuff -- putting them together will only sow more confusion. "Floating Point is Hard" Note how many really smart people on this list say things like " I'm no expert in numerical analysis.. " * nextafter
I think there is little question that these are useful for numerical analysis. I think the question is if enough people want to use Python for that kind of analysis to add it to the stdlib. My tendency is to say yes -- if someone wants to write the code, it shouldn't be that hard to maintain -- it's mostly going to be calls to the underlying C lib, yes? -CHB -- Christopher Barker, Ph.D. Oceanographer Emergency Response Division NOAA/NOS/OR&R (206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception Chris.Barker@noaa.gov
These days, the subset of C99 supported by MSVC is "most" of it, so feel free to start off by assuming the best, at least for new features (the version we use for 2.7 obviously is not improving). Cheers, Steve Top-posted from my Windows Phone -----Original Message----- From: "tritium-list@sdamon.com" <tritium-list@sdamon.com> Sent: 2/4/2017 3:53 To: "python-ideas@python.org" <python-ideas@python.org> Subject: Re: [Python-ideas] math.nextafter The presence of the function in C99’s math.h isn’t strictly useful unless it is also in the MSVC math.h. MSVC only supports a subset of C99 From: Python-ideas [mailto:python-ideas-bounces+tritium-list=sdamon.com@python.org] On Behalf Of Juraj Sukop Sent: Saturday, February 4, 2017 6:31 AM To: python-ideas@python.org Subject: [Python-ideas] math.nextafter Hello! Function `nextafter(x, y)` returns the next representable value of `x` in the direction of `y`, and if `x` equals to `y`, `y` is returned. [1] It is useful for incrementing/decrementing floating-point number by the smallest amount possible or for testing if two numbers are closest to each other without being the same. The following snippet written by Mark Dickinson emulates the functionality in pure Python [2]: import math import struct def next_up(x): # NaNs and positive infinity map to themselves. if math.isnan(x) or (math.isinf(x) and x > 0): return x # 0.0 and -0.0 both map to the smallest +ve float. if x == 0.0: x = 0.0 n = struct.unpack('<q', struct.pack('<d', x))[0] if n >= 0: n += 1 else: n -= 1 return struct.unpack('<d', struct.pack('<q', n))[0] def next_down(x): return -next_up(-x) def next_after(x, y): # If either argument is a NaN, return that argument. # This matches the implementation in decimal.Decimal if math.isnan(x): return x if math.isnan(y): return y if y == x: return y elif y > x: return next_up(x) else: return next_down(x) Other implementations can be found at [3], [4] or [5], for example. It would be useful to have `math.nextafter` function available in standard library, it also is provided by C99 <math.h>, and is similar in spirit to `math.copysign` or `math.isclose`. As to why to include it by default when the above snippet works just fine, a C implementation is likely to be much faster than using `struct.pack` and `struct.unpack`. Thank you for considering this proposal and any feedback is greatly welcomed! Juraj Sukop [1] http://en.cppreference.com/w/c/numeric/math/nextafter [2] http://stackoverflow.com/a/10426033 [3] http://git.musl-libc.org/cgit/musl/tree/src/math/nextafter.c [4] https://github.com/android/platform_bionic/blob/master/libm/upstream-freebsd... [5] https://github.com/golang/go/blob/master/src/math/nextafter.go
The presence of the function in C99’s math.h isn’t strictly useful unless it is also in the MSVC math.h. MSVC only supports a subset of C99 From: Python-ideas [mailto:python-ideas-bounces+tritium-list=sdamon.com@python.org] On Behalf Of Juraj Sukop Sent: Saturday, February 4, 2017 6:31 AM To: python-ideas@python.org Subject: [Python-ideas] math.nextafter Hello! Function `nextafter(x, y)` returns the next representable value of `x` in the direction of `y`, and if `x` equals to `y`, `y` is returned. [1] It is useful for incrementing/decrementing floating-point number by the smallest amount possible or for testing if two numbers are closest to each other without being the same. The following snippet written by Mark Dickinson emulates the functionality in pure Python [2]: import math import struct def next_up(x): # NaNs and positive infinity map to themselves. if math.isnan(x) or (math.isinf(x) and x > 0): return x # 0.0 and -0.0 both map to the smallest +ve float. if x == 0.0: x = 0.0 n = struct.unpack('<q', struct.pack('<d', x))[0] if n >= 0: n += 1 else: n -= 1 return struct.unpack('<d', struct.pack('<q', n))[0] def next_down(x): return -next_up(-x) def next_after(x, y): # If either argument is a NaN, return that argument. # This matches the implementation in decimal.Decimal if math.isnan(x): return x if math.isnan(y): return y if y == x: return y elif y > x: return next_up(x) else: return next_down(x) Other implementations can be found at [3], [4] or [5], for example. It would be useful to have `math.nextafter` function available in standard library, it also is provided by C99 <math.h>, and is similar in spirit to `math.copysign` or `math.isclose`. As to why to include it by default when the above snippet works just fine, a C implementation is likely to be much faster than using `struct.pack` and `struct.unpack`. Thank you for considering this proposal and any feedback is greatly welcomed! Juraj Sukop [1] http://en.cppreference.com/w/c/numeric/math/nextafter [2] http://stackoverflow.com/a/10426033 [3] http://git.musl-libc.org/cgit/musl/tree/src/math/nextafter.c [4] https://github.com/android/platform_bionic/blob/master/libm/upstream-freebsd... [5] https://github.com/golang/go/blob/master/src/math/nextafter.go
Hi all, Visual C++ 2015 supports this one: https://msdn.microsoft.com/en-us/library/h0dff77w.aspx In any case, this is easy to implement an efficient fallback in C, unlike the fma() function we discussed some time ago. To put this in a bit wider perspective: would it be useful to investigate how much of the C99 math library could be supported in Python in general? Stephan 2017-02-04 12:52 GMT+01:00 <tritium-list@sdamon.com>:
On 04.02.2017 12:59, Stephan Houben wrote:
+1 from me for those features which can be emulated for platforms which don't have the math lib function available and are not too esoteric (though many of those have already been added), e.g. cbt() may be useful. Now, with respect to the one mentioned in the subject, I'm not sure how useful this would be in the stdlib, since it's very much tied to whatever float type Python happens to use on a platform. Juraj: Could you provide some use cases, where such a function would help in Python applications ? (I can see use cases written in C, but due to the low level, find it hard to believe that people would use this at the Python level) Thanks, -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Experts (#1, Feb 06 2017)
::: We implement business ideas - efficiently in both time and costs ::: eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 http://www.egenix.com/company/contact/ http://www.malemburg.com/
On Mon, Feb 6, 2017 at 11:29 AM, M.-A. Lemburg <mal@egenix.com> wrote:
In my case, `nextafter` would be used to check if a number is close to polynomial zero, e.g.: def f(x): return 2.0*x**3 - 3.0*x**2 + 5.0*x - 7.0 # x = 1.4455284586795218 x = 1.445528458679522 # x = 1.4455284586795223 # x = 1.4455284586795225 left = nextafter(x, -float('inf')) right = nextafter(x, float('inf')) print((f(left) < 0.0) != (f(x) < 0.0) or (f(x) < 0.0) != (f(right) < 0.0))
On 06.02.2017 13:22, Juraj Sukop wrote:
Isn't this something you can do with math.isclose() ? This would even give you a predefined error range, not a dynamic one. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Experts (#1, Feb 06 2017)
::: We implement business ideas - efficiently in both time and costs ::: eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 http://www.egenix.com/company/contact/ http://www.malemburg.com/
On Mon, Feb 6, 2017 at 5:42 AM, Juraj Sukop <juraj.sukop@gmail.com> wrote:
isclose is mostly about "relative" closeness, so rel_tol is more-or-less the number of decimal digits you want the same: In [13]: isclose(1.11111111119, 1.11111111118, rel_tol=1e-11) Out[13]: True 11 digits are the same In [14]: isclose(1.11111111119, 1.11111111118, rel_tol=1e-12) Out[14]: False 12 aren't ( I say more or less, because FP values are in binary and we tend to think in decimal) You might think you'd want to know if the two values are as close as possible given the FP representation -- which is what nextafter() would give you -- but it is very, very, rare that a computation accurately preserves ALL the digits -- and if it did, you could use the regular old ==, <, > checks. But NOTHING is relatively close to zero, so if you want to compare to zero you need an absolute tolerance -- what is close to zero in your application: In [19]: isclose(1e-100, 0.0, abs_tol = 1e-101) Out[19]: False In [20]: isclose(1e-100, 0.0, abs_tol = 1e-100) Out[20]: True Again, you probably don't want the smallest possible representable FP value -- cause if your computation was that perfect, why not just check for zero? Figuring out what value makes sense in your use case requires either careful numerical analysis (that few of us are capable of!), or experimentation -- how small a value can you use and get the algorithm to converge? -Chris See PEP 485 for more detail: https://www.python.org/dev/peps/pep-0485/
-- Christopher Barker, Ph.D. Oceanographer Emergency Response Division NOAA/NOS/OR&R (206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception Chris.Barker@noaa.gov
On Fri, Feb 24, 2017 at 5:01 AM, Chris Barker <chris.barker@noaa.gov> wrote:
cause if your computation was that perfect, why not just check for zero?
A polynomial root may simply be not representable in double precision floating-point format. Per the example I posted above, the best one can hope for in such situation is to find (x, nextafter(x, float('inf'))) interval which has opposite function value signs.
On Fri, Feb 24, 2017 at 1:40 AM, Juraj Sukop <juraj.sukop@gmail.com> wrote:
Indeed.
I'm not much of a numerical analyst, but I think the number of applications in which the computation is accurate to the limit of float precision is vanishingly small -- so you need something larger than the smallest representable non-zero value anyway. Also many time when one is looking for a "zero", you are really looking for the difference between two values to be zero -- in which case isclose() is the right tool for the job. Thus this kind of thing is useful primarily for "exploring the behaviour of numerical algorithm implementations" as Nick said. Which doesn't mean it wouldn't' be useful to have in Python. Someone said: " it's very much tied to whatever float type Python happens to use on a platform." Which is the whole point -- if one could assume that Python is using IEEE 754 64 bit floats, then you could write these functions yourself but it's built-in then the implementation can make sure it's correct for the platform/etc that you are currently running on. By the way, it looks like math doesn't have machine epsilon either: https://en.wikipedia.org/wiki/Machine_epsilon which would be handy as well. -CHB -- Christopher Barker, Ph.D. Oceanographer Emergency Response Division NOAA/NOS/OR&R (206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception Chris.Barker@noaa.gov
On Feb 24, 2017, at 10:28 AM, Mahmoud Hashemi <mahmoud@hatnote.com> wrote: By the way, it looks like math doesn't have machine epsilon either:
https://en.wikipedia.org/wiki/Machine_epsilon
Pretty sure machine epsilon is in the sys module's float_info object.
Ahh, thanks! I though I remembered it was somewhere. Or are you saying it would be handy to alias sys.float_info.epsilon over to the math module too? That might be a good idea, yes. Particularly if other related functions are added as being discussed. -CHB
On 07.02.2017 00:46, Steven D'Aprano wrote:
(I'm not sure what you mean with "µPy") By far most implementations will use IEEE 754 64-bit doubles: https://en.wikipedia.org/wiki/Double-precision_floating-point_format MicroPython also supports single precision float configurations or no floats at all (depends on the availability of an FPU). What I wanted to say with the above comment is that Python itself does not make guarantees on how floats are represented internally (in CPython it's a C double, which only has minimum requirements). In theory, it would be possible to have a Python implementation using e.g. binary128 floats or MPFR (http://www.mpfr.org/). You are right in saying that this is a rather unlikely case :-) However, using the internal representation of floats to define a "valid" interval for roots doesn't sound like a good concept. Those intervals will depend on where the roots are. Close to 0, the intervals will be very small, near the outer range limits, very large: https://blogs.msdn.microsoft.com/dwayneneed/2010/05/06/fun-with-floating-poi... I guess there is some use in trying to determine the error intervals of calculations, though. Perhaps that's the main intent of those functions. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Experts (#1, Feb 07 2017)
::: We implement business ideas - efficiently in both time and costs ::: eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 http://www.egenix.com/company/contact/ http://www.malemburg.com/
On 6 February 2017 at 20:29, M.-A. Lemburg <mal@egenix.com> wrote:
Just from an API point of view, a more idiomatically-Pythonic concept seems to me to be a "floatrange()" construct, where the default step was defined by the internal representation (i.e. it would increment by the smallest possible value). I'm not sure of any use cases outside exploring the behaviour of numerical algorithm implementations in the presence of mathematical discontinuities, though. Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia
On 24.02.2017 10:13, Nick Coghlan wrote:
Perhaps closeto() could be extended to address the use case: "Match anything within N number of smallest float representable intervals around float value x" https://www.python.org/dev/peps/pep-0485/ This could then be used to detect cases where it doesn't make sense to run additional rounds of refinement to find roots or local minima, since IEEE floats simply don't provide enough accuracy to dig deeper. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Experts (#1, Feb 24 2017)
::: We implement business ideas - efficiently in both time and costs ::: eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 http://www.egenix.com/company/contact/ http://www.malemburg.com/
Marc-André slightly misspelled the recent-ish addition of math.isclose(), but I agree that it is absolutely where a "nextafter" belongs. The function signature is already relatively complex to cover several different but related use cases. I.e.: is_close(a, b, *, rel_tol=1e-09, abs_tol=0.0) -> bool I think an additional keyword-only argument for `nextafter_tol` (maybe a different spelling) would fit very naturally there. This would allow someone to specify 1 for that degree of closeness, but it would also allow them to specify some integer larger than 1 without needed to write a loop calling `nextafter()` repeatedly. Yours, David... On Fri, Feb 24, 2017 at 5:29 AM, M.-A. Lemburg <mal@egenix.com> wrote:
-- Keeping medicines from the bloodstreams of the sick; food from the bellies of the hungry; books from the hands of the uneducated; technology from the underdeveloped; and putting advocates of freedom in prisons. Intellectual property is to the 21st century what the slave trade was to the 16th.
On Feb 24, 2017 5:29 PM, "David Mertz" <mertz@gnosis.cx> wrote: Marc-André slightly misspelled the recent-ish addition of math.isclose(), but I agree that it is absolutely where a "nextafter" belongs. The function signature is already relatively complex to cover several different but related use cases. I.e.: is_close(a, b, *, rel_tol=1e-09, abs_tol=0.0) -> bool I think an additional keyword-only argument for `nextafter_tol` (maybe a different spelling) would fit very naturally there. This would allow someone to specify 1 for that degree of closeness, but it would also allow them to specify some integer larger than 1 without needed to write a loop calling `nextafter()` repeatedly. My 2c: I disagree -- numerical tolerance based closeness is fundamentally different than floating point representation based closeness (often discussed with the term "ulp" = "unit in the last place". For example, the number that's closest to 1.0 from above is twice as far away in numerical terms as the number that's closest to it from below. Mixing these two concepts in one function is just confusing, and we're not going to run out of names. It's also a little weird to jump from nextafter to isclose, since AFAIK they have pretty much non-overlapping use cases... FWIW, numpy provides all of the following as separate functions: * an isclose equivalent * nextafter * a function for counting the number of ulps between two floats * a function for checking that two floats differ by at most N ulps I'm not enough of an expert on numerical analysis to have an opinion on how useful these would be for Python itself. They certainly are part of a complete IEEE754 implementation, and useful for exploring the details of how floats work, if nothing else. -n
On 25 February 2017 at 12:43, Nathaniel Smith <njs@pobox.com> wrote:
"Does NumPy offer it?" is a useful data point when asking "Is this feature useful for numerical analysis?", so it'd helpful to know that it does have them.
It seems like a reasonable thing to expose to me, rather than asking folks to derive their own equivalents based on the information in sys.float_info. Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia
I do agree that a math.count_ulps() would be very nice to have. Or whatever spelling NumPy users for that concept. I see nextafter as more similar. Yes, it's not a uniform increment, which is the whole point. If you want a convergence that is as good as it can be in float, that's the relevant measure. Likewise for "almost as good" being N ulps. On Feb 24, 2017 9:43 PM, "Nathaniel Smith" <njs@pobox.com> wrote: On Feb 24, 2017 5:29 PM, "David Mertz" <mertz@gnosis.cx> wrote: Marc-André slightly misspelled the recent-ish addition of math.isclose(), but I agree that it is absolutely where a "nextafter" belongs. The function signature is already relatively complex to cover several different but related use cases. I.e.: is_close(a, b, *, rel_tol=1e-09, abs_tol=0.0) -> bool I think an additional keyword-only argument for `nextafter_tol` (maybe a different spelling) would fit very naturally there. This would allow someone to specify 1 for that degree of closeness, but it would also allow them to specify some integer larger than 1 without needed to write a loop calling `nextafter()` repeatedly. My 2c: I disagree -- numerical tolerance based closeness is fundamentally different than floating point representation based closeness (often discussed with the term "ulp" = "unit in the last place". For example, the number that's closest to 1.0 from above is twice as far away in numerical terms as the number that's closest to it from below. Mixing these two concepts in one function is just confusing, and we're not going to run out of names. It's also a little weird to jump from nextafter to isclose, since AFAIK they have pretty much non-overlapping use cases... FWIW, numpy provides all of the following as separate functions: * an isclose equivalent * nextafter * a function for counting the number of ulps between two floats * a function for checking that two floats differ by at most N ulps I'm not enough of an expert on numerical analysis to have an opinion on how useful these would be for Python itself. They certainly are part of a complete IEEE754 implementation, and useful for exploring the details of how floats work, if nothing else. -n
On Fri, Feb 24, 2017 at 6:43 PM, Nathaniel Smith <njs@pobox.com> wrote:
My 2c: I disagree -- numerical tolerance based closeness is fundamentally different than floating point representation based closeness
agreed -- isclose() is a convenience function to provide folks a way to "do a reasonable thing" without delving into floating point representation issues, etc. INdeed, essentially the same approach could be used for Decimal and Fraction, though they aren't supported by math.isclose() due to the math module being written in C, and the rest of math being all about floats. It's also a little weird to jump from nextafter to isclose, since AFAIK
they have pretty much non-overlapping use cases...
Exactly -- and you can tell by this this thread that confusion is easy with this stuff -- putting them together will only sow more confusion. "Floating Point is Hard" Note how many really smart people on this list say things like " I'm no expert in numerical analysis.. " * nextafter
I think there is little question that these are useful for numerical analysis. I think the question is if enough people want to use Python for that kind of analysis to add it to the stdlib. My tendency is to say yes -- if someone wants to write the code, it shouldn't be that hard to maintain -- it's mostly going to be calls to the underlying C lib, yes? -CHB -- Christopher Barker, Ph.D. Oceanographer Emergency Response Division NOAA/NOS/OR&R (206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception Chris.Barker@noaa.gov
These days, the subset of C99 supported by MSVC is "most" of it, so feel free to start off by assuming the best, at least for new features (the version we use for 2.7 obviously is not improving). Cheers, Steve Top-posted from my Windows Phone -----Original Message----- From: "tritium-list@sdamon.com" <tritium-list@sdamon.com> Sent: 2/4/2017 3:53 To: "python-ideas@python.org" <python-ideas@python.org> Subject: Re: [Python-ideas] math.nextafter The presence of the function in C99’s math.h isn’t strictly useful unless it is also in the MSVC math.h. MSVC only supports a subset of C99 From: Python-ideas [mailto:python-ideas-bounces+tritium-list=sdamon.com@python.org] On Behalf Of Juraj Sukop Sent: Saturday, February 4, 2017 6:31 AM To: python-ideas@python.org Subject: [Python-ideas] math.nextafter Hello! Function `nextafter(x, y)` returns the next representable value of `x` in the direction of `y`, and if `x` equals to `y`, `y` is returned. [1] It is useful for incrementing/decrementing floating-point number by the smallest amount possible or for testing if two numbers are closest to each other without being the same. The following snippet written by Mark Dickinson emulates the functionality in pure Python [2]: import math import struct def next_up(x): # NaNs and positive infinity map to themselves. if math.isnan(x) or (math.isinf(x) and x > 0): return x # 0.0 and -0.0 both map to the smallest +ve float. if x == 0.0: x = 0.0 n = struct.unpack('<q', struct.pack('<d', x))[0] if n >= 0: n += 1 else: n -= 1 return struct.unpack('<d', struct.pack('<q', n))[0] def next_down(x): return -next_up(-x) def next_after(x, y): # If either argument is a NaN, return that argument. # This matches the implementation in decimal.Decimal if math.isnan(x): return x if math.isnan(y): return y if y == x: return y elif y > x: return next_up(x) else: return next_down(x) Other implementations can be found at [3], [4] or [5], for example. It would be useful to have `math.nextafter` function available in standard library, it also is provided by C99 <math.h>, and is similar in spirit to `math.copysign` or `math.isclose`. As to why to include it by default when the above snippet works just fine, a C implementation is likely to be much faster than using `struct.pack` and `struct.unpack`. Thank you for considering this proposal and any feedback is greatly welcomed! Juraj Sukop [1] http://en.cppreference.com/w/c/numeric/math/nextafter [2] http://stackoverflow.com/a/10426033 [3] http://git.musl-libc.org/cgit/musl/tree/src/math/nextafter.c [4] https://github.com/android/platform_bionic/blob/master/libm/upstream-freebsd... [5] https://github.com/golang/go/blob/master/src/math/nextafter.go
participants (13)
-
Chris Barker -
Chris Barker - NOAA Federal
-
David Mertz -
Juraj Sukop -
M.-A. Lemburg -
Mahmoud Hashemi -
Nathaniel Smith -
Nick Coghlan -
Random832 -
Stephan Houben -
Steve Dower -
Steven D'Aprano -
tritium-list＠sdamon.com