On 11/10/12 09:56, Alexander Belopolsky wrote:

I gave this idea +float('inf') in the other thread and was thinking about it since. I am now toying with the idea to unify float and decimal in Python. IEEE combined their two FP standards in one recently, so we have a precedent for this.

We can start by extending decimal to support radix 2 and once that code is mature enough and has accelerated code for platform formats (single, double, long double),

I don't want to be greedy, but supporting minifloats would be a real boon to beginners trying to learn how floats work.

we can replace Python float with the new fully platform independent IEEE 754 compliant implementation. We can even supply a legacy context to support some current warts.

This all sounds very exciting, but also like a huge amount of work. -- Steven

On Wed, Oct 10, 2012 at 9:44 PM, Guido van Rossum wrote:

Anyway, as an indication of the amount of work, you might want to look at the fpectl module -- the module itself is tiny, but its introduction required a huge amount of changes to every place where CPython uses a double.

I would start from another end. I would look at decimal.py first. This is little over 6,400 line of code and I think most of it can be reused to implement base 2 (or probably better base 16) float. Multi-precision binary float can coexist with existing float until the code matures and accelerators are written for major platforms. At the same time we can make incremental improvements to builtin float until it can be replaced by a multi-precision float in some well-defined context.

On Oct 11, 2012, at 4:45 AM, Serhiy Storchaka wrote:

With base 16 floats you can't emulate x86 native 53-bit mantissa floats.

I realized that as soon as I hit send. :-( I also realized that it does not matter for python implementation because decimal stores mantissa as an int rather than a list of digits.

On 11 October 2012 06:45, Greg Ewing wrote:

Alexander Belopolsky wrote:

...

I gave this idea +float('inf') in the other thread and was thinking about it since. I am now toying with the idea to unify float and decimal in Python.

Are you sure there would be any point in this? People who specifically *want* base-2 floats are probably quite happy with the current float type, and wouldn't appreciate having it slowed down, even by a small amount.

It might make sense for them to share whatever parts of the fp context apply to both, and they might have a common base type, but they should probably remain distinct types with separate implementations.

This is what I was pitching at. It would be great if a single floating point context could be used to control the behaviour of float, decimal, ndarray etc simultaneously. Something that would have made my life easier yesterday would have been a way to enter a debugger at the point when a first NaN is created during execution. Something like: python -m pdb --error-nan broken_script.py Or perhaps: PYTHONRUNFIRST='import errornan' python broken_script.py With numpy you can already do: export PYTHONRUNFIRST='imoprt numpy; numpy.seterr(all='raise')' (Except that PYTHONRUNFIRST isn't implemented yet: http://bugs.python.org/issue14803) Oscar

On 11 October 2012 15:54, Guido van Rossum wrote:

I think you're mistaking my suggestion. I meant to recommend that there should be a way to control the behavior (e.g. whether to silently return Nan/Inf or raise an exception) of floating point operations, using the capabilities of the hardware as exposed through C, using Python's existing float type. I did not for a second consider reimplementing IEEE 754 from scratch. Therein lies insanity.

That's also why I recommended you look at the fpectl module.

I would like to have precisely the functionality you are suggesting and I don't want to reimplement anything (I assume this message is intended for me since it was addressed to me). I don't know enough about the implementation details to agree on the hardware capabilities part. From a quick glance at the fpectl module I see that it has problems with portability: http://docs.python.org/library/fpectl.html#fpectl-limitations Setting up a given processor to trap IEEE-754 floating point errors currently requires custom code on a per-architecture basis. You may have to modify fpectl to control your particular hardware. This presumably explains why I don't have the module in my Windows build or on the Linux machines in the HPC cluster I use. Are these problems that can be overcome? If it is necessary to have this hardware-specific accelerator for floating point exceptions then is it reasonable to expect implementations other than CPython to be able to match the semantics of floating point contexts without a significant degradation in performance? I was expecting the implementation to be some checks in straight forward C code for invalid values. I would expect this to cause a small degradation in performance (the kind that you wouldn't notice unless you went out of your way to measure it). Python already does this by checking for a zero value on every division. As far as I can tell from the numpy codebase this is how it works there. This function seems to be responsible for the integer division by zero result in numpy: https://github.com/numpy/numpy/blob/master/numpy/core/src/scalarmathmodule.c...

...

...

import numpy as np np.seterr() {'over': 'warn', 'divide': 'warn', 'invalid': 'warn', 'under': 'ignore'} np.int32(1) / np.int32(0) __main__:1: RuntimeWarning: divide by zero encountered in long_scalars 0 np.seterr(divide='ignore') {'over': 'warn', 'divide': 'warn', 'invalid': 'warn', 'under': 'ignore'} np.int32(1) / np.int32(0) 0 np.seterr(divide='raise') {'over': 'warn', 'divide': 'ignore', 'invalid': 'warn', 'under': 'ignore'} np.int32(1) / np.int32(0) Traceback (most recent call last): File "<stdin>", line 1, in <module> FloatingPointError: divide by zero encountered in long_scalars

This works perfectly well in numpy and also in decimal I see no reason why it couldn't work for float/int. But what would would be even better is if you could control all of them with a single context manager. Typically I don't care with the error occurred as a result of operations on ints/floats/ndarrays/decimals I just know that I got a NaN from somewhere and I need to debug it. Oscar

On 11/10/12 16:45, Greg Ewing wrote:

Alexander Belopolsky wrote:

...

I gave this idea +float('inf') in the other thread and was thinking about it since. I am now toying with the idea to unify float and decimal in Python.

Are you sure there would be any point in this? People who specifically *want* base-2 floats are probably quite happy with the current float type, and wouldn't appreciate having it slowed down, even by a small amount.

I would gladly give up a small amount of speed for better control over floats, such as whether 1/0.0 raised an exception or returned infinity. If I wanted fast code, I'd be using C. I'm happy with *fast enough*. For example, 1/0.0 in a continued fraction is generally harmless, provided it returns infinity. If it raises an exception, you have to write slow, ugly code to evaluate continued fractions robustly. I wouldn't expect 1/0.0 -> infinity to becomes the default, but I'd like a runtime switch to turn it on and off as needed. -- Steven

On Thu, Oct 11, 2012 at 11:31 PM, Sturla Molden wrote:

A global switch in the sys module would make life a lot easier. Even better would be a context manager that allows us to set up a "numerical" context for local expressions using a with statement. That would not have a lasting effect, but just affect the context. Preferably it should not even propagate across function calls. Something like this:

def foobar(): 1/0.0 # raise an exception 1/0 # raise an exception

with sys.numerical: 1/0.0 # return inf 1/0 # return 0 foobar()

Not propagating across function calls strikes me as messy, but I see why you'd want it. Would this be better as a __future__ directive? There's already the concept that they apply to a module but not to what that module calls. ChrisA

On Fri, Oct 12, 2012 at 12:36 AM, Oscar Benjamin wrote:

On 11 October 2012 14:18, Chris Angelico wrote:

...

Not propagating across function calls strikes me as messy, but I see why you'd want it. Would this be better as a __future__ directive? There's already the concept that they apply to a module but not to what that module calls.

__future__ directives are for situations in which the default behaviour will be changed in the future but you want to get the new behaviour now. The proposal is to always have widely supported, convenient ways to switch between different handling modes for numerical operations. The default Python behaviour would be unchanged by this.

Sure, it's not perfect for __future__ either, but it does seem odd for a function invocation to suddenly change semantics. This change "feels" to me more like a try/catch block - it's a change to this code that causes different behaviour around error conditions. That ought to continue into a called function. ChrisA

On 11 October 2012 15:55, Serhiy Storchaka wrote:

On 11.10.12 15:31, Sturla Molden wrote:

...

...

...

...

np.int64(1)/np.int64(0) 0

...

...

...

np.int32(1)/np.int32(0) 0

For such behavior must be some rationale.

I don't know what the rationale for that is but it is at least controllable in numpy:

...

...

import numpy as np np.seterr(all='raise') # Exceptions instead of mostly useless values {'over': 'raise', 'divide': 'raise', 'invalid': 'raise', 'under': 'raise'} np.int32(1) / np.int32(0) Traceback (most recent call last): File "<stdin>", line 1, in <module> FloatingPointError: divide by zero encountered in long_scalars np.float32(1e20) * np.float32(1e20) Traceback (most recent call last): File "<stdin>", line 1, in <module> FloatingPointError: overflow encountered in float_scalars np.float32('inf') inf np.float32('inf') / np.float32('inf') Traceback (most recent call last): File "<stdin>", line 1, in <module> FloatingPointError: invalid value encountered in float_scalars

Oscar

On 11 October 2012 17:05, Stephen J. Turnbull wrote:

Steven D'Aprano writes:

...

I would gladly give up a small amount of speed for better control over floats, such as whether 1/0.0 raised an exception or returned infinity.

Isn't that what the fpectl module is supposed to buy, albeit much less pleasantly than Decimal contexts do?

But the fpectl module IIUC wouldn't work for 1 / 0. Since Python has managed to unify integer/float division now it would be a shame to introduce any new reasons to bring in superfluous .0s again: with context(zero_division='infinity'): x = 1 / 0.0 # float('inf') y = 1 / 0 # I'd like to see float('inf') here as well I've spent 4 hours this week in computer labs with students using Python 2.7 as an introduction to scientific programming. A significant portion of that time was spent explaining the int/float division problem. They all get the issue now but not all of them understand that it is specifically about division: many are putting .0s everywhere. I expect it to be easier when we use Python 3 and I can simply explain that there are two types of division with two different operators. Oscar

On 11 October 2012 19:46, Guido van Rossum wrote:

On Thu, Oct 11, 2012 at 11:42 AM, Oscar Benjamin wrote:

...

I've spent 4 hours this week in computer labs with students using Python 2.7 as an introduction to scientific programming. A significant portion of that time was spent explaining the int/float division problem. They all get the issue now but not all of them understand that it is specifically about division: many are putting .0s everywhere. I expect it to be easier when we use Python 3 and I can simply explain that there are two types of division with two different operators.

You could have just told them to "from __future__ import division"

I know but the reason for choosing Python is the low barrier to getting started with procedural programming. When they're having trouble understanding the difference between the Python shell and the OS shell I'd like to avoid introducing the concept that the interpreter can change its calculation modes dynamically and forget those changes when you restart it. It's also unfortunate for the students to know that some of the things they're seeing on day one will change in the next version (you can't just tell people to import things from the "future" without some kind of explanation). I used the opportunity to think a little bit about types by running type(x) and explain that different types of objects behave differently. I would rather explain that using genuinely incompatible types like strings and numbers than ints and floats though. Oscar

On 12/10/12 03:05, Stephen J. Turnbull wrote:

Steven D'Aprano writes:

...

I would gladly give up a small amount of speed for better control over floats, such as whether 1/0.0 raised an exception or returned infinity.

Isn't that what the fpectl module is supposed to buy, albeit much less pleasantly than Decimal contexts do?

I can't test it, because I don't have that module installed, but I would think not. Reading the docs: http://docs.python.org/library/fpectl.html I would say that fpectl exists to turn on floating point exceptions where Python currently returns an inf or NaN, not to turn on special values where Python currently raises an exception, e.g. 1/0.0. Because it depends on a build-time option, using it is even less convenient that most other non-standard libraries. It only has a single exception type for any of Division by Zero, Overflow and Invalid, and doesn't appear to trap Underflow or Inexact at all. It's not just less pleasant than Decimal contexts, but much less powerful as well. -- Steven

On Thu, Oct 11, 2012 at 6:35 AM, Steven D'Aprano wrote:

On 11/10/12 16:45, Greg Ewing wrote:

...

Are you sure there would be any point in this? People who specifically *want* base-2 floats are probably quite happy with the current float type, and wouldn't appreciate having it slowed down, even by a small amount.

I would gladly give up a small amount of speed for better control over floats, such as whether 1/0.0 raised an exception or returned infinity.

Umm, you would be giving up a *lot* of speed. Native floating point happens right in the processor, so if you want special behavior, you'd have to take the floating point out of the CPU and into "user space". mark