On Tue, 15 Sep 2020 at 10:29, Stephen J. Turnbull

Paul Moore writes:

On Tue, 15 Sep 2020 at 08:12, Stephen J. Turnbull

wrote: Ben Rudiak-Gould writes:

1./0. is not a true infinity.

Granted, I was imprecise. To be precise, 1.0 / 0.0 *could* be a true infinity, and in some cases (1.0 / float(0)) *provably* is, while 1e1000 *provably* is not a true infinity.

I think we're getting to a point where the argument is getting way too theoretical to make sense any more. I'm genuinely not clear from the fragment quoted here,

1. What a "true infinity" is. Are we talking solely about IEEE-754?

Not solely. We're talking about mathematical infinities, such as the count of all integers or the length of the real line, and how they are modeled in IEEE-754 and Python.

OK, so to me, 1.0 / 0.0 *is* a "true infinity" in that sense. If you divide "one" by "zero" the only plausible interpretation - if you allow infinity in your number system - is that you get infinity as a result. Of course that just pushes the question back to whether you mean (mathematical) "one" and "zero" when you make that statement. I get that Ben is actually saying "things that round to 1" and "things that round to 0", but at that point error propagation and similar details come into play, making even equality a somewhat vague concept.

Because mathematically, different disciplines have different views, and many of them don't even include infinity in the set of numbers.

But in IEEE-754, inf behaves like the positive infinity of the extended real number line: inf + inf = inf, x > 0 => inf * x = inf, inf - inf = nan, and so on. We need to be talking about a mathematics that at least has defined the extended reals.

OK.

2. What do 1.0 and 0.0 mean? The literals in Python translate to specific bit patterns, so we can apply IEEE-754 rules to those bit patterns. There's nothing to discuss here, just the application of a particular set of rules.

Except that application is not consistent. That was the point of Ben's post.

(Unless we're discussing which set of rules to apply, but I thought IEEE-754 was assumed). So Ben's statement seems to imply he's not talking just about IEEE-754 bit patterns.

He's talking about the arithmetic of floating point numbers, which involves rounding rules intended to model the extended real line.

Again OK, but in the face of rounding rules, many "obvious" mathematical properties, such as associativity and distributivity, don't hold. So we have to be *very* careful when inferring equivalences like the one you use below, a * b = c => 1.0 / (a * b) = 1.0 / c. Note that I'm not saying here that Python's behaviour might not be inconsistent, but rather that it's not obvious (to me, at least) that there is an intuitively consistent set of rules - so Python has chosen one particular way of being inconsistent, and the question is more about whether we like an alternative form of inconsistency better than about "should Python be consistent". If a consistent set of rules *is* possible then "we should be consistent" is a purity argument that may or may not stand against practical considerations like "raising an exception is more user friendly for people who don't understand IEEE-754".

3. Can you give an example of 1.0/0.0 *not* being a "true infinity"? I have a feeling you're going to point to denormalised numbers close to zero, but why are you expressing them as "0.0"?

Not subnormal numbers, but non-zero numbers that round to zero.

1e200 1e200 1e-200 1e-200 1e200 == 1.0 / 1e-200 True 1e-200 * 1e-200 == 0.0 True 1.0 / (1e-200 * 1e-200) Traceback (most recent call last): File "<stdin>", line 1, in <module> ZeroDivisionError: float division by zero 1e200 * 1e200 inf

See above.

4. Can you provide the proofs you claim exist?

Proofs of what? All of this is just straightforward calculations in the extended real line, on the one hand, in IEEE 754 floats on the other hand, and in Python, on the gripping hand.[1]

You claimed that "1e1000 *provably* is not a true infinity". How would you prove that? Typing it into Python gives

1e1000 inf

That's a proof that in Python, 1e1000 *is* a true infinity. On the extended real line, 1e1000 is not a true infinity because they are different points on the line. In IEEE 754 I'd have to check the details, but I think someone posted that the standard defines parsing rules such that 1e1000 must translate to infinity. So when I asked for your proof, I was trying to determine which set of rules you were using.

For numbers with magnitudes "near" 1 (ie, less than about 100 orders of magnitude bigger or smaller), the three arithmetics agree tolerably well for practical purposes (and Python and IEEE 754 agree exactly). If you need trignometric or exponential functions, things get quite a bit more restricted -- things break down at pi/2.

For numbers with magnitudes "near" infinity, or nearly infinitesimal, things break down in Python. Whether you consider that breakdown to be problematic or not depends on how you view the (long) list of Curious Things The Dog Did (Not Do) In The Night in Ben's post.

Precisely. There are three models here, Python's, IEEE-754's and the extended real line. All have different rules. Python's rules have been established organically over time, possibly based on judgement calls about what is "useful behaviour", possibly by accident.

I think they're pretty disastrous. I'd be much happier with *either* consistently raising on non-representable finite results, or consistently returning -inf, 0.0, inf, or nan. That's not the Python we have.

I think we could have rules that are easier to reason with, but I doubt anyone but specialists are ever likely to care much. Specialists are an important group of users, so I'm not dismissing the issue, but specialists are unlikely to be the people who want inf to be a builtin rather than a name in the math module, so what specialists prefer isn't particularly relevant to the *original* request in this thread. Having said that, do I care about making better rules for how Python works? Well, sort of. I don't do higher maths in Python that often - most of my work is basically finite. I use floating point mostly for probability calculations and plotting. What's bothering me here is that Ben's main list is all related to OverflowError and ValueError exceptions, and I'm fine with debating the details of those. I personally don't care much, and it should be a different thread than the "add inf to builtins" proposal, but if the specialists want something different, that's fine with me. But then things spilled over to 1.0 / 0.0, and there I *do* care. Division is a fundamental operation, divide by zero is nearly always an error in non-specialist code, and non-specialists really don't want to see infinities cropping up in their calculations. So by all means discuss math.lgamma. But be very careful translating any intuitions you have over that across to division. Here's a question. 1.0 / 2.0 is 0.5. No question there. 1 / 2 is *also* 0.5. So integer division can give float values. And should give "the same result" (mathematically). 1.0 / 0.0 is being debated, but let's say we agree that it should produce inf, because that's IEEE-754. Now, what about 1 / 0? Do we violate the principle that led to 1 / 2 being 0.5? Or do we allow this to generate a floating infinity? 1 // 0 has to give ZeroDivisionError, as there's no possible *integer* result. But now I find myself uncomfortable that 1//0 raises an exception but 1/0 doesn't. And as soon as we start considering integer division, we're talking about breaking a *vast* amount of code. As I said above, pick your preferred form of inconsistency. But don't expect to get perfect consistency, it's not an option.

(Note: this has drifted a long way from anything that has real relevance to Python - arguments this theoretical will almost certainly fall foul of "practicality beats purity" when we get to arguing what the language should do. I'm just curious to see how the theoretical debate pans out).

I disagree. This is entirely on the practical side of things!

Calculations on the extended real line are "just calculations", as are calculations with IEEE 754 floats. Any attentive high school student can learn to do them (at least with the help of a computer for correct rounding in the case of IEEE 754 floats :-). There's no theory to discuss (except for terminology like "true infinity").

But extended reals are only one thing you might be trying to model with Python's floating point. Non-specialists are more likely to have an (imperfect) mental model of a calculator. Which typically errors on divide by zero.

The question of whether Python diverges from IEEE 754 (which turns on whether IEEE 754 recommends that you raise exceptions or return inf/nan but not both), and the question of whether the IEEE 754 model of "overflow as infinity" is intuitive/useful to users of Python who *don't* use NumPy, are the questions under discussion here.

Python floats exist in a context where they need to interact "consistently" with other types. So there's a tension between precisely modeling IEEE-754 and fitting with the rest of the language. In general, I'm in favour of floats accurately following IEEE-754, but we need to be careful not to be too dogmatic at the edge cases. Also, we need to remember that IEEE-754 might define rules for how division works, but it doesn't (can't) mandate how Python spells that operation, so in theory, we could have math.iee743_div for "a strictly conforming" version. And once we start thinking about that it becomes clear that "which set of rules gets to be called /" is a matter of user friendliness as much as "correctness".

Now, to be fair, there is a practicality beats purity argument. It can be argued (I think Ben or maybe Chris A alluded to this) that "overflows usually' just happen but division by zero usually' is a logic error". Therefore returning inf in the case of overflow (and continuing the computation), and raising on zero division (stopping the computation and getting the attention of the user), is the pragmatic sweet spot. I disagree, but it's a viable argument.

That's probably my main point, coupled with the points that "division, as a language built in operator, is very different from functions in the math module", and "division is polymorphic, and consistent behaviour across types is an extra consideration that math functions don't need to be concerned about".

Footnotes: [1] If you haven't read The Mote in God's Eye, that's the third hand of the aliens, which is big, powerful, clumsy, and potentially deadly. Pretty much the way I think of "inf"!

Yep, spot on! Paul