[Python-Dev] Numerical robustness, IEEE etc.

[Nick Maclaren]

Michael Hudson <mwh at python.net> wrote:
>
> This mail never appeared on python-dev as far as I can tell, so I'm  
> not snipping anything.

And it still hasn't :-(  I am on the list of recipients without posting
rights, and the moderator appears to be on holiday.

> >>> As I have posted to comp.lang.python, I am not happy with Python's
> >>> numerical robustness - because it basically propagates the  
> >>> 'features'
> >>> of IEEE 754 and (worse) C99.
> >>
> >> That's not really now I would describe the situation today.
> >
> > It is certainly the case in 2.4.2, however you would describe it.
> 
> I guess you could say it reflects the features of C89.  It certainly  
> doesn't do anything C99 specific.

Oh, yes, it does!  If you look at floatobject.c, you will find it solid
with constructions that make limited sense in C99 but next to no sense
in C89.

> But I wouldn't characterize anything Python does in the floating  
> point area as "designed", particularly.  Portability makes that hard.

Not really.  We managed even back in the 1970s, when there was a LOT
more variation.  Writing code that would work, unchanged, on an IBM 360,
an ICL 1900 and a CDC 6600 was, er, interesting ....

> Why are C99 and Java beyond redemption?  I know some of the mistakes  
> Java makes here, but still, you could at least hint at which you are  
> thinking of.

Where do I start?  Look at Kahan's "Why Java's Floating-Point Hurts
Everyone, Everywhere" and then at the UK's reasons for voting "NO"
to C99.  There wasn't even an agreement on the INTENT of most of the
new features in SC22WG14, and God alone knows what the C99 standard
means (if anything).  There is informative and optional text that
overrides normative; there is wording that contradicts itself, or is
otherwise meaningless; and so on.  Plus the fact that the C99 standard
is simultaneously unusable and unimplementable on many architectures.
And the fact that most of it is numerically insane.

I could go on ....

> Well, if you can't explain what your intentions are to *me*, as a  
> mathematics-degree holding core Python developer that has done at  
> least some work in this area, I posit that you aren't going to get  
> very far.    

My intentions are to provide some numerically robust semantics,
preferably of the form where straightforward numeric code (i.e. code
that doesn't play any bit-twiddling tricks) will never invoke
mathematically undefined behaviour without it being flagged.  See
Kahan on that.

> I'm not intimately familiar with the standards like 754 but I have  
> some idea what they contain, and I've read appendix F of C99, if that  
> helps you target your explanations.

Not a lot.  Annex F in itself is only numerically insane.  You need to
know the rest of the standard, including that which is documented only
in SC22WG14 messages, to realise the full horror.

> Why does it need to be program global?  In my not-really-thought-out  
> plans for straightening out CPython's floating point story I had  
> envisioned code to be written something like this:

No, you are thinking at too low a level.  The problem with such things
is that they related to the interfaces between types, and it is those
aspects where object-orientation falls down so badly.  For example,
consider conversion between float and long - which class should control
the semantics?

> This could be implemented by having a field in the threadstate of FPU  
> flags to check after each fp operation (or each set of fp operations,  
> possibly).  I don't think I have the guts to try to implement  
> anything sensible using HW traps (which are thread-local as well,  
> aren't they?).

Gods, NO!!!  Sorry, but I have implemented such things (but that was
on a far architecture, and besides the system is dead).  Modern CPU
architectures don't even DEFINE whether interrupt handling is local
to the core or chip, and document that only in the release notes,
but what is clear is that some BLACK incantations are needed in
either case.  Think of taking a machine check interrupt on a multi-
core, highly-pipelined architecture and blench.  And, if that is an
Itanic, gibber hysterically before taking early retirement on the
grounds of impending insanity.

Oh, that's the calm, moderate description.  The reality is worse.

> > Secondly, for things that don't need to be brings
> > up my point of adding methods to a built-in class.
> 
> This isn't very hard, really, in fact float has class methods in 2.5...

Thanks.  I will look, but remember this is being done at the C level.

> >> I'm interested in making Python's floating point story better, and
> >> have worked on a few things for Python 2.5 -- such as
> >> pickling/marshalling of special values -- but I'm not really a
> >> numerical programmer and don't like to guess what they need.
> >
> > Ah.  I must get a snapshot, then.  That was one of the lesser things
> > on my list.
> 
> It was fairly straightforward, and still caused portability problems...

Now, why did I predict that?  Did you, by any chance, include
System/390 and VAX support in your code :-)


Regards,
Nick Maclaren,
University of Cambridge Computing Service,
New Museums Site, Pembroke Street, Cambridge CB2 3QH, England.
Email:  nmm1 at cam.ac.uk
Tel.:  +44 1223 334761    Fax:  +44 1223 334679