[Python-Dev] Numerical robustness, IEEE etc.

[Michael Hudson]

Nick Maclaren <nmm1 at cus.cam.ac.uk> writes:

> Michael Hudson <mwh at python.net> wrote:
>>
>> This mail never appeared on python-dev as far as I can tell, so I'm  
>> not snipping anything.
>
> And it still hasn't :-(  I am on the list of recipients without posting
> rights, and the moderator appears to be on holiday.

They've appeared now, it seems the moderator is back :-)

>> But I wouldn't characterize anything Python does in the floating  
>> point area as "designed", particularly.  Portability makes that hard.
>
> Not really.  We managed even back in the 1970s, when there was a LOT
> more variation.  Writing code that would work, unchanged, on an IBM 360,
> an ICL 1900 and a CDC 6600 was, er, interesting ....

Maybe append " for me, at least" to what I wrote then.  But really, it
is hard: because Python runs on so many platforms, and platforms that
no current Python developer has access to.  If you're talking about
implementing FP in software (are you?), then I guess it gets easier.

>> Well, if you can't explain what your intentions are to *me*, as a  
>> mathematics-degree holding core Python developer that has done at  
>> least some work in this area, I posit that you aren't going to get  
>> very far.    
>
> My intentions are to provide some numerically robust semantics,
> preferably of the form where straightforward numeric code (i.e. code
> that doesn't play any bit-twiddling tricks) will never invoke
> mathematically undefined behaviour without it being flagged.  See
> Kahan on that.

That doesn't actually explain the details of your intent very much.

>> I'm not intimately familiar with the standards like 754 but I have  
>> some idea what they contain, and I've read appendix F of C99, if that  
>> helps you target your explanations.
>
> Not a lot.  Annex F in itself is only numerically insane.  You need to
> know the rest of the standard, including that which is documented only
> in SC22WG14 messages, to realise the full horror.

That's not why I was mentioning it.  I was mentioning it to give the
idea that I'm not a numerical expert but, for example, I know what a
denorm is.

>> Why does it need to be program global?  In my not-really-thought-out  
>> plans for straightening out CPython's floating point story I had  
>> envisioned code to be written something like this:
> 
> No, you are thinking at too low a level.

Well, I was just trying to get you to actually explain your intent :-)

> The problem with such things is that they related to the interfaces
> between types, and it is those aspects where object-orientation
> falls down so badly.  For example, consider conversion between float
> and long - which class should control the semantics?

This comment seems not to relate to anything I said, or at least not
obviously.

>> This could be implemented by having a field in the threadstate of FPU  
>> flags to check after each fp operation (or each set of fp operations,  
>> possibly).  I don't think I have the guts to try to implement  
>> anything sensible using HW traps (which are thread-local as well,  
>> aren't they?).
>
> Gods, NO!!!

Good :-)

> Sorry, but I have implemented such things (but that was on a far
> architecture, and besides the system is dead).  Modern CPU
> architectures don't even DEFINE whether interrupt handling is local
> to the core or chip, and document that only in the release notes,
> but what is clear is that some BLACK incantations are needed in
> either case.

Well, I only really know about the PowerPC at this level...

> Think of taking a machine check interrupt on a multi- core,
> highly-pipelined architecture and blench.  And, if that is an
> Itanic, gibber hysterically before taking early retirement on the
> grounds of impending insanity.

What does a machine check interrupt have to do with anything?

> Oh, that's the calm, moderate description.  The reality is worse.

Yes, but fortunately irrelevant...

>> > Secondly, for things that don't need to be brings
>> > up my point of adding methods to a built-in class.
>> 
>> This isn't very hard, really, in fact float has class methods in 2.5...
>
> Thanks.  I will look, but remember this is being done at the C level.

So is my code.

>> >> I'm interested in making Python's floating point story better, and
>> >> have worked on a few things for Python 2.5 -- such as
>> >> pickling/marshalling of special values -- but I'm not really a
>> >> numerical programmer and don't like to guess what they need.
>> >
>> > Ah.  I must get a snapshot, then.  That was one of the lesser things
>> > on my list.
>> 
>> It was fairly straightforward, and still caused portability problems...
>
> Now, why did I predict that?  Did you, by any chance, include
> System/390 and VAX support in your code :-)

God no, it was just mundane stuff like SIZEOF_FLOAT not being defined
on windows.

Now, a more general reply: what are you actually trying to acheive
with these posts?  I presume it's more than just make wild claims
about how much more you know about numerical programming than anyone
else...

I get the impression that you would like to see floatobject.c
rewritten to make little or no use of the FPU, is that right?  Also,
you seem to have an alternate model for floating point calculations in
mind, but seem very reluctant to actually explain what this is.

I think you should probably write a PEP.

Cheers,
mwh

-- 
  GET   *BONK*
  BACK  *BONK*
  IN    *BONK*
  THERE *BONK*             -- Naich using the troll hammer in cam.misc