[Python-Dev] Strategy for converting the decimal module to C

[Nick Maclaren]

James Y Knight <foom at fuhm.net> wrote:
> 
> > To cut a long story short, it is impractical for a language run-time
> > system to call user-defined handlers with any degree of reliability
> > unless the compiled code and run-time interoperate carefully - I have
> > been there and done that many times, but few people still working  
> > have.
> > On architectures with out-of-order execution (and interrupts), you
> > have to assume that an interrupt may occur anywhere, even when the
> > code does not use the relevant facility.  Floating-point overflow
> > in the middle of a list insertion?  That's to be expected.
> 
> While this _is_ a real problem, is it _not_ a general problem as you  
> are describing it. Processors are perfectly capable of generating  
> precise interrupts, and the inability to do so has nothing to do with  
> the out-of-order execution, etc. Almost all interrupts are precise.  

I am sorry, but this is almost totally wrong, though I agree that you
will get that impression upon reading the architecture books unless
you are very deeply into that area.

Let's skip the hardware issues, as they aren't what I am talking about
(though see [*]).  I am referring to the interaction between the
compiled code, deep library functions and run-time interrupt handler.

It is almost universal for some deep library functions and common for
compiled code to leave data structures inconsistent in a short window
that "cannot possibly fail" - indeed, most system interfaces do this
around system calls.  If an interrupt occurs then, the run-time system
will receive control with those data structures in a state where they
must not be accessed.  And it is fairly common for such data structures
to include ones critical to the functioning of the run-time system.

Now, it IS possible to write run-time systems that are safe against
this, and still allow asynchronous interrupts, but I am one of three
people in the world that I know have done it in the past two decades.
There may be as many as six, but I doubt more, and I know of no such
implementation on any Unix or Microsoft system.  It is even possible
to do this for compiled code, but that is where the coordination between
the compiler and run-time system comes in.

> The only interesting one which is not, on x86 processors, is the x87  
> floating point exception, ...

Er, no.  Try a machine-check in a TLB miss handler.  But it is all
pretty irrelevant, as the problem arises with asychronous exceptions
(e.g. timer interrupts, signals from other processes), anyway.

> Also, looking forward, the "simd" floating point instructions (ie mmx/ 
> sse/sse2/sse3) _do_ ...

The critical problems with the x87 floating-point exception were
resolved in the 80386.


[*]  Whether or not it is a fundamental problem, it is very much
a general problem at present, and it will become more so as more CPUs
implement micro-threading.  For why it is tied up with out-of-order
execution etc., consider a system with 100 operations flying, of which
10 are memory accesses, and then consider what happens when you have
combinations of floating-point exceptions, TLB misses, machine-checks
(e.g. ECC problems on memory) and device/timer interrupts.  Once you
add user-defined handlers into that mix, you either start exposing
that mess to the program or have to implement them by stopping the
CPU, unwinding the pipeline, and rerunning in very, very serial mode
until the handler is called.  Look at IA64 ....


Regards,
Nick Maclaren,
University of Cambridge Computing Service,
New Museums Site, Pembroke Street, Cambridge CB2 3QH, England.
Email:  nmm1 at cam.ac.uk
Tel.:  +44 1223 334761    Fax:  +44 1223 334679