I would gladly give up a small amount of speed for better control over floats, such as whether 1/0.0 raised an exception or returned infinity.
Umm, you would be giving up a *lot* of speed. Native floating point happens right in the processor, so if you want special behavior, you'd have to take the floating point out of hardware and into "user space".
Even in user-space, you're not giving up that much speed in practical terms, at least not for my needs. The new decimal module in Python 3.3 is less than a factor of 10 times slower than Python's floats, which makes it pretty much instantaneous to my mind :)
Hmm, well, if it's only that much slower, then we should implement Rationals and get rid of the issue altogether.