C extension + libm oddity [fmod(2.0, 2.0) == nan ?!]
Sat Nov 5 02:13:26 CET 2005
"Lonnie Princehouse" <finite.automaton at gmail.com> writes:
> Now, fmod(2.0, 2.0) should be 0.0. The problem? ans is getting
> assigned nan! I have stepped through it in the debugger now dozens of
> times. Either fmod is putting the wrong return value on the stack, or
> the stack is getting corrupted by something else and "ans" is getting
> assigned the wrong value.
Have you compiled the C extension with all optimization turned off?
Especially with optimizations on, you can't really tell what is going
to get assigned in a variable because the code isn't computing the
intermediate values you might expect it to. I suggest disassembling
it and stepping through it instruction by instruction if you haven't
More information about the Python-list