[issue789290] Minor FP bug in object.c

[Mark Dickinson]

Mark Dickinson <dickinsm at gmail.com> added the comment:

I'll take this.

I think there *is* a problem here: on a system with 64-bit long, 
_Py_HashDouble can end up trying to cast the float value 2.**63 to a C 
long.  Since 2.**63 doesn't fit in a long, we get undefined behaviour, 
according to the C standards.

In more detail: when computing the hash of the floating-point value x = 
2.**63 on a 64-bit system, we end up in _Py_HashDouble in 
Objects/object.c.  The intpart of x will be 2.**63, and fractpart is 0.0.  
The comparison:

  if (intpart > LONG_MAX || -intpart > LONG_MAX) {

*fails*, because LONG_MAX (== 2**63-1) is implicitly converted to the 
float value 2.**63 before the comparison.  So we end up at the line:

  x = (long)intpart;

which attempts to convert intpart(== 2.0**63) to a long.  Even if this 
conversion doesn't signal, it could return some random long number, so 
unless we're very lucky, hash(2**63) == hash(2.**63) will fail.

On my system (OS X 10.5.4/x86_64), I *am* lucky:  when 2.**63 is cast to 
long I get LONG_MIN (!).  And it just so happens that hash(LONG_MIN) == 
hash(2**63), so we're okay.

----------
assignee: nobody -> marketdickinson
nosy: +marketdickinson
type:  -> behavior
versions: +Python 2.6, Python 2.7, Python 3.0, Python 3.1 -Python 2.5

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue789290>
_______________________________________