int(long(-sys.maxint-1)) fails on Linux

[Chad Netzer]

Here I go, replying to my own posts again...

Chad Netzer wrote:

> C code:
>    if ((long)x < 0 && (sign > 0 || (x << 1) != 0))
>             goto overflow;
>     return (long)x * sign;
>
> not working:

deleted... It was actually working code, assembled with pgcc WITHOUT any optimization.
The bug I've describing only manifests itself when -O is selected (or even -O1 or -O2).
So I apologize if anyone looked at the code and couldn't find a problem; it didn't exist in
that version.

Here is some assembly of the same region, compiled with the -O1 flag, which DOES
exhibit the incorrect behavior.  Notice that it is quite different from the previous code,
and I think the bug is that it uses the "LEA" instruction improperly:

 <PyLong_AsLong+89>: test   %edx,%edx                            ; x < 0
 <PyLong_AsLong+91>: jge    0x806c22a <PyLong_AsLong+106>
 <PyLong_AsLong+93>: test   %ebx,%ebx                            ; sign > 0
 <PyLong_AsLong+95>: jg     0x806c231 <PyLong_AsLong+113> ; goto overflow
 <PyLong_AsLong+97>: lea    0x0(,%edx,2),%eax             ; ?? Where is (x << 1)?
 <PyLong_AsLong+104>: jne    0x806c231 <PyLong_AsLong+113> ; goto overflow
 <PyLong_AsLong+106>: mov    %edx,%eax
 <PyLong_AsLong+108>: imul   %ebx,%eax
 <PyLong_AsLong+111>: jmp    0x806c246 <PyLong_AsLong+134>

As you can see, the original code had add %eax, %eax instruction to shift
left by one, the optimized one doesn't.  I don't know what the heck it
was trying to do, and I don't know what condition codes "lea" sets, so
I can't know what it intended.  In any case, I speculate that line +104
was causing a jump to the overflow handler when it should not.

Anyway, I apologize for cluttering the newsgroup with assembly;  clearly the
problem is with pgcc w/ optimization turned on, which is a Mandrake 6.0 issue.

The lesson is NOT to trust pgcc with optimizations, and when compiling
Python 1.5.2, to turn them off (ie. make OPT=-g), at least when compiling
the longobject.c file.

Thanks again for all the help, it was at least a mildly interesting exercise to
track it down.  Sorry for the not double-checking the results of my last compilation
without optimizations.

Chad Netzer
chad at vision.arc.nasa.gov