[Python-Dev] Deprecation warning on integer shifts and such

Guido van Rossum guido@python.org
Sun, 11 Aug 2002 20:25:27 -0400

> Jack Jansen wrote:
> > As of recently I'm getting deprecation warnings on lots of constructs of 
> > the form "0xff << 24", telling me that in Python 2.4 this will return a 
> > long.
> Interesting. I wonder why the implementation warns about 0xff << 24...
> 0xff000000 fits nicely into a 32-bit integer. I don't see why the
> "changing sign" is relevant here or even why it is mentioned in the
> warning since the PEP doesn't say anything about it.

PEP 237 *tries* mention it:

    - Currently, x<<n can lose bits for short ints.  This will be
      changed to return a long int containing all the shifted-out
      bits, if returning a short int would lose bits.

Maybe you don't see changing sign as "losing bits"?  I do!  Maybe I
have to clarify this in the PEP.

PEP 237 is about erasing all differences between int and long.  When
seen as a long, 0xff000000 has the value 4278190080.  But currently it
is an int and has the value -16777216.  As a bit pattern that doesn't
make much of a difference, but as a numeric value it makes a huge
difference (2**32 to be exact :-).  So in Python 2.4, 0xff<<24, as
well as the constant 0xff000000, will have the value 4278190080.

Note that larger constants are already longs in 2.2: e.g. 0x100000000
equals 4294967296 (which happens to be representable only as a long).
It's the oct and hex constants in range(2**31, 2**32) that currently
behave anomalously, returning negative numbers despite looking like
positive numbers (to everyone except people whose minds have been
exposed to 32-bit bit-fiddling too long :-).

> Changing these semantics would cause compatibility problems for
> applications doing low-level bit manipulations or ones which use
> the Python integer type to store unsigned integer values, e.g.
> for use as bitmapped flags.

That's why I'm adding the warnings to 2.3.  Note that the bit pattern
in the lower 32 bits will remain the same; it's just the
interpretation of the sign that will change.

> > As these things are bitpatterns (they're all generated from .h
> > files for system call interfaces and such) that the user will pass
> > to methods that wrap underlying API calls I don't want them to be
> > longs. How do I force them to remain ints?

Why do you want them to remain ints?  Does a long whose lower 32 bits
have the right bit pattern not work?

If you really want the int value, you have to do a little arithmetic.
Here's something that's independent of the Python version and won't
issue a warning:

def toint32(x):
    x = x & 0xffffffffL # Force it to be a long in range(0, 2**32)
    if x & 0x80000000L: # If sign bit set
        x -= 0x100000000L # flip sign
    return int(x)

You can also write it as a single expression:

def toint32(x):
    return int((x & 0xffffffffL) - ((x & 0x80000000L) << 1))

In the long run you'll thank me for this.

--Guido van Rossum (home page: http://www.python.org/~guido/)