[Python-Dev] Size of int across machines (was RE: Blowfish in Python?)

[Tim Peters]

[Markus Stenberg]
> ...
>  speed was horrendous. >
> I think the main reason was the fact that I had to use _long ints_ for
> calculations, as the normal ints are signed, and apparently the bitwise
> operators do not work as advertised when bit32 is set (=number is
> negative).

[Tim, takes "bitwise operators" to mean & | ^ ~, and expresses surprise]

[Markus, takes umbrage, and expresses umbrage <wink>]
> Hmm.. As far as I'm concerned, shifts for example do screw up.

Do you mean "for example" as in "there are so many let's just pick one at
random", or as in "this is the only one I've stumbled into" <0.9 wink>?

> i.e.
>
> 0xffffffff >> 30
>
> [64bit Python: 3]
> [32bit Python: -1]
>
> As far as I'm concerned, that should _not_ happen. Or maybe it's just me.

I could not have guessed that your complaint was about 64-bit Python from
your "when bit32 is set (=number is negative)" description <wink>.

The behavior shown in a Python compiled under a C in which sizeof(long)==4
matches the Reference Manual (see the "Integer and long integer literals"
and "shifting operations" sections).  So that can't be considered broken
(you may not *like* it, but it's functioning as designed & as documented).

The behavior under a sizeof(long)==8 C seems more of an ill-documented (and
debatable to me too) feature.  The possibility is mentioned in the "The
standard type hierarchy" section (under Numbers -> Integers -> Plain
integers) but really not fleshed out, and the "Integer and long integer
literals" section plainly contradicts it.

Python's going to have to clean up its act here -- 64-bit machines are
getting more common.  There's a move afoot to erase the distinction between
Python ints and longs (in the sense of auto-converting from one to the other
under the covers, as needed).  In that world, your example would work like
the "64bit Python" one.  There are certainly compatability issues, though,
in that int left shifts are end-off now, and on a 32-bit machine any int for
which i & 0x8000000 is true "is negative" (and so sign-extends on a right
shift; note that Python guarantees sign-extending right shifts *regardless*
of what the platform C does (C doesn't define what happens here -- Python
does)).

[description of pain getting a fast C-like "mod 2**32 int +" to work too]

Python really wasn't designed for high-performance bit-fiddling, so you're
(as you've discovered <wink>) swimming upstream with every stroke.  Given
that you can't write a C module here, there's nothing better than to do the
^ & | ~ parts with ints, and fake the rest slowly & painfully.  Note that
you can at least determine the size of a Python int via inspecting
sys.maxint.

sympathetically-unhelpfully y'rs  - tim