[Tutor] bit shifting

[Peter Otten]

Ian D wrote:

> I am trying to follow some code. It is basically a python scratch
> interfacing script.
> Anyway part of the script has this code.
> Searching google for >> greater than signs in code with python has its
> issues.
> Can anyone clarify this stuff.
> I know its about 4 bytes of data. It looks like its setting all bits HIGH
> to me?
 
> n = len(cmd)
> a = array('c')
> a.append(chr((n>> 24) & 0xFF))
> a.append(chr((n>> 16) & 0xFF))
> a.append(chr((n>>  8) & 0xFF))
> a.append(chr(n & 0xFF))
[...]

> scratchSock.send(a.tostring() + cmd)

This is a way to serialize the integer n. The code assumes that it consists 
of 32 bits or 4 bytes.

Example: given a cmd that is 2271560481 bytes long the corresponding int n 
written in binary is

>>> n = 2271560481
>>> bin(n)
'0b10000111011001010100001100100001'

To extract the highest byte you shift-right 24 bits and get

>>> bin(n>>24)
'0b10000111'

In this case the mask 0xff or 0b11111111 has no effect

>>> bin(n>>24 & 0xff)
'0b10000111'

(or rather it silences the error that commands longer than 2**32 are not 
supported) but to get the second highest byte the mask is necessary as

>>> bin(n>>16)
'0b1000011101100101'

gives the 16 highest bits. Let's clip the high bits off:

>>> bin(0b1000011101100101
...   & 0b0000000011111111)
'0b1100101' # one leading 0-bit implied

Two more bytes, and we get the string "\x87eC":

>>> import array
>>> a = array.array("c")
>>> n = 0x87654321 # now you see how I picked the number for the example
>>> n
2271560481
>>> a.append(chr(n>>24 & 0xff))
>>> a.append(chr(n>>16 & 0xff))
>>> a.append(chr(n>>8 & 0xff))
>>> a.append(chr(n>>0 & 0xff)) # 'n >> 0' is the same as juse 'n'
>>> a.tostring()
'\x87eC!' # "\x86" is Python's way to to display chr(0x87)

To recognize the pattern underneath this odd assembly of bytes we can encode 
it:

>>> a.tostring().encode("hex")
'87654321'

So "e" is the same as "\x65", "C" is "\x43", "!" is "\x21".
By the way, there's a function in the library that does the above:

>>> import struct
>>> struct.pack("!I", 0x87654321)
'\x87eC!'

It also does the right thing when the number is too large, it raises an 
exception:

>>> struct.pack("!I", 0x987654321)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
struct.error: 'I' format requires 0 <= number <= 4294967295