[Python-ideas] bitwise operations on bytes

[Eric Eisner]

Hello,

As previously mentioned on python-ideas [1] (circa 2006), it would
make sense to be able to perform bitwise operations on
bytes/bytearray.

Stealing the example from the original suggestion:

Suppose I have a string (say, read in from a binary file) that has a
4-byte field in it. Bit 11 indicates whether the accompanying data is
in glorped form (something I'm just making up for this example).

For example, if the field has 0x000480c4, the data is not glorped. If
the data is glorped, bit 11 would be on, i.e., 0x001480c4.

Let's say I want to turn on the glorp bit; what I have to do now:

GLORPED = 0x10
newchar = flags[1] | GLORPED
flags = flags[0] + newchar + flags[2:]

What I'd like to be able to do is something like:

GLORPED = b"\x00\x10\x00\x00"
flags |= GLORPED

# test if the glorped bit is on
any(flags & GLORPED)


I have run into this a few times, at least when reading/writing binary
formats etc. This approach is more intuitive than the typical/archaic
way of converting to an integer, performing a bitwise operation on the
integer, converting back to bytes. Arguably, bitwise operations on a
high-level integer type don't make sense, as base 2 is an
implementation detail.

At the very least, bytes and bytearray should be usable with the ~ ^ |
& operators

Example behavior:

>>> ~b'\x55\xff'
b'\xaa\x00'
>>> b'\xf5\x60' & b'\xa9\x3c'
b'\xa1\x20'

Unresolved problems:
If the two arguments are of different length, either it could either
raise a ValueError or mimic the behavior of ints.
Xoring an int to a byte seems less than well defined in general, due
to endianness ambiguity of the int and size ambiguity. I would think
this should not be allowed.
Also conceivable is using the shift operators >> and << on bytes, but
I personally would use that less often, and the result of such an
operation is ambiguous due to endianness.


-Eric Eisner

[1] http://mail.python.org/pipermail/python-ideas/2006-December/000001.html