[Python-ideas] bitwise operations on bytes

[Tal Einat]

Eric Eisner wrote:
> Hello,
>
> As previously mentioned on python-ideas [1] (circa 2006), it would
> make sense to be able to perform bitwise operations on
> bytes/bytearray.
>
> Stealing the example from the original suggestion:
>
> Suppose I have a string (say, read in from a binary file) that has a
> 4-byte field in it. Bit 11 indicates whether the accompanying data is
> in glorped form (something I'm just making up for this example).
>
> For example, if the field has 0x000480c4, the data is not glorped. If
> the data is glorped, bit 11 would be on, i.e., 0x001480c4.
>
> Let's say I want to turn on the glorp bit; what I have to do now:
>
> GLORPED = 0x10
> newchar = flags[1] | GLORPED
> flags = flags[0] + newchar + flags[2:]
>
> What I'd like to be able to do is something like:
>
> GLORPED = b"\x00\x10\x00\x00"
> flags |= GLORPED

You can already do bitwise operations on bytes, and retrieving the
relevant byte and using single-byte operations on it is simple enough.
As I see it, the major underlying problem here is that byte-arrays are
immutable, since what you really want is to be able to change a single
byte of the array in-place. In general, Python's byte arrays and
strings are very ill suited for dealing with data which isn't
byte-based, and definitely horrible at building or modifying data
streams.

I recommend using an external library for working with data structures
and/or data streams. Construct[1] is my personal favorite, since it's
especially Pythonic and easy to use.

In any case, if you wish to propose a mutable byte/bit array which
supports array-wise binary operators, I can say I would certainly be
glad to have such a class at my disposal.

- Tal

[1] http://construct.wikispaces.com/