[Python-ideas] Bitwise operations on bytes class

[Nathaniel McCallum]

On Mon, 2014-06-16 at 16:16 -0400, Nathaniel McCallum wrote:
> On Mon, 2014-06-16 at 21:55 +0200, Stefan Behnel wrote:
> > Nathaniel McCallum, 16.06.2014 21:43:
> > > Perhaps some code will clarify what I'm proposing. Attached is a class I
> > > have found continual reuse for over the last few years. It implements
> > > bitwise operators on a bytes subclass. Something similar could be done
> > > for bytearray.
> > 
> > Ok, according to your code, you don't want a SIMD type but rather an
> > arbitrary size integer type. Why don't you just use the "int" ("long" in
> > Py2) type for that? It has way faster operations than your multiple copy
> > implementation.
> 
> Of course my attached code is slow. This is precisely why I'm proposing
> native additions to the bytes class.
> 
> However, in most algorithms, there is a single operation like this on a
> block of data which is otherwise not treated as an integer. This
> operation often takes the form of something like:
> 
>   blocks.append(blocks[-1] ^ block)
> 
> In all the surrounding code, you are dealing with bytes *as* bytes.
> Converting into alternate types breaks up the readability of the
> algorithm. And given the security requirements of such algorithms,
> readability is extremely important.
> 
> The above code example has both simplicity and obviousness. Currently,
> in py3k, this is AFAICS the best alternative for readability:
> 
>  blocks.append([a ^ b for a, b in zip(blocks[-1], block)]
> 
> While this is infinitely better than Python 2.x, I think my proposal is
> still significantly more readable. When implemented natively, my
> proposal is also far more performant than this.

Also, when implemented on bytearray, you can get things like this:
  cksum ^= block.

This can be very fast as it can be done with no copies. It is also
extremely readable.

Nathaniel