[Python-ideas] String and bytes bitwise operations

[Terry Reedy]

On 6/22/2018 7:08 AM, INADA Naoki wrote:
> Bitwise xor is used for "masking" code like these:
> 
> https://github.com/PyMySQL/PyMySQL/blob/37eba60439039eff17b32ef1a63b45c25ea28cec/pymysql/connections.py#L139-L146

This points to a function _my_crypt that is O(n*n) because of using 
bytes.append.  Using bytearray.append makes it O(n).
---
import random
import struct
import timeit

range_type = range

def _my_crypt(message1, message2):
     length = len(message1)
     result = b''
     for i in range_type(length):
         x = (struct.unpack('B', message1[i:i+1])[0] ^
              struct.unpack('B', message2[i:i+1])[0])
         result += struct.pack('B', x)
     return result

def _my_crypt2(message1, message2):
     length = len(message1)
     result = bytearray()
     for i in range_type(length):
         x = (struct.unpack('B', message1[i:i+1])[0] ^
              struct.unpack('B', message2[i:i+1])[0])
         result += struct.pack('B', x)
     return bytes(result)

def make(n):
     result = bytearray()
     for i in range(n):
         result.append(random.randint(0, 255))
     return result

for m in (10, 100, 1000, 10_000, 100_000, 1000_000):
     m1 = make(m)
     m2 = make(m)

     n = 1000_000 // m
     print(f'bytes len {m}, timeit reps {n}')
     print('old ', timeit.timeit('_my_crypt(m1, m2)', number = n, 
globals=globals()))
     print('new ', timeit.timeit('_my_crypt2(m1, m2)', number = n, 
globals=globals()))
---
prints

bytes len 10, timeit reps 100000
old  1.2277594129999998
new  1.2174212309999999
bytes len 100, timeit reps 10000
old  1.145566423
new  1.0924002120000003
bytes len 1000, timeit reps 1000
old  1.2860306190000002
new  1.1168685839999999
bytes len 10000, timeit reps 100
old  1.6543344650000003
new  1.118191714
bytes len 100000, timeit reps 10
old  4.2568492110000005
new  1.1266137560000011
bytes len 1000000, timeit reps 1
old  60.651238144000004
new  1.1315020199999992

I tried to submit this to https://github.com/PyMySQL/PyMySQL/issues/new
but [Submit] does not work for me.

-- 
Terry Jan Reedy