[Python-ideas] String and bytes bitwise operations

[INADA Naoki]

Hi Terry,

Thanks, but I didn't care because my password is not so long.
I just want to illustrate real world bytes xor usage.
BTW, New MySQL auth methods (sha256 and caching_sha2)
use bytes xor too.

For performance point of view, websocket masking is performance
critical.  Tornado uses extension module only for it.  If bytearray ^= bytes
is supported, websocket frame masking may look like:

  frame ^= mask * ((len(frame)+3)//4)  # mask is 4 bytes long

On Sat, Jun 23, 2018 at 1:26 AM Terry Reedy <tjreedy at udel.edu> wrote:

> On 6/22/2018 7:08 AM, INADA Naoki wrote:
> > Bitwise xor is used for "masking" code like these:
> >
> >
> https://github.com/PyMySQL/PyMySQL/blob/37eba60439039eff17b32ef1a63b45c25ea28cec/pymysql/connections.py#L139-L146
>
> This points to a function _my_crypt that is O(n*n) because of using
> bytes.append.  Using bytearray.append makes it O(n).
> ---
> import random
> import struct
> import timeit
>
> range_type = range
>
> def _my_crypt(message1, message2):
>      length = len(message1)
>      result = b''
>      for i in range_type(length):
>          x = (struct.unpack('B', message1[i:i+1])[0] ^
>               struct.unpack('B', message2[i:i+1])[0])
>          result += struct.pack('B', x)
>      return result
>
> def _my_crypt2(message1, message2):
>      length = len(message1)
>      result = bytearray()
>      for i in range_type(length):
>          x = (struct.unpack('B', message1[i:i+1])[0] ^
>               struct.unpack('B', message2[i:i+1])[0])
>          result += struct.pack('B', x)
>      return bytes(result)
>
> def make(n):
>      result = bytearray()
>      for i in range(n):
>          result.append(random.randint(0, 255))
>      return result
>
> for m in (10, 100, 1000, 10_000, 100_000, 1000_000):
>      m1 = make(m)
>      m2 = make(m)
>
>      n = 1000_000 // m
>      print(f'bytes len {m}, timeit reps {n}')
>      print('old ', timeit.timeit('_my_crypt(m1, m2)', number = n,
> globals=globals()))
>      print('new ', timeit.timeit('_my_crypt2(m1, m2)', number = n,
> globals=globals()))
> ---
> prints
>
> bytes len 10, timeit reps 100000
> old  1.2277594129999998
> new  1.2174212309999999
> bytes len 100, timeit reps 10000
> old  1.145566423
> new  1.0924002120000003
> bytes len 1000, timeit reps 1000
> old  1.2860306190000002
> new  1.1168685839999999
> bytes len 10000, timeit reps 100
> old  1.6543344650000003
> new  1.118191714
> bytes len 100000, timeit reps 10
> old  4.2568492110000005
> new  1.1266137560000011
> bytes len 1000000, timeit reps 1
> old  60.651238144000004
> new  1.1315020199999992
>
> I tried to submit this to https://github.com/PyMySQL/PyMySQL/issues/new
> but [Submit] does not work for me.
>
> --
> Terry Jan Reedy
>
> _______________________________________________
> Python-ideas mailing list
> Python-ideas at python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
>


-- 
INADA Naoki  <songofacandy at gmail.com>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20180623/0afbb3f2/attachment.html>