[Python-ideas] Adding bytes.frombuffer() constructor

INADA Naoki songofacandy at gmail.com
Fri Sep 16 06:41:25 EDT 2016


>>
>> The main question to be answered here would be whether adding a dedicated
>> spelling for "bytes(memoryview(bytearray)[:n])" actually smooths out the
>> learning curve for memoryview in general, where folks would learn:
>>
>> 1. "bytes(mybytearray[:n])" copies the data twice for no good reason
>> 2. "bytes.frombuffer(mybytearray, n)" avoids the double copy
>> 3. "bytes(memoryview(mybytearray)[:n])" generalises to arbitrary slices
>>
>> With memoryview being a builtin, I'm not sure that argument can be made
>> successfully - the transformation in going from step 1 direct to step 3 is
>> just "wrap the original object with memoryview before slicing to avoid the
>> double copy", and that's no more complicated than using a different
>> constructor method.
>
>
> I'm not sure, too.
> memoryview may and may not be bytes-like object which os.write or
> socket.send accepts.
>
> But memoryview is successor of buffer.  So we should encourage to
> use it for zero copy slicing.
>

Today, @socketpair found my bug relating to this.
https://github.com/python/asyncio/pull/395#r79136785

For example,

Bad (but works on CPython):

packet_bytes = bytes(memoryview(buff)[:packet_len])
del buff[:packet_len]

Good:

with memoryview(buff) as m:
    packet_bytes = bytes(m[:packet_len])
del buff[:packet_len]


There are two problem.

1. Avoiding bad code is difficult.  It works without any warning on CPython

2. Good code has significant overhead.  Slicing bytes from bytearray buffer is
    usually in low level code and performance may be important.

So I feel dedicated API for slicing bytes from bytes-like is worth enough.

Any thought about it?


More information about the Python-ideas mailing list