[Python-ideas] Re: Integer concatenation to byte string

We where no longer on the ideas list...

On 2 Mar 2021, at 13:04, Memz wrote:

There is no specific scenario it solves. The lack of efficiency of the timed code should speak for itself. Non-mutable bytes is a limit of python, since it's reliant on using function calls.

b"\x00\x00\x00\x00\x00" bytearray( b"\x00\x00\x00\x00\x00" ) struct.pack("iiiii",0,0,0,0,0) b"\x00\x00\x00\x00\x00" + bytes([1,0,0,0,0])

You mean the above is what you timed? It is not a realistic problem you are measuring. You also did not share how you measured the the code. If you are not experienced in benchmarking there are variables that you must control for to have meaningful results.

All function calls take time and resources, it would be impossible to streamline a function call to make it faster than building it in. This goes for most languages, including C.

All python byte code is interpreted by calling functions. They take time and resources. Barry

On Tue, Mar 2, 2021 at 3:29 AM Barry Scott mailto:barry@barrys-emacs.org> wrote:

...

On 2 Mar 2021, at 02:03, Memz mailto:mmax42852@gmail.com> wrote:

...

When I needed to do the creation of bytes objects from a mix of types the struct.pack() method has been the obvious way to go.

...

What is the use case that leads to needing the above?

...

Barry

The use of my suggestion is to reduce the reliance of function calls for bytes and bytearrays, which currently can only be done through function calls, and making it more efficient all-together. Here is a timed version of this: b-strings: 5-5.4 s bytearray() function call: 67.9 s struct.pack() 80.9 s b-string + bytes([1,...]) 54.3 s

What code are you benchmarking? What problem does that code solve?

My experience with creating byte objects is that struct is the fastest way to get the job done. But that could be becuase of the problems that I need bytes for. For example calling ioctl().

...

Moving bytearray to a non-function call would overhaul and optimize code that works with bytes, increase flexibility, and reduce reliance on imports, including struct.pack(). There is a lot of code that could, and should be using bytearray but can't, because other, more slow and painful than should be methods are more efficient because of the function call nature of bytearray.

You are assuming that the problem is the function calls. Surely its the algorithms that lead to issues in most code that is slow?

Barry