[portland] Raw buffers in python

[Michael Schurter]

On Mon, Oct 31, 2011 at 4:30 PM, Benjamin van der Veen
<b at bvanderveen.com> wrote:
> Hey all,
>
> What's the best way to make a raw buffer out of a string in Python? They
> key requirement is that inspecting a range of characters in the resulting
> object does not cause a copy/allocation. Correct me if I'm wrong, but it
> seems like sequence types (produced by memoryview, bytearray, etc) all
> return a character or integer object when you access items by index.
>
> An example usecase is a parser—you have a string buffer the user gave you,
> and you're iterating over each character. It would be nice not to allocate
> a heap object for every iteration.

If I remember correctly, instances of all single byte ints (0 <= x <
256) are created automatically, so iterating over a bytearray may
avoid new object creation.

> Hm, although, now that I'm typing this up, I'm realizing it might be a bit
> ridiculous to be asking for a raw char or int in Python. :S

Yeah, everything is a heap allocated PyObject in CPython. Check out
Cython for writing Python that compiles to C and can use raw C types
(bytes & chars) without much effort. Depending on your use case and
dependencies, PyPy might be worth checking out. Doing lots of
iterations over homogeneous data types is where JITs shine.