[pypy-dev] Speeding up zlib in standard library

[Peter Cock]

On Mon, Mar 19, 2012 at 6:36 PM, Maciej Fijalkowski <fijall at gmail.com> wrote:
>> http://mail.python.org/mailman/listinfo/pypy-dev
>
> append_charpsize is special - it's not the *actual* implementation,
> the actual implementation is buried somewhere in
> rpython/lltypesystem/rbuilder.py, with the one you're mentioning being
> just fake implementation for tests. StringBuilder is special in a
> sense that it has some special GC support (which we can probably
> improve upon).
>
> Cheers,
> fijal

I guess you are referring to the copy_string_contents function here:
https://bitbucket.org/pypy/pypy/src/default/pypy/rpython/lltypesystem/rstr.py

However, methods ll_append_multiple_char not ll_append_charpsize
defined in rbuilder seem to use this - they both use a for loop char-by-char,
https://bitbucket.org/pypy/pypy/src/default/pypy/rpython/lltypesystem/rbuilder.py

My hunch would be to replace this:

    @staticmethod
    def ll_append_charpsize(ll_builder, charp, size):
        used = ll_builder.used
        if used + size > ll_builder.allocated:
            ll_builder.grow(ll_builder, size)
        for i in xrange(size):
            ll_builder.buf.chars[used] = charp[i]
            used += 1
        ll_builder.used = used

with this:

    @staticmethod
    def ll_append_charpsize(ll_builder, charp, size):
        used = ll_builder.used
        if used + size > ll_builder.allocated:
            ll_builder.grow(ll_builder, size)
        assert size >= 0
        ll_str.copy_contents(charp, ll_builder.buf, 0, used, size)
        ll_builder.used += size

(and similarly for ll_append_multiple_char above it)

Like an onion - more and more layers ;) I'm beginning to suspect
speeding up append_charpsize in order to make passing strings
to/from C code faster is a bit too ambitious for a first contribution
to PyPy! [*]

Peter

[*] Especially as after three hours it is still building from source:
$ python translate.py --opt=jit targetpypystandalone.py