<br><br><div class="gmail_quote">On Mon, Mar 19, 2012 at 2:32 PM, Maciej Fijalkowski <span dir="ltr"><<a href="mailto:fijall@gmail.com">fijall@gmail.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div class="HOEnZb"><div class="h5">On Mon, Mar 19, 2012 at 9:15 PM, Peter Cock <<a href="mailto:p.j.a.cock@googlemail.com">p.j.a.cock@googlemail.com</a>> wrote:<br>
> On Mon, Mar 19, 2012 at 6:36 PM, Maciej Fijalkowski <<a href="mailto:fijall@gmail.com">fijall@gmail.com</a>> wrote:<br>
>>> <a href="http://mail.python.org/mailman/listinfo/pypy-dev" target="_blank">http://mail.python.org/mailman/listinfo/pypy-dev</a><br>
>><br>
>> append_charpsize is special - it's not the *actual* implementation,<br>
>> the actual implementation is buried somewhere in<br>
>> rpython/lltypesystem/rbuilder.py, with the one you're mentioning being<br>
>> just fake implementation for tests. StringBuilder is special in a<br>
>> sense that it has some special GC support (which we can probably<br>
>> improve upon).<br>
>><br>
>> Cheers,<br>
>> fijal<br>
><br>
> I guess you are referring to the copy_string_contents function here:<br>
> <a href="https://bitbucket.org/pypy/pypy/src/default/pypy/rpython/lltypesystem/rstr.py" target="_blank">https://bitbucket.org/pypy/pypy/src/default/pypy/rpython/lltypesystem/rstr.py</a><br>
><br>
> However, methods ll_append_multiple_char not ll_append_charpsize<br>
> defined in rbuilder seem to use this - they both use a for loop char-by-char,<br>
> <a href="https://bitbucket.org/pypy/pypy/src/default/pypy/rpython/lltypesystem/rbuilder.py" target="_blank">https://bitbucket.org/pypy/pypy/src/default/pypy/rpython/lltypesystem/rbuilder.py</a><br>
><br>
> My hunch would be to replace this:<br>
><br>
> @staticmethod<br>
> def ll_append_charpsize(ll_builder, charp, size):<br>
> used = ll_builder.used<br>
> if used + size > ll_builder.allocated:<br>
> ll_builder.grow(ll_builder, size)<br>
> for i in xrange(size):<br>
> ll_builder.buf.chars[used] = charp[i]<br>
> used += 1<br>
> ll_builder.used = used<br>
><br>
> with this:<br>
><br>
> @staticmethod<br>
> def ll_append_charpsize(ll_builder, charp, size):<br>
> used = ll_builder.used<br>
> if used + size > ll_builder.allocated:<br>
> ll_builder.grow(ll_builder, size)<br>
> assert size >= 0<br>
> ll_str.copy_contents(charp, ll_builder.buf, 0, used, size)<br>
> ll_builder.used += size<br>
><br>
> (and similarly for ll_append_multiple_char above it)<br>
><br>
> Like an onion - more and more layers ;) I'm beginning to suspect<br>
> speeding up append_charpsize in order to make passing strings<br>
> to/from C code faster is a bit too ambitious for a first contribution<br>
> to PyPy! [*]<br>
><br>
> Peter<br>
><br>
> [*] Especially as after three hours it is still building from source:<br>
> $ python translate.py --opt=jit targetpypystandalone.py<br>
<br>
</div></div>ok, so let me reply a bit more :)<br>
<br>
First of all, you don't have to translate pypy to see changes. We<br>
mostly run tests to see if they work. You can also write a very small<br>
rpython program in translator/goal (look at targetnopstandalone.py) if<br>
you want to just test the performance of single function.<br>
<br>
I suppose your code is indeed a bit faster, but my bet would be it's<br>
not too much faster (feel free to prove me wrong, especially on older<br>
GCCs, they might not figure out that a loop is vectorizable for<br>
example).<br>
<br>
The main source of why passing strings to C is slow is however copying<br>
the string from the GC area to non-moving one, raw malloced in C.<br>
There are various strategies how to approach this, one of those would<br>
be pinning, so the GC structures don't move and you can pass a pointer<br>
to C. This is however definitely not a good first patch to pypy ;-)<br>
<br>
What I would suggest:<br>
<br>
* Your patch looks good to me, although I'm not sure if<br>
copy-string-contents would accept a raw memory. Check if tests pass.<br>
* If you want to benchmark, write a small test for passing such<br>
strings in translator/goal and see if it works.<br>
<br>
We're usually available for help on IRC and thanks for tackling this problem!<br>
<br>
Cheers,<br>
fijal<br>
<div class="HOEnZb"><div class="h5">_______________________________________________<br>
pypy-dev mailing list<br>
<a href="mailto:pypy-dev@python.org">pypy-dev@python.org</a><br>
<a href="http://mail.python.org/mailman/listinfo/pypy-dev" target="_blank">http://mail.python.org/mailman/listinfo/pypy-dev</a><br>
</div></div></blockquote></div><br>FWIW, copy_string_contents definitely doesn't take raw memory, it takes an rstr.<div><br></div><div>Alex<br clear="all"><div><br></div>-- <br>"I disapprove of what you say, but I will defend to the death your right to say it." -- Evelyn Beatrice Hall (summarizing Voltaire)<br>
"The people's good is the highest law." -- Cicero<br><br>
</div>