[pypy-dev] Converting from Python strings to C strings slow?

[Martin C. Martin]

Maciek Fijalkowski wrote:
> Martin C. Martin wrote:
>> Hi,
>>
>> There seems to be a lot of overhead when passing a large string (23 
>> Meg) to C compiled RPython code.  For example, this code:
>>
>> def small(text):
>>     return 3
>>
>> t = Translation(small)
>> t.annotate()
>> t.rtype()
>> f3 = t.compile_c()
>>
>> st = time.time()
>> z = f3(xml)
>> print time.time() - st
>>
>>   
> This is wrong. You should even get a warning, the proper command is 
> t.annotate([str]).

Oops, yes, I've been working with variations of this all day, and I 
hadn't actually compiled & run the example in the email, although I'd 
done something equivalent.

> Besides, this is not the official way of writing rpython standalone 
> programs.

Thanks, but I'm not trying to write a standalone program, I need to call 
some 3rd party libraries.  For example, the string comes from one of a 
couple dozen of socket connections, managed by Twisted.  So I just want 
my inner loop in RPython.  The inner loop turns XML into a MySQL 
statement, which the main python program can then send to a database.

So I need to get a big string into RPython, and a smaller (but still 
pretty big) string out of it.

I see some other targets in there for shared libraries.  The docs 
mention that translate.py takes a --gc=generation argument, but when I 
try that I get:

$ python ./translator/goal/translate.py --gc=generation fun3.py
Usage
=====
   translate [options] [target] [target-specific-options]

translate: error: invalid value generation for option gc

Am I specifying it wrong?

Thanks,
Martin

> The official way is to go to pypy/translator/goal and for 
> example modify targetnopstandalone for a standalone target (the entry 
> function is entry_point).
> 
> You should translate this by ./translate.py targetnopstandalone.py (or 
> whatever target you choose). You can even use some fancy options (like 
> different gcs). In your example the xml was converted to python object, 
> which will never happen the official way.
> 
> Cheers,
> fijal
> 
> 
> :.
>