[pypy-dev] work on libpypy.so

[Maciej Fijalkowski]

On Fri, Aug 19, 2011 at 12:54 PM, Roberto De Ioris <roberto at unbit.it> wrote:
>
> Il giorno 25/lug/2011, alle ore 14:31, Armin Rigo ha scritto:
>
>>
>> Indeed, there is nothing like that so far.  We need to think out the
>> kind of C-level API which makes sense for libpypy.so to expose.  There
>> are two options: either we tweak a little bit cpyext to expose the
>> CPython C API, or we design some higher-performance PyPy-only C API...
>>
>> For the latter case the first issue is how to store a reference to a
>> PyPy object in C, given that it moves around --- maybe with an
>> indirection, which would be a bit similar to what cpyext does but can
>> be done with much better performance if we are free to design the API.
>>
>> Feel free to come to our IRC channel to discuss things (#pypy on
>> irc.freenode.net).
>>
>>
>
> Sorry for the big late/silence but i spent last days working/hacking on cpyext.
>
> I have managed to have a working uWSGI+pypy inverting the initial concept
> (not very useful, but this is a base for benchmark) :
>
> 1) i build the whole uWSGI stack core (so, without cpython plugin) as a shared library
> 2) i created a uwsgi_pypy c extension:
>
> http://projects.unbit.it/hg/uwsgi_pypy/file/7bbae6efc126/uwsgi_pypy.c
>
> This extension export a bunch of functions, with the only useful one being uwsgi_pypy.init()
> This function calls the original main() of the uWSGI server, so effectively pypy lose the process control.
>
> 3) i have developed a very raw/minimal plugin for uWSGI that inspect a symbol in the process space.
>
> http://projects.unbit.it/uwsgi/browser/plugins/symcall/symcall_plugin.c
>
> This symbol is defined in the uwsgi_pypy.c module. So, this is the way uWSGI give control back to pypy at each request.
>
> This is an example WSGI-handler implementation (incomplete, it misses post handling, but can already execute werkzeug test app)
>
> http://projects.unbit.it/hg/uwsgi_pypy/file/7bbae6efc126/uwsgi-pypy.py
>
> Well, running it and benchmarking showed a 10% improvement in response time, so i tried with a more complex app
> (fibonacci's sequence and some prime number generator) and …damn it destroyed cpython :) it served the double of the connections in half of the time….
>
> So, cpyext looks really in good shape, so (probably) the best way would be tweek it to have a libpypy.so.
>
> i will work on the current setup a bit more to have a full WSGI environment, then i will start investigating the original idea (but this time
> base on cpyext)
>
> Thanks for the amazing work

Cool stuff, thanks for letting us know :)

Cheers,
fijal