On Fri, Apr 13, 2012 at 8:15 AM, Roberto De Ioris <roberto@unbit.it> wrote:
I am afraid it's not interesting. I just wanted to test it with our small project - geoip server which resolves ip address to region. Now we use bjoern wsgi server for running this project. Basically it's does nothing but bisect search on array.array. When i run this code without wsgi wrapper using pypy it shows about 400% speed boots. Unfortunately uwsgi with pypy plugin shows me pretty same perfomance as usual uwsgi. When i wrote simple application which loop over "for" cycle and increment some variable, i got same result. I'll try running more tests to figure out what I'm doing wrong. I am a fan of your work and looking forward to using pypy in our production project.
Hi,
when you compile libpypy, the translator should generate a pypy-c binary.
Try to run your code without the wsgi wrapper with this new binary, to check if you still have a 400 times improvement.
I think he said 400%, that's not as good ;-)
If you get 'slower' values over a standard binary release of pypy, it means you have compiled it wrongly (maybe without the jit).
In my tests, 99% of the webapps (being IO-based) do not gain too much power on pypy, but if you have a CPU-bound webapp (like you), you will end screaming WOOOW all of the time :)
working on it...
And take in account that in uWSGI we suggest using lua-jit for all of the need-to-be-fast parts, so to make us scream you have to be faster than it :)
FYI we outperform luajit on richards (giving *ample* warmup, luajit warms up *so* fast), did not measure on anything else. Just sayin
I would like to suggest you another approach, vastly pushed in the past years in pypy talks. It includes a bit of overhead, but you will end with a more solid platform:
delegate the CPU-bound part to a pypy daemon (listening on unix sockets or whatever you want) and run your public webapp with your server of choice running on cpython. In your webapp you simply 'enqueue' tasks to the pypy daemon and wait for the result. Do not make the error to use a complex protocol for it. Go line-based. No need to add more overhead i often see.
If for some reason the pypy daemon crashes (come on it happens ;), you can simply automatically restart it without propagating the down to your public web-services (if it crashes during a request you can simply re-enqueue it in the same transaction)
I think crashes are less of an issue (pypy is relatively stable) compared to the libraries that you have to interface with (like lxml).
With this approach you can continue using bjoern for the public IO-bound part, and abuse pypy for the cpu-heavy one.
In uWSGI this kind of approach is a lot easier (from a sysadmin point-of-view) as you can use the --attach-daemon option, allowing you to 'attach' an external process (your pypy daemon) that will be monitored (and respawned) automatically.
-- Roberto De Ioris http://unbit.it
Thanks for insights Roberto! I'm really glad uWSGI community is interested *and* seem to be taking a reasonable, non-overhyped approach Cheers, fijal