[pypy-dev] Playing with PyPy and Django

Tin Tvrtković tinchester at gmail.com
Mon Feb 9 00:02:11 CET 2015 

Wow, that's what I call customer support! :) Can confirm PyPy is better 
than CPython on the admin now, post warmup.

You're right, it doesn't help my site very much, but that's my fault for 
preparing the wrong benchmark fixture. :)

On the other hand, it doesn't really make sense for you PyPy devs to 
help each and every user with their Django site. Would it be worth it 
for PyPy to add a few benchmarks involving Django (and say, Flask) to 
the regular benchmark suite? I see there's a "django" benchmark but it 
only involves the templating subsystem.

For example, one benchmark might be serving the admin page; another 
might involve a dummy site with a custom app, 3-4 models from sqlite, 
and 1-2 templates? And the same for Flask with Flask-SQLAlchemy. Another 
option would be using a complex, existing free software Django app 
(Review Board, Seafile, ...)

This would probably need to be worked out further (i.e. which versions 
to use, when to bump them, etc) but there's probably a large number of 
users out there using Python for exactly these kinds of things.

In any case, thanks for looking into this, it sure does feel nice when 
upstream devs engage with users. :)

On 08.02.2015 23:09, Maciej Fijalkowski wrote:
> It got merged into django, PyPy (2, didn't test the 3) is now faster
> than cpython on django admin. It likely does not help your cause
> though so you need to come up with a better benchmark :-)
>
> On Sun, Feb 8, 2015 at 8:30 PM, Maciej Fijalkowski <fijall at gmail.com> wrote:
>> Hey Tin.
>>
>> We are in the process of two (trivial) pull requests to django that
>> drop the rendering time from 25ms -> 8ms for this case. I'm not a
>> farseer, however I suspect something like this can be done with your
>> site too (depending what it really does).
>>
>> On Sun, Feb 8, 2015 at 8:23 PM, Tin Tvrtković <tinchester at gmail.com> wrote:
>>> Hello,
>>>
>>> thanks to everyone for their input (especially Maciej).
>>>
>>> Since I've ripped out all my code from the test project, it's not a Python 3
>>> project anymore, so I did try PyPy 2 with it too. It's faster, yes; the last
>>> test looked something like:
>>>
>>> PyPy 2         20.206 [ms] (mean)
>>> PyPy 3         27.588 [ms] (mean)
>>> CPython 3.4    10.865 [ms] (mean)
>>>
>>> (tried both PyPy 2.5, downloaded directly, and 2.4 from the Ubuntu PPA -
>>> about the same.)
>>>
>>> I don't know, maybe the admin page isn't a good test candidate since it
>>> might do more introspective/reflective/meta stuff. I've tried doing this
>>> same test (10k warmup requests, 100 benchmark requests) on the main page of
>>> the site I'm working on; the results are:
>>>
>>> PyPy 3         15.863 [ms] (mean)
>>> CPython 3.48.668 [ms] (mean)
>>>
>>> and I can be certain there's nothing much going on there. Just some
>>> permission checks, grabbing some stuff from the database, checking some
>>> query parameters and a template render.
>>>
>>> I suppose if folks have CPU intensive stuff in their view functions (nested
>>> for-loops etc) the optimizations PyPy can do to that far outweigh the
>>> non-optimized Django parts; my sites generally don't though.
>>>
>>>
>>> On 08.02.2015 18:44, Maciej Fijalkowski wrote:
>>>> then it's used in the wrong ways in say django admin, look at
>>>> invocations to lazy there (this is how it showed up in my profile)
>>>>
>>>> On Sun, Feb 8, 2015 at 5:08 PM, Alex Gaynor <alex.gaynor at gmail.com> wrote:
>>>>> FWIW, I've definitely seen and worked on Django sites that got major
>>>>> improvements out of PyPy -- both the templates and ORM are both sped up
>>>>> substantially by it. I think we should try to fix issues as we see them,
>>>>> before writing it off.
>>>>>
>>>>> FWIW: lazy does not create a new class per call of a function, it's only
>>>>> one
>>>>> class per decorated function.
>>>>>
>>>>> Alex
>>>>>
>>>>> On Sun, Feb 8, 2015 at 6:59 AM, Maciej Fijalkowski <fijall at gmail.com>
>>>>> wrote:
>>>>>> I don't know :-( not sure if fixing those issues one by one is the way
>>>>>> to go, it's not like *this* particular code is a problem, it's the
>>>>>> culture that breeds those sort of things
>>>>>>
>>>>>> On Sun, Feb 8, 2015 at 2:09 PM, Omer Katz <omer.drow at gmail.com> wrote:
>>>>>>> Isn't there anything we can do about it?
>>>>>>> We can open issues at the Django issue tracker if some code slows down
>>>>>>> execution in PyPy.
>>>>>>>
>>>>>>> 2015-02-08 12:17 GMT+02:00 Maciej Fijalkowski <fijall at gmail.com>:
>>>>>>>> Hi Tin
>>>>>>>>
>>>>>>>> I have a bit sad news for you, but essentially from what I looked at
>>>>>>>> profling, the answer seems to be "don't use django, it has not been
>>>>>>>> written with performance in mind". For example here (which features in
>>>>>>>> django admin quite prominently, some stuff calls lazy() at each call):
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> https://github.com/django/django/blob/master/django/utils/functional.py#L48
>>>>>>>>
>>>>>>>> What does it, per call that pypy does not like:
>>>>>>>>
>>>>>>>> * redefines classes
>>>>>>>>
>>>>>>>> * walks the class __mro__ and defines tons of new methods
>>>>>>>>
>>>>>>>> * proxies everything through a not so efficietn mechanisms
>>>>>>>>
>>>>>>>> Those things quite defy the purpose of the JIT - you're making tons of
>>>>>>>> very dynamic things that pypy generally assumes is "static enough", if
>>>>>>>> you wanted to do the same thing in C++, you would need to call gcc
>>>>>>>> with each request, that would not be so much fun.
>>>>>>>>
>>>>>>>> If you really care about performance, consider swapping your web
>>>>>>>> framework to something more performance-oriented where JIT compilation
>>>>>>>> can help.
>>>>>>>>
>>>>>>>> As a side note, we (me and my company, not to be confused with PyPy
>>>>>>>> open source project) do offer looking at proprietary code for
>>>>>>>> benchmarking purposes as a commercial service, look at
>>>>>>>> baroquesoftware.com
>>>>>>>>
>>>>>>>> Cheers,
>>>>>>>> fijal
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> On Sat, Feb 7, 2015 at 1:12 AM, Tin Tvrtković <tinchester at gmail.com>
>>>>>>>> wrote:
>>>>>>>>> Hello, PyPy folks!
>>>>>>>>>
>>>>>>>>> While trying to speed up one of my Django sites, I noticed a new
>>>>>>>>> version
>>>>>>>>> of
>>>>>>>>> PyPy
>>>>>>>>> had just been released. So I grabbed a fresh download of PyPy 3
>>>>>>>>> (since
>>>>>>>>> this
>>>>>>>>> is
>>>>>>>>> a Python 3 codebase) and tried taking it out for a spin.
>>>>>>>>>
>>>>>>>>> However, as far as I can see, whatever I try PyPy is consistently
>>>>>>>>> slower
>>>>>>>>> than
>>>>>>>>> CPython for this.
>>>>>>>>>
>>>>>>>>> Since this is a proprietary site, I've basically ripped out all the
>>>>>>>>> code
>>>>>>>>> except
>>>>>>>>> my settings.py and my requirements; and am benchmarking the Django
>>>>>>>>> admin
>>>>>>>>> index.
>>>>>>>>> The results are about the same.
>>>>>>>>>
>>>>>>>>> I've set up a small repo that can be used to reproduce the
>>>>>>>>> environment:
>>>>>>>>> https://github.com/Tinche/PyPy-Django-Playground. There's additional
>>>>>>>>> info in
>>>>>>>>> the README there.
>>>>>>>>>
>>>>>>>>> These tests have been carried out on Ubuntu Trusty, 64-bit. CPython 3
>>>>>>>>> is
>>>>>>>>> the
>>>>>>>>> system Python, 3.4. PyPy has been downloaded from the official site
>>>>>>>>> and
>>>>>>>>> unzipped.
>>>>>>>>>
>>>>>>>>> So what I basically do is set up an admin session, and use the Django
>>>>>>>>> main
>>>>>>>>> admin
>>>>>>>>> page. 200 warmup requests, then 100 benchmarked requests, look at the
>>>>>>>>> mean
>>>>>>>>> request time.
>>>>>>>>>
>>>>>>>>> Some results:
>>>>>>>>>
>>>>>>>>> Django's runserver, DEBUG mode:
>>>>>>>>>
>>>>>>>>> PyPy3            485.389 [ms]
>>>>>>>>> CPython 3.4      105.777 [ms]
>>>>>>>>>
>>>>>>>>> Django's runserver, no debug:
>>>>>>>>>
>>>>>>>>> PyPy3             44.661 [ms]
>>>>>>>>> CPython 3.4       18.697 [ms]
>>>>>>>>>
>>>>>>>>> Gunicorn, 1 worker, no debug:
>>>>>>>>>
>>>>>>>>> PyPy3             28.615 [ms]
>>>>>>>>> CPython 3.4       13.532 [ms]
>>>>>>>>>
>>>>>>>>> I don't exactly claim to be an expert on benchmarking, but assuming
>>>>>>>>> my
>>>>>>>>> site
>>>>>>>>> is similar to the Django admin, CPython's gonna be giving me better
>>>>>>>>> performance.
>>>>>>>>> Also the debug runserver performance is kinda worrying. Nobody's
>>>>>>>>> going
>>>>>>>>> to be
>>>>>>>>> running this in production, but it makes development a little slower
>>>>>>>>> and
>>>>>>>>> more
>>>>>>>>> annoying than it should be.
>>>>>>>>>
>>>>>>>>> Is there anything to make PyPy more competitive in these kinds of
>>>>>>>>> scenarios?
>>>>>>>>>
>>>>>>>>> Kind regards,
>>>>>>>>> Tin
>>>>>>>>> _______________________________________________
>>>>>>>>> pypy-dev mailing list
>>>>>>>>> pypy-dev at python.org
>>>>>>>>> https://mail.python.org/mailman/listinfo/pypy-dev
>>>>>>>> _______________________________________________
>>>>>>>> pypy-dev mailing list
>>>>>>>> pypy-dev at python.org
>>>>>>>> https://mail.python.org/mailman/listinfo/pypy-dev
>>>>>>>
>>>>>> _______________________________________________
>>>>>> pypy-dev mailing list
>>>>>> pypy-dev at python.org
>>>>>> https://mail.python.org/mailman/listinfo/pypy-dev
>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> "I disapprove of what you say, but I will defend to the death your right
>>>>> to
>>>>> say it." -- Evelyn Beatrice Hall (summarizing Voltaire)
>>>>> "The people's good is the highest law." -- Cicero
>>>>> GPG Key fingerprint: 125F 5C67 DFE9 4084
>>>

More information about the pypy-dev mailing list