<div dir="rtl"><div dir="ltr">I have some Django benchmarks that I was too lazy to contribute yet.<br></div><div dir="ltr">I will do so soon.<br></div></div><div class="gmail_extra"><br><div class="gmail_quote"><div dir="ltr">2015-02-09 8:41 GMT+02:00 Maciej Fijalkowski <span dir="ltr"><<a href="mailto:fijall@gmail.com" target="_blank">fijall@gmail.com</a>></span>:</div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">The answer is - yes maybe, but there is no such thing as "django<br>
site". Most of the time the bottlenecks are somewhere else (as you can<br>
see, the admin does not help you) and is often something stupid (e.g.<br>
like here people defining classes at runtime for no good reason) that<br>
has to be looked on a case to case basis. Ideally we would not care<br>
and just optimize everything away anyway, but it's difficult and the<br>
sort of Python that PyPy improves is growing but still not covering<br>
everything we would want.<br>
<br>
Cheers,<br>
fijal<br>
<div class="HOEnZb"><div class="h5"><br>
<br>
On Mon, Feb 9, 2015 at 1:02 AM, Tin Tvrtković <<a href="mailto:tinchester@gmail.com">tinchester@gmail.com</a>> wrote:<br>
> Wow, that's what I call customer support! :) Can confirm PyPy is better than<br>
> CPython on the admin now, post warmup.<br>
><br>
> You're right, it doesn't help my site very much, but that's my fault for<br>
> preparing the wrong benchmark fixture. :)<br>
><br>
> On the other hand, it doesn't really make sense for you PyPy devs to help<br>
> each and every user with their Django site. Would it be worth it for PyPy to<br>
> add a few benchmarks involving Django (and say, Flask) to the regular<br>
> benchmark suite? I see there's a "django" benchmark but it only involves the<br>
> templating subsystem.<br>
><br>
> For example, one benchmark might be serving the admin page; another might<br>
> involve a dummy site with a custom app, 3-4 models from sqlite, and 1-2<br>
> templates? And the same for Flask with Flask-SQLAlchemy. Another option<br>
> would be using a complex, existing free software Django app (Review Board,<br>
> Seafile, ...)<br>
><br>
> This would probably need to be worked out further (i.e. which versions to<br>
> use, when to bump them, etc) but there's probably a large number of users<br>
> out there using Python for exactly these kinds of things.<br>
><br>
> In any case, thanks for looking into this, it sure does feel nice when<br>
> upstream devs engage with users. :)<br>
><br>
><br>
> On 08.02.2015 23:09, Maciej Fijalkowski wrote:<br>
>><br>
>> It got merged into django, PyPy (2, didn't test the 3) is now faster<br>
>> than cpython on django admin. It likely does not help your cause<br>
>> though so you need to come up with a better benchmark :-)<br>
>><br>
>> On Sun, Feb 8, 2015 at 8:30 PM, Maciej Fijalkowski <<a href="mailto:fijall@gmail.com">fijall@gmail.com</a>><br>
>> wrote:<br>
>>><br>
>>> Hey Tin.<br>
>>><br>
>>> We are in the process of two (trivial) pull requests to django that<br>
>>> drop the rendering time from 25ms -> 8ms for this case. I'm not a<br>
>>> farseer, however I suspect something like this can be done with your<br>
>>> site too (depending what it really does).<br>
>>><br>
>>> On Sun, Feb 8, 2015 at 8:23 PM, Tin Tvrtković <<a href="mailto:tinchester@gmail.com">tinchester@gmail.com</a>><br>
>>> wrote:<br>
>>>><br>
>>>> Hello,<br>
>>>><br>
>>>> thanks to everyone for their input (especially Maciej).<br>
>>>><br>
>>>> Since I've ripped out all my code from the test project, it's not a<br>
>>>> Python 3<br>
>>>> project anymore, so I did try PyPy 2 with it too. It's faster, yes; the<br>
>>>> last<br>
>>>> test looked something like:<br>
>>>><br>
>>>> PyPy 2 20.206 [ms] (mean)<br>
>>>> PyPy 3 27.588 [ms] (mean)<br>
>>>> CPython 3.4 10.865 [ms] (mean)<br>
>>>><br>
>>>> (tried both PyPy 2.5, downloaded directly, and 2.4 from the Ubuntu PPA -<br>
>>>> about the same.)<br>
>>>><br>
>>>> I don't know, maybe the admin page isn't a good test candidate since it<br>
>>>> might do more introspective/reflective/meta stuff. I've tried doing this<br>
>>>> same test (10k warmup requests, 100 benchmark requests) on the main page<br>
>>>> of<br>
>>>> the site I'm working on; the results are:<br>
>>>><br>
>>>> PyPy 3 15.863 [ms] (mean)<br>
>>>> CPython 3.48.668 [ms] (mean)<br>
>>>><br>
>>>> and I can be certain there's nothing much going on there. Just some<br>
>>>> permission checks, grabbing some stuff from the database, checking some<br>
>>>> query parameters and a template render.<br>
>>>><br>
>>>> I suppose if folks have CPU intensive stuff in their view functions<br>
>>>> (nested<br>
>>>> for-loops etc) the optimizations PyPy can do to that far outweigh the<br>
>>>> non-optimized Django parts; my sites generally don't though.<br>
>>>><br>
>>>><br>
>>>> On 08.02.2015 18:44, Maciej Fijalkowski wrote:<br>
>>>>><br>
>>>>> then it's used in the wrong ways in say django admin, look at<br>
>>>>> invocations to lazy there (this is how it showed up in my profile)<br>
>>>>><br>
>>>>> On Sun, Feb 8, 2015 at 5:08 PM, Alex Gaynor <<a href="mailto:alex.gaynor@gmail.com">alex.gaynor@gmail.com</a>><br>
>>>>> wrote:<br>
>>>>>><br>
>>>>>> FWIW, I've definitely seen and worked on Django sites that got major<br>
>>>>>> improvements out of PyPy -- both the templates and ORM are both sped<br>
>>>>>> up<br>
>>>>>> substantially by it. I think we should try to fix issues as we see<br>
>>>>>> them,<br>
>>>>>> before writing it off.<br>
>>>>>><br>
>>>>>> FWIW: lazy does not create a new class per call of a function, it's<br>
>>>>>> only<br>
>>>>>> one<br>
>>>>>> class per decorated function.<br>
>>>>>><br>
>>>>>> Alex<br>
>>>>>><br>
>>>>>> On Sun, Feb 8, 2015 at 6:59 AM, Maciej Fijalkowski <<a href="mailto:fijall@gmail.com">fijall@gmail.com</a>><br>
>>>>>> wrote:<br>
>>>>>>><br>
>>>>>>> I don't know :-( not sure if fixing those issues one by one is the<br>
>>>>>>> way<br>
>>>>>>> to go, it's not like *this* particular code is a problem, it's the<br>
>>>>>>> culture that breeds those sort of things<br>
>>>>>>><br>
>>>>>>> On Sun, Feb 8, 2015 at 2:09 PM, Omer Katz <<a href="mailto:omer.drow@gmail.com">omer.drow@gmail.com</a>><br>
>>>>>>> wrote:<br>
>>>>>>>><br>
>>>>>>>> Isn't there anything we can do about it?<br>
>>>>>>>> We can open issues at the Django issue tracker if some code slows<br>
>>>>>>>> down<br>
>>>>>>>> execution in PyPy.<br>
>>>>>>>><br>
>>>>>>>> 2015-02-08 12:17 GMT+02:00 Maciej Fijalkowski <<a href="mailto:fijall@gmail.com">fijall@gmail.com</a>>:<br>
>>>>>>>>><br>
>>>>>>>>> Hi Tin<br>
>>>>>>>>><br>
>>>>>>>>> I have a bit sad news for you, but essentially from what I looked<br>
>>>>>>>>> at<br>
>>>>>>>>> profling, the answer seems to be "don't use django, it has not been<br>
>>>>>>>>> written with performance in mind". For example here (which features<br>
>>>>>>>>> in<br>
>>>>>>>>> django admin quite prominently, some stuff calls lazy() at each<br>
>>>>>>>>> call):<br>
>>>>>>>>><br>
>>>>>>>>><br>
>>>>>>>>><br>
>>>>>>>>><br>
>>>>>>>>> <a href="https://github.com/django/django/blob/master/django/utils/functional.py#L48" target="_blank">https://github.com/django/django/blob/master/django/utils/functional.py#L48</a><br>
>>>>>>>>><br>
>>>>>>>>> What does it, per call that pypy does not like:<br>
>>>>>>>>><br>
>>>>>>>>> * redefines classes<br>
>>>>>>>>><br>
>>>>>>>>> * walks the class __mro__ and defines tons of new methods<br>
>>>>>>>>><br>
>>>>>>>>> * proxies everything through a not so efficietn mechanisms<br>
>>>>>>>>><br>
>>>>>>>>> Those things quite defy the purpose of the JIT - you're making tons<br>
>>>>>>>>> of<br>
>>>>>>>>> very dynamic things that pypy generally assumes is "static enough",<br>
>>>>>>>>> if<br>
>>>>>>>>> you wanted to do the same thing in C++, you would need to call gcc<br>
>>>>>>>>> with each request, that would not be so much fun.<br>
>>>>>>>>><br>
>>>>>>>>> If you really care about performance, consider swapping your web<br>
>>>>>>>>> framework to something more performance-oriented where JIT<br>
>>>>>>>>> compilation<br>
>>>>>>>>> can help.<br>
>>>>>>>>><br>
>>>>>>>>> As a side note, we (me and my company, not to be confused with PyPy<br>
>>>>>>>>> open source project) do offer looking at proprietary code for<br>
>>>>>>>>> benchmarking purposes as a commercial service, look at<br>
>>>>>>>>> <a href="http://baroquesoftware.com" target="_blank">baroquesoftware.com</a><br>
>>>>>>>>><br>
>>>>>>>>> Cheers,<br>
>>>>>>>>> fijal<br>
>>>>>>>>><br>
>>>>>>>>><br>
>>>>>>>>><br>
>>>>>>>>> On Sat, Feb 7, 2015 at 1:12 AM, Tin Tvrtković<br>
>>>>>>>>> <<a href="mailto:tinchester@gmail.com">tinchester@gmail.com</a>><br>
>>>>>>>>> wrote:<br>
>>>>>>>>>><br>
>>>>>>>>>> Hello, PyPy folks!<br>
>>>>>>>>>><br>
>>>>>>>>>> While trying to speed up one of my Django sites, I noticed a new<br>
>>>>>>>>>> version<br>
>>>>>>>>>> of<br>
>>>>>>>>>> PyPy<br>
>>>>>>>>>> had just been released. So I grabbed a fresh download of PyPy 3<br>
>>>>>>>>>> (since<br>
>>>>>>>>>> this<br>
>>>>>>>>>> is<br>
>>>>>>>>>> a Python 3 codebase) and tried taking it out for a spin.<br>
>>>>>>>>>><br>
>>>>>>>>>> However, as far as I can see, whatever I try PyPy is consistently<br>
>>>>>>>>>> slower<br>
>>>>>>>>>> than<br>
>>>>>>>>>> CPython for this.<br>
>>>>>>>>>><br>
>>>>>>>>>> Since this is a proprietary site, I've basically ripped out all<br>
>>>>>>>>>> the<br>
>>>>>>>>>> code<br>
>>>>>>>>>> except<br>
>>>>>>>>>> my settings.py and my requirements; and am benchmarking the Django<br>
>>>>>>>>>> admin<br>
>>>>>>>>>> index.<br>
>>>>>>>>>> The results are about the same.<br>
>>>>>>>>>><br>
>>>>>>>>>> I've set up a small repo that can be used to reproduce the<br>
>>>>>>>>>> environment:<br>
>>>>>>>>>> <a href="https://github.com/Tinche/PyPy-Django-Playground" target="_blank">https://github.com/Tinche/PyPy-Django-Playground</a>. There's<br>
>>>>>>>>>> additional<br>
>>>>>>>>>> info in<br>
>>>>>>>>>> the README there.<br>
>>>>>>>>>><br>
>>>>>>>>>> These tests have been carried out on Ubuntu Trusty, 64-bit.<br>
>>>>>>>>>> CPython 3<br>
>>>>>>>>>> is<br>
>>>>>>>>>> the<br>
>>>>>>>>>> system Python, 3.4. PyPy has been downloaded from the official<br>
>>>>>>>>>> site<br>
>>>>>>>>>> and<br>
>>>>>>>>>> unzipped.<br>
>>>>>>>>>><br>
>>>>>>>>>> So what I basically do is set up an admin session, and use the<br>
>>>>>>>>>> Django<br>
>>>>>>>>>> main<br>
>>>>>>>>>> admin<br>
>>>>>>>>>> page. 200 warmup requests, then 100 benchmarked requests, look at<br>
>>>>>>>>>> the<br>
>>>>>>>>>> mean<br>
>>>>>>>>>> request time.<br>
>>>>>>>>>><br>
>>>>>>>>>> Some results:<br>
>>>>>>>>>><br>
>>>>>>>>>> Django's runserver, DEBUG mode:<br>
>>>>>>>>>><br>
>>>>>>>>>> PyPy3 485.389 [ms]<br>
>>>>>>>>>> CPython 3.4 105.777 [ms]<br>
>>>>>>>>>><br>
>>>>>>>>>> Django's runserver, no debug:<br>
>>>>>>>>>><br>
>>>>>>>>>> PyPy3 44.661 [ms]<br>
>>>>>>>>>> CPython 3.4 18.697 [ms]<br>
>>>>>>>>>><br>
>>>>>>>>>> Gunicorn, 1 worker, no debug:<br>
>>>>>>>>>><br>
>>>>>>>>>> PyPy3 28.615 [ms]<br>
>>>>>>>>>> CPython 3.4 13.532 [ms]<br>
>>>>>>>>>><br>
>>>>>>>>>> I don't exactly claim to be an expert on benchmarking, but<br>
>>>>>>>>>> assuming<br>
>>>>>>>>>> my<br>
>>>>>>>>>> site<br>
>>>>>>>>>> is similar to the Django admin, CPython's gonna be giving me<br>
>>>>>>>>>> better<br>
>>>>>>>>>> performance.<br>
>>>>>>>>>> Also the debug runserver performance is kinda worrying. Nobody's<br>
>>>>>>>>>> going<br>
>>>>>>>>>> to be<br>
>>>>>>>>>> running this in production, but it makes development a little<br>
>>>>>>>>>> slower<br>
>>>>>>>>>> and<br>
>>>>>>>>>> more<br>
>>>>>>>>>> annoying than it should be.<br>
>>>>>>>>>><br>
>>>>>>>>>> Is there anything to make PyPy more competitive in these kinds of<br>
>>>>>>>>>> scenarios?<br>
>>>>>>>>>><br>
>>>>>>>>>> Kind regards,<br>
>>>>>>>>>> Tin<br>
>>>>>>>>>> _______________________________________________<br>
>>>>>>>>>> pypy-dev mailing list<br>
>>>>>>>>>> <a href="mailto:pypy-dev@python.org">pypy-dev@python.org</a><br>
>>>>>>>>>> <a href="https://mail.python.org/mailman/listinfo/pypy-dev" target="_blank">https://mail.python.org/mailman/listinfo/pypy-dev</a><br>
>>>>>>>>><br>
>>>>>>>>> _______________________________________________<br>
>>>>>>>>> pypy-dev mailing list<br>
>>>>>>>>> <a href="mailto:pypy-dev@python.org">pypy-dev@python.org</a><br>
>>>>>>>>> <a href="https://mail.python.org/mailman/listinfo/pypy-dev" target="_blank">https://mail.python.org/mailman/listinfo/pypy-dev</a><br>
>>>>>>>><br>
>>>>>>>><br>
>>>>>>> _______________________________________________<br>
>>>>>>> pypy-dev mailing list<br>
>>>>>>> <a href="mailto:pypy-dev@python.org">pypy-dev@python.org</a><br>
>>>>>>> <a href="https://mail.python.org/mailman/listinfo/pypy-dev" target="_blank">https://mail.python.org/mailman/listinfo/pypy-dev</a><br>
>>>>>><br>
>>>>>><br>
>>>>>><br>
>>>>>><br>
>>>>>> --<br>
>>>>>> "I disapprove of what you say, but I will defend to the death your<br>
>>>>>> right<br>
>>>>>> to<br>
>>>>>> say it." -- Evelyn Beatrice Hall (summarizing Voltaire)<br>
>>>>>> "The people's good is the highest law." -- Cicero<br>
>>>>>> GPG Key fingerprint: 125F 5C67 DFE9 4084<br>
>>>><br>
>>>><br>
><br>
</div></div></blockquote></div><br></div>