[CentralOH] Profiling Django Apps?

[Nick Albright]

Whew!  OK, thanks guys, really helped me figure out what was OK and what
wasn't OK.

I ultimately did 3 things:

1) Set AWS_S3_CUSTOM_DOMAIN variable in settings.py (First refinement, I
should go back and check on this now.  But at the time I think eliminated an
expensive 'make_request' call to S3 to just get the proper domain for a url)

2) set pre_cache=True on all my imagekit specs.py files  As per:
https://bitbucket.org/jdriscoll/django-imagekit/issue/28/slow-to-fetch-image-urls

3) added in the width and height method changes from:
https://github.com/Yipit/django-imagekit/commit/e50425d2390dac11fdda7c0b258e55c763a9d7d8

This took my initial page response time from 2.5-3+ seconds down to a much
more responsive feeling .7-.9 secs

Whew! Django for the win!  (Was concerned at first our box was too slow to
run the django site I've been coding!)

Thanks again!
 -Nick


On Thu, Aug 25, 2011 at 3:48 PM, Nick Albright <nick.albright at gmail.com>wrote:

> Yeah, the chrome debugger is telling me that the first html page takes 2.5+
> seconds to return, so it is someplace in django I believe.
>
> Thanks E!  That snippet ruled!  Looks like alot of time spent in those aws
> external calls!
>
> And Thanks Issac for that Cache panel.  (Turns out it is debug_toolbar.panels.cache.CacheDebugPanel
> (ie, add 'Debug' in the middle)
>
> Peace,
>  -Nick
>
> On Thu, Aug 25, 2011 at 2:29 PM, Eric Floehr <eric at intellovations.com>wrote:
>
>> Nick,
>>
>> If it isn't web-specific stuff that DDT or Yslow would catch, you can
>> try pdb (the Python debugger) which can be invoked within emacs, vi
>> (vimpdb), or your favorite IDE, or you can try profiling your methods
>> via the standard library cProfile module.  You could create a cProfile
>> middleware something like:
>>
>> http://djangosnippets.org/snippets/727/
>>
>> Cheers,
>> Eric
>>
>>
>> On Thu, Aug 25, 2011 at 2:25 PM, Nick Albright <nick.albright at gmail.com>
>> wrote:
>> > Well, I've configured s3 boto storage for the default storage of django,
>> and
>> > using imagekit to process all the diff versions of the photos. (All that
>> is
>> > really being uploaded is photos), so dunno if stores it in the DB
>> someplace?
>> > or makes calls?
>> > Though, checking the Log Messages section in Django Debug Toolbar, seems
>> to
>> > show alot of boto calls,, so it could be external calls in real time.
>> > I'm gonna keep digging! Thanks!
>> >  -Nick
>> > On Thu, Aug 25, 2011 at 2:13 PM, Issac Kelly <issac.kelly at gmail.com>
>> wrote:
>> >>
>> >> Are you making any external API requests during the request cycle? That
>> >> can definitely slow you down.
>> >>
>> >> It seems like your SQL stuff is probably alright.
>> >>
>> >> On Thu, Aug 25, 2011 at 2:10 PM, Nick Albright <
>> nick.albright at gmail.com>
>> >> wrote:
>> >>>
>> >>> <G>  Heck, what'd I'd love is like a python profiler type output
>> (Well, I
>> >>> think I'd love that, but I may rue that day ;)
>> >>> Initial look through, the resource usage
>> >>> (http://dpaste.com/602618/)  generally seems to be like .5 sec CPU
>> time, and
>> >>> 2.5 sec elapsed time.  (2.5 secs for just raw/initial HTML is the
>> obvious
>> >>> bottleneck)
>> >>> Then 12 SQL queries taking like .4 to .9 ms each
>> >>> And the html is < 1K lines.
>> >>> And this is before any JS gets processed by the browser.  Just the
>> >>> initial HTML page.
>> >>> I'm currently wildly guessing that it may be the boto library I'm
>> using
>> >>> for the amazon S3?  But would love more solid info.
>> >>>  -Nick
>> >>> On Thu, Aug 25, 2011 at 1:48 PM, Issac Kelly <issac.kelly at gmail.com>
>> >>> wrote:
>> >>>>
>> >>>> What else do you want out of DDT? I've found that between the cache
>> >>>> panel, and the SQL panel, I've been able to find most of my problems,
>> my
>> >>>> worst problems have been a result of not properly using
>> select_related, or
>> >>>> iterating over big querysets.
>> >>>> If it's slow, but not showing up in DDT, it could be one of several
>> >>>> things:
>> >>>> * Really, Really big pages (>10k lines) the template engine seems to
>> >>>> crap out on some of this
>> >>>> * Bad/Really Big JS.
>> >>>>
>> >>>>
>> >>>> On Thu, Aug 25, 2011 at 1:42 PM, Nick Albright <
>> nick.albright at gmail.com>
>> >>>> wrote:
>> >>>>>
>> >>>>> Hello Everybody!
>> >>>>> I was wondering if anyone had any experience with profiling django
>> >>>>> apps. Things seem to be taking longer than I'd like in my one app,
>> and I'm
>> >>>>> trying to figure out where the bottleneck may be.  And was wondering
>> if
>> >>>>> anyone has done this before?  (I'm using django-debug-toolbar, which
>> doesn't
>> >>>>> give me enough info)
>> >>>>> Just curious!
>> >>>>>  -Nick
>> >>>>> _______________________________________________
>> >>>>> CentralOH mailing list
>> >>>>> CentralOH at python.org
>> >>>>> http://mail.python.org/mailman/listinfo/centraloh
>> >>>>>
>> >>>>
>> >>>>
>> >>>> _______________________________________________
>> >>>> CentralOH mailing list
>> >>>> CentralOH at python.org
>> >>>> http://mail.python.org/mailman/listinfo/centraloh
>> >>>>
>> >>>
>> >>>
>> >>> _______________________________________________
>> >>> CentralOH mailing list
>> >>> CentralOH at python.org
>> >>> http://mail.python.org/mailman/listinfo/centraloh
>> >>>
>> >>
>> >>
>> >> _______________________________________________
>> >> CentralOH mailing list
>> >> CentralOH at python.org
>> >> http://mail.python.org/mailman/listinfo/centraloh
>> >>
>> >
>> >
>> > _______________________________________________
>> > CentralOH mailing list
>> > CentralOH at python.org
>> > http://mail.python.org/mailman/listinfo/centraloh
>> >
>> >
>> _______________________________________________
>> CentralOH mailing list
>> CentralOH at python.org
>> http://mail.python.org/mailman/listinfo/centraloh
>>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/mailman/private/centraloh/attachments/20110825/9408767a/attachment-0001.html>