[Flask] uWSGI and Flask - Slow and varying response times

[Mark Pors]

On Wed, Jul 6, 2016 at 9:40 AM, Tim van der Linden <tim at shisaa.jp> wrote:

> Hi All
>
> I'm in a bit of a head scratching situation and need some fresh pair of
> eyes on this.
>
> I have chosen to post this on the Flask mailing list as my application is
> pure Flask ... but I think the problem might involve the whole server stack.
>
> I'm hoping you guys can give me some pointers as to where I should look
> for solutions.
>
> The situation: I have a freshly deployed Flask application running under
> uWSGI with Nginx in front and PostgreSQL 9.5 as the database backend.
> Nothing special there, a typical setup.
>
> This stack is running on a large, dedicated server (not a VPS) with 64
> cores, 130 Gb of RAM and 1.4 TB of SSDs. Also not too bad.
>
> The application is finished and I am now benchmarking it and optimizing
> where needed.
>
> The problem:
>
> On my local development machine the most heavy pages return in around ~200
> ms, which is quite fast considering what is happening on such a page. Some
> database queries need to be pulled, some JSON needs to be decoded and
> Jinja2 needs to construct and render a fairly large page (without Jinja
> caching). Flask rocks!
>
> On the production machine (the huge machine described above), however, I
> get response times that are all over the place. The lowest is ~500 ms the
> longest is well over 2 seconds.
>
> Even the lowest time is more than double of what I can get out of my
> development machine which is less than 10% of the server specs and running
> with the built in Werkzeug development sever.
>
> Now, of course, the production server is not as isolated as my development
> machine but it is not sweating either. It currently gets around ~10 hits
> per second on uWSGI (static content not counted as this is directly handled
> by Nginx).
>
> The database itself is handling a few dozen queries a second and is almost
> not showing up in the process list. Only the uWSGI workers (about 10 of
> them, the rest are idle) consume around 30~60% CPU with some peaks to 90%.
>
> In other words: the whole server isn't doing a whole lot but the response
> times are not only much slower than local but also greatly vary in between
> requests.
>
> I even profiled the server with Werkzeug's ProfilerMiddleware and SnakeViz
> which doesn't give me much direction. The function load is spread quite
> identically in every request - if the request takes four times as long, all
> functions take four times as long as well.
>
>

All functions? Also if there is no DB access or other I/O initiated by
these functions (or waiting on it)?



> This would indicate that Python in general is slow ... ?
>
> Do you guys have any experience with running Flask behind uWSGI/Nginx and
> having slow and varying response times?
>
> Some extra info:
>
> The current vassal configuration for uWSGI is running with 20 processes
> and 50 threads and talking directly with Nginx over a UNIX socket (no TCP).
>
> PostgreSQL is set to handle around 250 concurrent connections with shared
> buffer of 48Gb.
>
> All insane values.
>
> Cheers,
> Tim
> _______________________________________________
> Flask mailing list
> Flask at python.org
> https://mail.python.org/mailman/listinfo/flask
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/flask/attachments/20160706/af50edb5/attachment-0001.html>