[Flask] Multiple backend servers with Flask

[Jonathan Chen]

There shouldn't be an issue. Try running multiple celery workers. If you
are not experiencing duplication of work there then it shouldn't make a
difference when you scale it out.


~Jonathan C.

On Tue, Aug 4, 2015 at 9:41 AM, Dorian Hoxha <dorian.hoxha at gmail.com> wrote:

> Scaling the app is the ~easiest way.
>
> On celery, what is should do (haven't checked) is that it reserves a job
> so other processes don't get it while 1 is working on it (and if it
> succeeds, it deletes the job, and if it failes the job can be retrieved by
> another process)
>
> So you should be fine.
>
> On Tue, Aug 4, 2015 at 4:31 PM, Morgan Connolly <connollymorg at gmail.com>
> wrote:
>
>> Hello!
>>
>> I have a Flask application that has an architecture like so:
>>
>> http://i.stack.imgur.com/wfVRI.png
>>
>> It currently all runs on a single server, but I need to be able to scale
>> the application by adding in more servers. I know how I can scale the
>> database(by creating a cluster), I do not really care about scaling the
>> message queue at this point, but I can create a cluster for that too.
>>
>> What I am not sure about scaling is the application itself.
>>
>> My sessions are stored client side in cookies (Flask-Login), so that
>> shouldn't be a problem. I am using Redis for storing temporary data that
>> needs to be accessible from every node(application server). This data has a
>> lifetime of 5 minutes, so no big deal.
>>
>> I am using Celery as a task queue, with RabbitMQ as the broker. This is
>> used for running background processes and executing tasks asynchronously.
>>
>> All pages served are basically static HTML, with no AJAX or anything
>> fancy like that.
>>
>> I've thought of setting up two application servers, have each one connect
>> to the same database, same RabbitMQ instance, and same Redis instance, and
>> set up a load balancer in front of it to forward requests to either server.
>> But I'm worried this might cause problems like tasks getting executed twice
>> by Celery; would it?
>>
>> Is there a better way to scale Flask applications?
>>
>> Thanks.
>>
>> _______________________________________________
>> Flask mailing list
>> Flask at python.org
>> https://mail.python.org/mailman/listinfo/flask
>>
>>
>
> _______________________________________________
> Flask mailing list
> Flask at python.org
> https://mail.python.org/mailman/listinfo/flask
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/flask/attachments/20150804/34d560d8/attachment.html>