[Web-SIG] question about connection pool, task queue in WSGI

[Graham Dumpleton]

On 12 July 2012 19:50, est <electronixtar at gmail.com> wrote:
> Hi list,
>
> I am running a site with django + uwsgi, I have few questions about how WSGI
> works.
>
> 1. Is db connection open/close handled by Django? If it's open/closed per
> request,

Yes it is.

> can we make a connection pool in wsgi level, then multiple django
> views can share it?

Only by changing the Django code base from memory. Better off asking
on the Django users list.

> 2. As a general design consideration, can we execute some task *after* the
> response has returned to client? I have some heavy data processing need to
> be done after return HttpResponse() in django, the standard way to do this
> seems like Celery or other task queue with a broker. It's just too
> heavyweight. Is it possible to do some simple background task in WSGI
> directly?

Read:

http://code.google.com/p/modwsgi/wiki/RegisteringCleanupCode

In doing this though, it ties up the request thread and so it would
not be able to handle other requests until your task has finished.

Creating background threads at the end of a request is not a good idea
unless you do it using a pooling mechanism such that you limit the
number of worker threads for your tasks. Because the process can crash
or be shutdown, you loose the job as only in memory and thus not
persistent.

Better to use Celery, or if you think that is too heavy weight, have a
look at Redis Queue (RQ) instead.

Graham