I am using Twisted Web WSGIResource to host a Django site. Here's the code I use to setup the WSGI resource (other boilerplate skipped)
def wsgi_resource(): from django.core.handlers.wsgi import WSGIHandler pool = threadpool.ThreadPool() pool.start() # Allow Ctrl-C to get you out cleanly: reactor.addSystemEventTrigger('after', 'shutdown', pool.stop) wsgi_resource = wsgi.WSGIResource(reactor, pool, WSGIHandler()) return wsgi_resource
This generally worked quite well. However I have an API endpoint that gets hit with multiple POST requests per second (not very much - I'd say 10-20 req/sec at most). Sometimes I see this in the logs:
Internal Server Error: /cas/eventlog/ Traceback (most recent call last): File "<pythonpath>/site-packages/django/core/handlers/base.py", line 114, in get_response response = wrapped_callback(request, *callback_args, **callback_kwargs) File "<pythonpath>/site-packages/django/views/decorators/csrf.py", line 57, in wrapped_view return view_func(*args, **kwargs) File "<djangoapp>/views.py", line 488, in event_log data = json.loads(request.body) File "<pythonpath>/site-packages/django/http/request.py", line 192, in body self._body = self.read() File "<pythonpath>/site-packages/django/http/request.py", line 246, in read return self._stream.read(*args, **kwargs) File "<pythonpath>/site-packages/django/core/handlers/wsgi.py", line 45, in read result = self.buffer + self._read_limited() File "<pythonpath>/site-packages/django/core/handlers/wsgi.py", line 39, in _read_limited result = self.stream.read(size) File "<pythonpath>/site-packages/twisted/web/wsgi.py", line 94, in read return self._wrapped.read(size) ValueError: I/O operation on closed file.
Any pointers on why would this happen? Why would Twisted close the file early? Could it be that I need to increase the threadpool size?
I use treq 0.2.0 for the requests, with the "persistent" flag turned on, if that makes any difference. Twisted 13.2.0 and Django 1.6, python 2.7.1 on OS X 10.7.5.
Many thanks, Orestis