[Twisted-Python] Accessing the request object from a different thread
I have an application that uses synchronous access to a MySQL database through SQLAlchemy. Unwilling to part with this synchronous access, our application uses threads to perform database I/O outside of the main Twisted thread. It would be nice to still have some access to the request object while processing a request in a new thread. Is there any thread-safe way to access the request object data? Here is a typical Resource subclass: def parseRequest(self, request, session): try: # parse the relevant portion of the path namingPortionOfPath = self.idParserRE.match(request.path).group(1) cls = self.dbClass() return cls.getAllInstances(namingPortionOfPath) except: return None return None def render_GET(self, request): def sessionFn(session): instance = self.parseRequest(request, session) rendered = None if instance: stream = self.template.generate(object=instance, resource=self, session=session) rendered = stream.render('html', doctype='html', encoding='latin1') else: rendered = "finished" return rendered def otherThreadFn(): return pulse.web.mainStore().withSession(sessionFn) def deferedCallback(r): request.write(r) request.finish() twisted.internet.threads.deferToThread(otherThreadFn).addCallback(deferedCallback) return NOT_DONE_YET All of the accesses to the request object are read operations. If those read operations occur on data that does not get messed with by the main thread, then this should be thread-safe code. However, it seems that the official word is "do not use request objects from other threads." Maybe I am mistaken. In any case, is there a safe way to get at request headers and parameter values outside of the reactor loop. Thanks, Red
All of the accesses to the request object are read operations. If those read operations occur on data that does not get messed with by the main thread, then this should be thread-safe code. However, it seems that the official word is "do not use request objects from other threads." Maybe I am mistaken. In any case, is there a safe way to get at request headers and parameter values outside of the reactor loop.
The rule is not to call any twisted functions IIRC. You can do this safely: args_copy = dict(request.args) hdr_copy = dict(request.requestHeaders.getAllRawHeaders()) deferToThread(func, args_copy, hdr_copy)
participants (2)
-
Phil Mayers
-
Red Daly