I am almost done with my web app. Now just before finalizing the project I noticed that when I request two pages at the same time (e.g. double clicking on a link) my app including twisted just exits.
I am almost certain that this behavior has something to do with the fact that I put every request in a seperate thread. My render methode looks pretty much like this:
def render(self, request): threads.deferToThread(handleRequest, request).addCallback(\ writeRequestAndFinish, request).addErrback(\ request.processingFailed) return server.NOT_DONE_YET
Any idea why this does not work? Do you think there is a problem with accessing the request variable from a different thread?
_stephan
Well, the request object certainly doesn't do any locking for multithreadeded access, so it could easily be in an inconsistant state sometimes. I'd certainly suggest not accessing the request object from a different thread, but, that shouldn't cause python to just up and quit without any error.
Maybe you can try running it with the python debugger, or if that doesn't catch anything, gdb. Are you using any external C libraries? (database/etc?) Perhaps one of those isn't multithreaded safe?
James
On Feb 24, 2004, at 2:19 PM, stephan wrote:
def render(self, request): threads.deferToThread(handleRequest, request).addCallback(\ writeRequestAndFinish, request).addErrback(\ request.processingFailed) return server.NOT_DONE_YET
I do use external libs, but they are all instanced within the thread. I create a connection to a mysql dbm within each thread but that should not cause any conflicts since each thread has its seperate connection object.
Personally I suspect the request object. Unfortunately I cannot just deepcopy it and pass it to the thread.
Is there an alternative way doing this? The only way I can imagine is to do the threading later down the line which would make it harder to maintain a version that also runs within apache.
_stephan
On Tue, 24 Feb 2004 14:36:03 -0500, James Y Knight foom@fuhm.net wrote:
Well, the request object certainly doesn't do any locking for multithreadeded access, so it could easily be in an inconsistant state sometimes. I'd certainly suggest not accessing the request object from a different thread, but, that shouldn't cause python to just up and quit without any error.
Maybe you can try running it with the python debugger, or if that doesn't catch anything, gdb. Are you using any external C libraries? (database/etc?) Perhaps one of those isn't multithreaded safe?
James
On Feb 24, 2004, at 2:19 PM, stephan wrote:
def render(self, request): threads.deferToThread(handleRequest, request).addCallback(\ writeRequestAndFinish, request).addErrback(\ request.processingFailed) return server.NOT_DONE_YET
Twisted-web mailing list Twisted-web@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-web
stephan wrote:
I do use external libs, but they are all instanced within the thread. I create a connection to a mysql dbm within each thread but that should not cause any conflicts since each thread has its seperate connection object.
Personally I suspect the request object. Unfortunately I cannot just deepcopy it and pass it to the thread.
Is there an alternative way doing this? The only way I can imagine is to do the threading later down the line which would make it harder to maintain a version that also runs within apache.
If you make _any_ calls to Twisted code, put them in a reactor.callFromThread(func). It runs the function in the main thread.
But you shouldn't suspect the request object, because plain Python code won't make python simply die without error. Third party C code is almost always the culprit in stuff like that.
I see what you are saying: Python would throw an error in case of a conflictive access to a shared object.
Relating to your first suggestion: what do you excactly mean with calling twisted code in the main thread with reactor.callFromThread(func)? Can you give an example? When I retrieve my cookie should I do that with reactor.callFromThread(func)?
_stephan
On Tue, 24 Feb 2004 18:35:17 -0500, Christopher Armstrong radix@twistedmatrix.com wrote:
stephan wrote:
I do use external libs, but they are all instanced within the thread. I create a connection to a mysql dbm within each thread but that should not cause any conflicts since each thread has its seperate connection object.
Personally I suspect the request object. Unfortunately I cannot just deepcopy it and pass it to the thread.
Is there an alternative way doing this? The only way I can imagine is to do the threading later down the line which would make it harder to maintain a version that also runs within apache.
If you make _any_ calls to Twisted code, put them in a reactor.callFromThread(func). It runs the function in the main thread.
But you shouldn't suspect the request object, because plain Python code won't make python simply die without error. Third party C code is almost always the culprit in stuff like that.
participants (3)
-
Christopher Armstrong -
James Y Knight -
stephan