[Twisted-Python] Newbie question: Is this possible with twisted?
Hi, first of all: I've no experience with network-programming yet, but need to go in for creating a "simple" client-server application. In the end I'd like to connect a web-form (django) with a twisted-server, which then should do some long-lasting calculations. So far I got a simple xmlrpc-server, which "defers" the calculation to a new thread and a client which sends the initial request to do the calculation. Though I have the problem that the client stays connected until the calculations are done. Could someone tell me if it's somehow possible to disconnect the client right after sending the request to the server? Thanks, Tim
Hi Tim,
first of all: I've no experience with network-programming yet, but need to go in for creating a "simple" client-server application.
No problem.
In the end I'd like to connect a web-form (django) with a twisted-server, which then should do some long-lasting calculations.
So far I got a simple xmlrpc-server, which "defers" the calculation to a new thread and a client which sends the initial request to do the calculation. Though I have the problem that the client stays connected until the calculations are done.
Keep in mind you can run this "calculation server" as a separate server-process and connect to it via loopback (localhost:SOME_PORT). This makes it easier to put it on another machine one day without changing the code. Also, if you're doing a blocking calculation in Python (i.e. if it's not a special C-module), then because of the Python Global Interpreter Lock (GIL), you're not going to be able to paralllelise that calculation effectively.
Could someone tell me if it's somehow possible to disconnect the client right after sending the request to the server?
Well, it depends. Does the client need the result of that calculation? Probably not, so what you can do is issue a job id back to the client, and associate the job id to the result of that job. This is all very doable in Twisted. However, I'm going to suggest something different, which seems like a better fit for what you're trying to do. Check out Celery (http://celeryproject.org/) which is a distributed task queue. The idea is that you set up a queue system, like RabbitMQ, which implement a job queue via Celery. You write tasks as Python functions in a module somewhere. When a client connects, you issue a task, and return the task id back to the client. By and by the job is picked up by some worker, and the result is stored in some persistent database, like MySQL or Redis. Celery also has excellent integration with Django. Hope that helps you out. Cheers, Reza -- Reza Lotun mobile: +44 (0)7521 310 763 email: rlotun@gmail.com work: reza@tweetdeck.com twitter: @rlotun
Hi Reza.
However, I'm going to suggest something different, which seems like a better fit for what you're trying to do. Check out Celery (http://celeryproject.org/) which is a distributed task queue. The idea is that you set up a queue system, like RabbitMQ, which implement a job queue via Celery. You write tasks as Python functions in a module somewhere. When a client connects, you issue a task, and return the task id back to the client. By and by the job is picked up by some worker, and the result is stored in some persistent database, like MySQL or Redis. Celery also has excellent integration with Django.
Great, this seems to be exactly what I am searching for. :) Cheers, Tim
participants (2)
-
Reza Lotun -
Tim Bordemann