[Twisted-Python] Better usage of Twisted on multi core processor
![](https://secure.gravatar.com/avatar/4b7641b56701735da04364fffb9e5675.jpg?s=120&d=mm&r=g)
Hi all, I have a twisted TCP server to listens to client, processes requests, do mysql database operations if needed (using adbapi Connection pool) and return the result. Before deploying this in production, I want to know right way to configure the server. Since twisted is single threaded, how can I leverage multiple cores of my production machine (which has 6 cores and 16 GB RAM) ? One approach that I thought of was to start multiple instances of twisted server on different ports. This would help in using the other cores as well. What do you guys suggest ? Thanks -Sumanth
![](https://secure.gravatar.com/avatar/1846c8040fcf70e9b55bb7bfcdb78bc4.jpg?s=120&d=mm&r=g)
Hi Sumanth, The first answer by JP Calderone (exarkun) on the following stackoverflow question may be of some assistance. http://stackoverflow.com/questions/10077745/twistedweb-on-multicore-multipro... This answer is from 2012; the good news is that the good API that JP talks about in his answer is now available in released Twisteds IIUC :) cheers lvh
![](https://secure.gravatar.com/avatar/e1554622707bedd9202884900430b838.jpg?s=120&d=mm&r=g)
There's nothing special about a .tac file with respect to spawning processes, it's just another way to instantiate a Service object. You just need to change the spawned process's arg list from "[executable, __file__, str(port.fileno())]" to something like "[executable, 'twistd', '-n', '-y', __file__, str(port.fileno())]". -glyph
![](https://secure.gravatar.com/avatar/e1554622707bedd9202884900430b838.jpg?s=120&d=mm&r=g)
On Mar 13, 2015, at 3:03 AM, Nagy, Attila <bra@fsn.hu> wrote:
Sounds logical. Do you plan a nicer interface for this maybe? :)
Plan? Not right now. Hope for? Yes, for many years :). I would very much like to have Twisted contain a straightforward process-pool interface, perhaps a deferToProcess. But attempting to guess when this will happen is a fool's errand. Perhaps you could help us do it, though! :) -glyph
![](https://secure.gravatar.com/avatar/1846c8040fcf70e9b55bb7bfcdb78bc4.jpg?s=120&d=mm&r=g)
Hi Sumanth, The first answer by JP Calderone (exarkun) on the following stackoverflow question may be of some assistance. http://stackoverflow.com/questions/10077745/twistedweb-on-multicore-multipro... This answer is from 2012; the good news is that the good API that JP talks about in his answer is now available in released Twisteds IIUC :) cheers lvh
![](https://secure.gravatar.com/avatar/e1554622707bedd9202884900430b838.jpg?s=120&d=mm&r=g)
There's nothing special about a .tac file with respect to spawning processes, it's just another way to instantiate a Service object. You just need to change the spawned process's arg list from "[executable, __file__, str(port.fileno())]" to something like "[executable, 'twistd', '-n', '-y', __file__, str(port.fileno())]". -glyph
![](https://secure.gravatar.com/avatar/e1554622707bedd9202884900430b838.jpg?s=120&d=mm&r=g)
On Mar 13, 2015, at 3:03 AM, Nagy, Attila <bra@fsn.hu> wrote:
Sounds logical. Do you plan a nicer interface for this maybe? :)
Plan? Not right now. Hope for? Yes, for many years :). I would very much like to have Twisted contain a straightforward process-pool interface, perhaps a deferToProcess. But attempting to guess when this will happen is a fool's errand. Perhaps you could help us do it, though! :) -glyph
participants (5)
-
Glyph
-
Glyph Lefkowitz
-
Laurens Van Houtven
-
Nagy, Attila
-
Sumanth Puram