Atilla wrote:
When it comes to deferToThread - is that actually going to be of any use? Do I get an advantage of running the "send" loop in a seperate thread? What if, say, I never had to send more than 10-50 messages, so the loop is relatively small - then I'd have to keep spawning threads for no reason.
Exactly, no reason. Don't spawn threads, unless you have no other way.
But - when it comes to large send queues, how can that be chopped in an efficient manner? Chop it into equal pieces and schedule them with small delay, one after the other?
Yes, and the delay can be very small. Send 50-100 messages at a time, then run the next batch with a reactor.callLater(0.00000001, ...). That way the reactor will be able to handle network events with low latency. Or just use coiterate, as Manlio said: http://twistedmatrix.com/documents/current/api/twisted.internet.task.html It works in 10ms batches: that's your latency. It also uses a 10us delay on the callLater.
Defer them to another thread?
Don't. Only use threads in Twisted if you have to call external, synchronous libraries, like database clients. -- Nicola Larosa - http://www.tekNico.net/ Restricting yourself to the intersection of the two versions [Python 2.6 and 3.0] is very painful and limited. We're not introducing backwards compatibility syntax in 3.0, because that would defeat the purpose (we've had backwards compatibility forever in 2.x, and the whole point of 3.0 is to clean up the mess). -- Guido van Rossum, July 2007