On Wed, 13 Nov 2002 22:19:34 +0100
"Matthias Urlichs"
I find myself more in immediate need of a queueing mechanism; the quick-and-dirty solution seems to be to fork off two threads and to use a bounded Python Queue. However, the continuous switching between the reader(which blocks on the queue, therefore needs to be in its own thread) and Twisted's main thread (which writes to the process-or-whatever that's going to eventually consume the data) seems rather sub-optimal for performance. :-/
I don't see why you need a thread for queueing - a simple list would do, or a class wrapping one. Or look at transport's producer/consumer interface if this is for a protocol. Adding to queue is appending to list, dequeuing is getting item from beginning of list. If, and *only* if you need threads - i.e. you are doing either blocking operations or really long running calculations, there are some options for you in Twisted. We have a thread pool dispatcher that queues methods to be run in the thread pool, that covers some the use cases. If you want to use a queue I suggest just sticking to Python's Queue or the slighly nicer twisted.python.timeoutqueue. In general using threads in Python is slow anyway do to the global interpreter lock, which means in most cases only one thread at a time is running anyway. That being said, Queue.Queue is rather slow in and of itself, and can be implemented in a much faster way by using one lock, and use mxQueue. -- Itamar Shtull-Trauring http://itamarst.org/ Available for Python, Twisted, Zope and Java consulting ***> http://VoteNoWar.org -- vote/donate/volunteer <***