cool-RR wrote:
On Wed, Jan 20, 2010 at 2:08 AM, Raymond Hettinger <raymond.hettinger@gmail.com <mailto:raymond.hettinger@gmail.com>> wrote:
> Could it be made threadsafe?
To me, this is exactly the right line of questioning.
Is there a use case for iterating through a queue that is being mutated by consumers and producers? If so, should the consumers and producers be locked out during iteration or should the iterator fail if it detects a mutation (like we currently do for dictionaries that change during iteration).
In my program, I have one thread putting data into a queue, and the main thread periodically iterates over the whole queue to integrate all the data into a central data structure.
The general use case is when you want a thread to take items from a queue and process them until the queue's empty. Like:
for work_item in queue: process(work_item)
The semantics should be dictated by use cases. When someone writes "for t in q: action(q)" what is the most useful thing to happen when another thread does a get or put?
(I think you did a typo in your code sample, I think you meant `action(t)`)
What I'm suggesting is that the iteration won't care about what other threads do with the queue. It'll just `get` until it's empty.
I agree. I don't see why __iter__ wouldn't work when .get does. 'Empty' could mean either a non-blocking .get and stopping when there are no more items in the queue, or a blocking .get and stopping when there are no more items in the queue _and_ the queue has been closed by the producer(s).
My personal opinion is that queues are better-off without an iterator. They serve mainly to buffer consumers and producers and iteration does fit in well. In other words, adding __iter__ to queues in threaded environment may do more harm than good.
Raymond
Okay. I don't have much experience in this, so unless someone else will find this suggestion useful, I'll stick with my custom function.