On Wed, Jan 20, 2010 at 2:08 AM, Raymond Hettinger < raymond.hettinger@gmail.com> wrote:
Could it be made threadsafe?
To me, this is exactly the right line of questioning.
Is there a use case for iterating through a queue that is being mutated by consumers and producers? If so, should the consumers and producers be locked out during iteration or should the iterator fail if it detects a mutation (like we currently do for dictionaries that change during iteration).
In my program, I have one thread putting data into a queue, and the main thread periodically iterates over the whole queue to integrate all the data into a central data structure. The general use case is when you want a thread to take items from a queue and process them until the queue's empty. Like: for work_item in queue: process(work_item)
The semantics should be dictated by use cases. When someone writes "for t in q: action(q)" what is the most useful thing to happen when another thread does a get or put?
(I think you did a typo in your code sample, I think you meant `action(t)`) What I'm suggesting is that the iteration won't care about what other threads do with the queue. It'll just `get` until it's empty.
My personal opinion is that queues are better-off without an iterator. They serve mainly to buffer consumers and producers and iteration does fit in well. In other words, adding __iter__ to queues in threaded environment may do more harm than good.
Raymond
Okay. I don't have much experience in this, so unless someone else will find this suggestion useful, I'll stick with my custom function. Ram.