[Python-ideas] `__iter__` for queues?
cool-rr at cool-rr.com
Wed Jan 20 01:24:47 CET 2010
On Wed, Jan 20, 2010 at 2:08 AM, Raymond Hettinger <
raymond.hettinger at gmail.com> wrote:
> > Could it be made threadsafe?
> To me, this is exactly the right line of questioning.
> Is there a use case for iterating through a queue
> that is being mutated by consumers and producers?
> If so, should the consumers and producers be locked
> out during iteration or should the iterator fail if it detects
> a mutation (like we currently do for dictionaries that
> change during iteration).
In my program, I have one thread putting data into a queue, and the main
thread periodically iterates over the whole queue to integrate all the data
into a central data structure.
The general use case is when you want a thread to take items from a queue
and process them until the queue's empty. Like:
for work_item in queue:
> The semantics should be dictated by use cases.
> When someone writes "for t in q: action(q)" what
> is the most useful thing to happen when another
> thread does a get or put?
(I think you did a typo in your code sample, I think you meant `action(t)`)
What I'm suggesting is that the iteration won't care about what other
threads do with the queue. It'll just `get` until it's empty.
> My personal opinion is that queues are better-off without
> an iterator. They serve mainly to buffer consumers and
> producers and iteration does fit in well. In other words,
> adding __iter__ to queues in threaded environment
> may do more harm than good.
Okay. I don't have much experience in this, so unless someone else will find
this suggestion useful, I'll stick with my custom function.
-------------- next part --------------
An HTML attachment was scrubbed...
More information about the Python-ideas