[Python-ideas] `__iter__` for queues?

[cool-RR]

On Wed, Jan 20, 2010 at 2:08 AM, Raymond Hettinger <
raymond.hettinger at gmail.com> wrote:

>
> > Could it be made threadsafe?
>
> To me, this is exactly the right line of questioning.
>
> Is there a use case for iterating through a queue
> that is being mutated by consumers and producers?
> If so, should the consumers and producers be locked
> out during iteration or should the iterator fail if it detects
> a mutation (like we currently do for dictionaries that
> change during iteration).
>

In my program, I have one thread putting data into a queue, and the main
thread periodically iterates over the whole queue to integrate all the data
into a central data structure.

The general use case is when you want a thread to take items from a queue
and process them until the queue's empty. Like:

for work_item in queue:
    process(work_item)



> The semantics should be dictated by use cases.
> When someone writes "for t in q: action(q)" what
> is the most useful thing to happen when another
> thread does a get or put?
>

(I think you did a typo in your code sample, I think you meant `action(t)`)

What I'm suggesting is that the iteration won't care about what other
threads do with the queue. It'll just `get` until it's empty.



> My personal opinion is that queues are better-off without
> an iterator.  They serve mainly to buffer consumers and
> producers and iteration does fit in well.  In other words,
> adding __iter__ to queues in threaded environment
> may do more harm than good.
>
> Raymond
>

Okay. I don't have much experience in this, so unless someone else will find
this suggestion useful, I'll stick with my custom function.

Ram.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20100120/17adbc2d/attachment.html>