[Python-ideas] `__iter__` for queues?

[Anh Hai Trinh]

> I was thinking of methods like these, for example:
>
> def close(self):
>    self.put(sentinel)
> def __iter__(self):
>    while True:
>        item = self.get()
>        if item is sentinel:
>            break
>        yield item
>    raise StopIteration

That would not be thread-safe because your sentinel is only seen once
by a single lucky consumer, other listeners who called iter() will
still execute self.get() and block forever.

The solution I found is to put the sentinel back in the queue so that
other consumers will see it (this use StopIteration as the sentinel):

def _iterqueue(queue):
  # Turn a either a threading.Queue or a multiprocessing.queues.SimpleQueue
  # into an thread-safe iterator which will exhaust when StopIteration is
  # put into it.
  while 1:
    item = queue.get()
    if item is StopIteration:
      # Re-broadcast, in case there is another listener blocking on
      # queue.get().  That listener will receive StopIteration and
      # re-broadcast to the next one in line.
      queue.put(StopIteration)
      break
    else:
      yield item

I agree, however, that this should be left out of the stdlib API.

-- 
// aht
http://blog.onideas.ws