[Python-ideas] `__iter__` for queues?

MRAB python at mrabarnett.plus.com
Wed Jan 20 00:52:42 CET 2010

Cameron Simpson wrote:
> On 19Jan2010 23:01, MRAB <python at mrabarnett.plus.com> wrote:
> | Georg Brandl wrote:
> | >Am 19.01.2010 21:20, schrieb cool-RR:
> | >>On Tue, Jan 19, 2010 at 11:10 PM, Simon Brunning <simon at brunningonline.net>:
> | >>    2010/1/19 cool-RR <cool-rr at cool-rr.com>:
> | >>    > Is there a reason that queues don't have an `__iter__` method? I
> | >>    > mean both `Queue.Queue` and `multiprocessing.Queue`.
> | >>
> | >>    Could it be made threadsafe?
> | >>
> | >>For me, iterating on the queue means just calling `get` repeatedly until
> | >>it's empty. Now that I think about it, maybe this is not the most
> | >>obvious meaning? I'm not sure now.
> |
> | >Your obvious queue iterator would call get(block=False) and stop on Empty.
> | >The other obvious meaning is be to call get(block=True) forever.  IMO they
> | >are both too "obvious" to make a call -- an explicit while loop is better.
> | >
> | To me the 'obvious' meaning is to call get(block=True) and have it raise
> | Empty (actually, StopIteration) when the queue is empty and the 'sender'
> | has somehow signalled that no more items will be put into the queue
> | (q.finished()?). This would also eliminate the need for a sentinel!
> Personally, I have long had an IterableQueue subclass, but it uses a
> tunable sentinel (None by default).
> But it adds a .close() method, and doesn't iterate until empty, it
> iterates until closed. This is because I want to write a handler like
> so:
>   for item in Q:
>     ...do stuff...
> and have it block if the queue is empty.
> So clearly there are two reasonable approaches to the end-of-iteration
> idea; extending Queue to do iteration probably would want to choose one.
> So maybe two helper iteration methods might be the go: it's easy enough
> to write a generator to iterate-until-empty or iterate-until-sentinel.
Ah, yes, .close() would be better. :-)

As for iterate-until-empty or iterate-until-sentinel, perhaps only a
keyword argument is needed when the queue is created.

Another possible feature (although this might be going too far) is a
keyword argument for the number of 'producers' (default=1). Each can
close the queue when it's finished, and StopIteration will be raised
only when all have closed and the queue is empty.

More information about the Python-ideas mailing list