Multiprocessing.Queue - I want to end.
cs at zip.com.au
Fri May 1 07:22:26 CEST 2009
On 01May2009 08:37, I wrote:
| On 30Apr2009 22:57, MRAB <google at mrabarnett.plus.com> wrote:
| > The producer could send just one None to indicate that it has finished
| > producing.
| > Each consumer could get the data from the queue, but if it's None then
| > put it back in the queue for the other consumer, then clean up and
| > finish.
| > When all the consumers have finished, the queue will contain just the
| > single None.
| And as it happens I have an IterableQueue class right here which does
| _exact_ what was just described. You're welcome to it if you like.
| Added bonus is that, as the name suggests, you can use the class as
| an iterator:
| for item in iterq:
| The producer calls iterq.close() when it's done.
Someone asked, so code appended below.
Cameron Simpson <cs at zip.com.au> DoD#743
''' A Queue obeying the iterator protocol.
Note: Iteration stops when a None comes off the Queue.
def __init__(self, *args, **kw):
''' Initialise the queue.
Queue.__init__(self, *args, **kw)
def put(self, item, *args, **kw):
''' Put an item on the queue.
assert not self.__closed, "put() on closed IterableQueue"
assert item is not None, "put(None) on IterableQueue"
return Queue.put(self, item, *args, **kw)
if not self.__closed:
# this should be a real log message
print >>sys.stderr, "close() on closed IterableQueue"
''' Iterable interface for the queue.
if item is None:
Queue.put(self,None) # for another iterator
More information about the Python-list