Multiprocessing.Queue - I want to end.

[Cameron Simpson]

On 01May2009 08:37, I wrote:
| On 30Apr2009 22:57, MRAB <google at mrabarnett.plus.com> wrote:
| > The producer could send just one None to indicate that it has finished
| > producing.
| > Each consumer could get the data from the queue, but if it's None then
| > put it back in the queue for the other consumer, then clean up and
| > finish.
| > When all the consumers have finished, the queue will contain just the
| > single None.
| 
| And as it happens I have an IterableQueue class right here which does
| _exact_ what was just described. You're welcome to it if you like.
| Added bonus is that, as the name suggests, you can use the class as
| an iterator:
|   for item in iterq:
|     ...
| The producer calls iterq.close() when it's done.

Someone asked, so code appended below.

Cameron Simpson <cs at zip.com.au> DoD#743
http://www.cskk.ezoshosting.com/cs/

class IterableQueue(Queue): 
  ''' A Queue obeying the iterator protocol.
      Note: Iteration stops when a None comes off the Queue.
  '''

  def __init__(self, *args, **kw):
    ''' Initialise the queue.
    '''
    Queue.__init__(self, *args, **kw)
    self.__closed=False

  def put(self, item, *args, **kw):
    ''' Put an item on the queue.
    '''
    assert not self.__closed, "put() on closed IterableQueue"
    assert item is not None, "put(None) on IterableQueue"
    return Queue.put(self, item, *args, **kw)

  def _closeAtExit(self):
    if not self.__closed:
      self.close()

  def close(self):
    ##logFnLine("%s.close()"%(self,),frame=sys._getframe(1))
    if self.__closed:
      # this should be a real log message
      print >>sys.stderr, "close() on closed IterableQueue"
    else:
      self.__closed=True
      Queue.put(self,None)

  def __iter__(self):
    ''' Iterable interface for the queue.
    '''
    return self

  def next(self):
    item=self.get()
    if item is None:
      Queue.put(self,None)      # for another iterator
      raise StopIteration
    return item