An iteration idiom (Was: Re: [Guppy-pe-list] loading files containing multiple dumps)

[Sverker Nilsson]

A question arose on guppy-pe-list about how to iterate over objects
returned one by one by a method (load) called repeatedly. I defined a
generator to do this (loadall), but this seems unwieldy in general. Is
there a common idiom here that can usefully be encapsulated in a general
method?

On Mon, 2009-08-31 at 17:57 +0100, Chris Withers wrote: 
> Sverker Nilsson wrote:
> > It reads one Stat object at a time and wants to report something
> > when there is no more to be read from the file. 
> 
> Hmm, am I right in thinking the above can more nicely be written as:
> 
>  >>> from guppy import hpy
>  >>> h = hpy()
>  >>> f = open(r'your.hpy')
>  >>> sets = []
>  >>> for s in iter(h.load(f)): sets.append(s)
> ...
> 
The above iterates over one Stat object returned by h.load(f). I assume
you want to iterate over all the objects loaded.

A way to iterate over the objects (not having to define the loadall()
function from previous mail) that I came up with now, is via itertools
imap with h.load applied to an infinite iterator repeating the file:

from guppy import hpy
from itertools import imap, repeat
h=hpy()
f=open('your.hpy')
sets = list(imap(h.load,repeat(f)))

Maybe the above idiom could usefully be encapsulated in a standard
function? 

def iapply(func, *args):
        '''Iterator repeatedly calling func with *args, infinitely or
until it raises StopIteration'''
        from itertools import imap, repeat
        return imap(func, *[repeat(arg) for arg in args])

Usage, eg: 

>>> sets = list(iapply(h.load,f))

or

>>> for s in iapply(h.load,f):
    ...

What do you think, should something like iapply be added to itertools?
What is the better name? :-)

Sverker

-- 
Expertise in Linux, embedded systems, image processing, C, Python...
        http://sncs.se