Parallel/distributed generator

[George Sakkis]

On May 23, 3:35 pm, half.ital... at gmail.com wrote:
> On May 23, 11:00 am, George Sakkis <george.sak... at gmail.com> wrote:
>
>
>
> > I'm looking for any existing packages or ideas on how to implement the
> > equivalent of a generator (in the Python sense, i.e.http://www.python.org/dev/peps/pep-0255/) in a parallel/distributed
> > way. As a use case, imagine a function that generates a range of
> > primes. I'd like to be able to do something along the following lines:
>
> > def iterprimes(start=1, end=None):
> >    # ...
> >    yield prime
>
> > # rpc-related initialization
> > ...
> > rpc_proxy = some_rpc_lib(iterprimes, start=1e6, end=1e12)
> > for prime in proxy:
> >     print prime
>
> > Is there any module out there that does anything close to this ?
>
> > George
>
> Parellel Python?
>
> http://www.parallelpython.com/content/view/17/31/#SUM_PRIMES
>
> I've never used it, but looks like it does what you are looking for.

Looked promising, until I saw that it requires you to pass explicitly
the dependent functions and the modules to be imported for the
callable to work. I guess this has to be done recursively for all the
dependent functions/modules, which is impractical for anything but toy
examples.

George