[IPython-dev] Musings: syntax for high-level expression of parallel (and other) execution control

[Darren Dale]

Hi Fernando,

On Fri, Sep 4, 2009 at 4:31 AM, Fernando Perez<fperez.net at gmail.com> wrote:
> # This is the actual code of the decorator-based loop:
> def loop_deco():
>    results = [None]*count
>
>    @for_each(range(count))
>    def loop(i):
>        results[i] = do_work(data, i)
>
>    return summarize(results, count)

If I have understood correctly, this example passes around a lot more
data than is necessary. for_each passes the whole data list to
do_work, which picks off the bit its looking for. I guess this doesn't
seem like an intuitive interface to me, plus its expensive. What is
being parallelized is:

results = [None]*len(data)
for i, d in enumerate(data):
    results[i] = do_work(d)

Why not use a simple function instead of a decorator:

results = for_each(data, do_work)

?

Darren