[Python-ideas] fork

[Terry Reedy]

On 7/29/2015 12:46 PM, Sven R. Kunze wrote:
> Hi everybody,
>
> well during the discussion of the concurrency capabilities of Python, I
> found this article reading worthwhile:
> http://chriskiehl.com/article/parallelism-in-one-line/

I found this very helpful.

> That is sequential code (almost plain English):
>
> for image in images:
>      create_thumbnail(image)

Write this more succinctly as

map(create_thumbnail, images)

> In order to have a start with parallelism and concurrency, we need to do
> the following:
>
> pool = Pool()
> pool.map(create_thumbnail, images)
> pool.close()
> pool.join()

and define

def pmap(func, iterable, *args, **kwargs):
     pool = Pool(*args, **kwargs)
     pool.map(func, iterable)
     pool.close()
     pool.join()

then the replacement requires only 1 char.

pmap(create_thumbnail, images)

This is, of course, limited to making exactly one .map call and closing, 
but if this is the common case, it might be sensible to request that 
this be added to multiprocessing (and m.dummy) as a utility function.

> Not bad (considering the other approaches), but why couldn't it not look
> just like the sequential one, maybe like this:
>
> for image in images:
>      fork create_thumbnail(image)

An new keyword, which is a pain it itself, cannot take arguments.
Blogger Chris Kiehl why they are needed.

> What I like about the Pool concept is that it frees me of thinking about
> the interprocess/-thread communication and processes/threads management

A keyword would not offer the choice of threads versus processes.

> What I would like to be freed of as well is: pool management.

Then use the wrapper function above.

-- 
Terry Jan Reedy