Proper way to pass Queue to process when using multiprocessing.imap()?
Peter Otten
__peter__ at web.de
Tue Sep 3 14:49:20 EDT 2019
Israel Brewster wrote:
> When using pool.imap to apply a function over a list of values, what is
> the proper way to pass additional arguments to the function, specifically
> in my case a Queue that the process can use to communicate back to the
> main thread (for the purpose of reporting progress)? I have seen
> suggestions of using starmap, but this doesn’t appear to have a “lazy”
> variant, which I have found to be very beneficial in my use case. The
> Queue is the same one for all processes, if that makes a difference.
>
> I could just make the Queue global, but I have always been told not too.
> Perhaps this is an exception?
How about wrapping the function into another function that takes only one
argument? A concise way is to do that with functools.partial():
def f(value, queue): ...
pool.imap(partial(f, queue=...), values)
>
> ---
> Israel Brewster
> Software Engineer
> Alaska Volcano Observatory
> Geophysical Institute - UAF
> 2156 Koyukuk Drive
> Fairbanks AK 99775-7320
> Work: 907-474-5172
> cell: 907-328-9145
>
More information about the Python-list
mailing list