Proper way to pass Queue to process when using multiprocessing.imap()?
Israel Brewster
ijbrewster at alaska.edu
Tue Sep 3 15:09:25 EDT 2019
>
> On Sep 3, 2019, at 10:49 AM, Peter Otten <__peter__ at web.de> wrote:
>
> Israel Brewster wrote:
>
>> When using pool.imap to apply a function over a list of values, what is
>> the proper way to pass additional arguments to the function, specifically
>> in my case a Queue that the process can use to communicate back to the
>> main thread (for the purpose of reporting progress)? I have seen
>> suggestions of using starmap, but this doesn’t appear to have a “lazy”
>> variant, which I have found to be very beneficial in my use case. The
>> Queue is the same one for all processes, if that makes a difference.
>>
>> I could just make the Queue global, but I have always been told not too.
>> Perhaps this is an exception?
>
> How about wrapping the function into another function that takes only one
> argument? A concise way is to do that with functools.partial():
>
> def f(value, queue): ...
>
> pool.imap(partial(f, queue=...), values)
That looks like exactly what I was looking for. I’ll give it a shot. Thanks!
---
Israel Brewster
Software Engineer
Alaska Volcano Observatory
Geophysical Institute - UAF
2156 Koyukuk Drive
Fairbanks AK 99775-7320
Work: 907-474-5172
cell: 907-328-9145
>
>
>
>>
>> ---
>> Israel Brewster
>> Software Engineer
>> Alaska Volcano Observatory
>> Geophysical Institute - UAF
>> 2156 Koyukuk Drive
>> Fairbanks AK 99775-7320
>> Work: 907-474-5172
>> cell: 907-328-9145
>>
>
>
> --
> https://mail.python.org/mailman/listinfo/python-list
More information about the Python-list
mailing list