[issue11271] concurrent.futures.ProcessPoolExecutor.map() doesn't batch function arguments by chunks

Dan O'Reilly report at bugs.python.org
Thu Jul 31 04:00:37 CEST 2014


Dan O'Reilly added the comment:

re: Diverging ThreadPoolExecutor and ProcessPoolExecutor APIs. I thought about this as well. It would of course be possible to make ThreadPoolExecutor's API match, but it would serve no useful purpose that I can think of. I guess we could make the ThreadPoolExecutor API accept the chunksize argument just so the APIs match, but then not actually use it (and document that it's not used, of course). That would make it easier to switch between the two APIs, at least.

On the other hand, the two APIs are slightly different already: ThreadPoolExecutor requires max_workers be supplied, while ProcessPoolExecutor will default to multiprocessing.cpu_count(). So this wouldn't completely be without precedent.

Anyway, I will review your comments on the patch, add unit tests, and re-submit. Thanks!

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue11271>
_______________________________________


More information about the Python-bugs-list mailing list