[Python-ideas] Add `n_threads` argument to `concurrent.futures.ProcessPoolExecutor`

Andrew Barnert abarnert at yahoo.com
Tue Jan 21 13:27:36 CET 2014


On Jan 21, 2014, at 4:20, Andrew Barnert <abarnert at yahoo.com> wrote:

> And this is very easy to solve: run the downloads on a thread pool, and as each one finishes, kick its post processing off to a process pool.

Wait, that's stupid. Even simpler: just use a flat process pool of 2N for everything (or whatever multiplier is appropriate for your load--although often a downloader doesn't want to do more than about 4-12 simultaneous downloads, which is already below 2N on most modern computers...).


More information about the Python-ideas mailing list