On Sat, 15 Feb 2020 14:16:39 -0800 Andrew Barnert via Python-ideas <python-ideas@python.org> wrote:
On Feb 15, 2020, at 13:36, Jonathan Crall <erotemic@gmail.com> wrote:
Also, there is no duck-typed class that behaves like an executor, but does its processing in serial. Often times a develop will want to run a task in parallel, but depending on the environment they may want to disable threading or process execution. To address this I use a utility called a `SerialExecutor` which shares an API with ThreadPoolExecutor/ProcessPoolExecutor but executes processes sequentially in the same python thread:
This makes sense. I think most futures-and-executors frameworks in other languages have a serial/synchronous/immediate/blocking executor just like this. (And the ones that don’t, it’s usually because they have a different way to specify the same functionality—e.g., in C++, you only use executors via the std::async function, and you can just pass a launch option instead of an executor to run synchronously.)
FWIW, I agree with Andrew here. Being able to swap a ThreadPoolExecutor or ProcessPoolExecutor with a serial version using the same API can have benefits in various situations. One is easier debugging (in case the problem you have to debug isn't a race condition, of course :-)). Another is writing a library a command-line tool or library where the final decision of whether to parallelize execution (e.g. through a command-line option for a CLI tool) is up to the user, not the library developer. It seems there are two possible design decisions for a serial executor: - one is to execute the task immediately on `submit()` - another is to execute the task lazily on `result()` This could for example be controlled by a constructor argument to SerialExecutor. Regards Antoine.