On Feb 15, 2020, at 13:36, Jonathan Crall <erotemic@gmail.com> wrote:
Also, there is no duck-typed class that behaves like an executor, but does its processing in serial. Often times a develop will want to run a task in parallel, but depending on the environment they may want to disable threading or process execution. To address this I use a utility called a `SerialExecutor` which shares an API with ThreadPoolExecutor/ProcessPoolExecutor but executes processes sequentially in the same python thread:
This makes sense. I think most futures-and-executors frameworks in other languages have a serial/synchronous/immediate/blocking executor just like this. (And the ones that don’t, it’s usually because they have a different way to specify the same functionality—e.g., in C++, you only use executors via the std::async function, and you can just pass a launch option instead of an executor to run synchronously.) And I’ve wanted this, and even built it myself at least once—it’s a great way to get all of the logging in order to make things easier to debug, for example. However, I think you may have overengineered this. Why can’t you use the existing Future type as-is? Yes, there’s a bit of unnecessary overhead, but your reimplementation seems to add almost the same unnecessary overhead. And does it make enough difference in practice to be worth worrying about anyway? (It doesn’t for my uses, but maybe you’re are different.) Also, why are you overriding set_result to restore pre-3.8 behavior? The relevant change here seems to be the one where 3.8 prevents executors from finishing already-finished (or canceled) futures; why does your executor need that? Finally, why do you need a wrapper class that constructs one of the three types at initialization and then just delegates all methods to it? Why not just use a factory function that constructs and returns an instance of one of the three types directly? And, given how trivial that factory function is, does it even need to be in the stdlib? I may well be missing something that makes some of these choices necessary or desirable. But otherwise, I think we’d be better off adding a SerialExecutor (that works with the existing Future type as-is) but not adding or changing anything else.