Decorators for running a function in a Process or Thread
The decorators would abstract the logic of spawning a process or thread and maintaining its lifecycle. I think it could be a good fit for the `concurrent.futures` module. Decorated functions would return a `Future` object and run the logic in a separate thread or process. @concurrent.futures.thread def function(arg, kwarg=0): return arg + kwarg future = function(1, kwarg=2) future.result() The Process decorator in particular would support use cases such as running unstable code within an application taking advantage of process separation benefits. I often found myself relying on external APIs which would either crash or run indefinitely affecting my application stability and performance. @concurrent.futures.process(timeout=60): def unstable_function(arg, kwarg=0): # hang or segfault here return arg + kwarg future = unstable_function(1, kwarg=2) try: future.result() except TimeoutError as error: print("Function took more than %d seconds" % error.args[1]) except ProcessExpired as error: print("Process exit code %d" % error.exitcode) In case of timeout, the process would be terminated reclaiming back its resources. Few years ago I wrote a library for this purpose which turned out pretty handy in reducing the boilerplate code when dealing with threads and processes. It could be a starting point for discussing about the implementation. https://pypi.python.org/pypi/Pebble
On 1 May 2017 at 22:02, Paul Moore <p.f.moore@gmail.com> wrote:
It allows function designers to deliberately increase the friction of calling the function "normally". Consider an async library, for example - in such cases, it's useful to be able ensure that a blocking function never runs in the *current* thread, and instead always runs in a different one. One of the problems for the proposal is that we don't have the notion of a "default executor", the way we do with the default event loop in asyncio, so functions decorated with these would need to accept an additional parameter specifying the executor to use. Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia
On 1 May 2017 at 22:02, Paul Moore <p.f.moore@gmail.com> wrote:
It allows function designers to deliberately increase the friction of calling the function "normally". Consider an async library, for example - in such cases, it's useful to be able ensure that a blocking function never runs in the *current* thread, and instead always runs in a different one. One of the problems for the proposal is that we don't have the notion of a "default executor", the way we do with the default event loop in asyncio, so functions decorated with these would need to accept an additional parameter specifying the executor to use. Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia
participants (3)
-
Nick Coghlan -
NoxDaFox -
Paul Moore