Yes, thread pools are unfortunately necessary evils.

Twisted comes with a few tools to handle the use cases we're discussing. The 1:1 equivalent for call_on_thread would be deferToThread/deferToThreadPool (deferToThread == deferToThreadPool except with the default thread pool instead of a specific one).

There are a few other tools:

- spawnProcess (equiv to subprocess module, except with async communication with the subprocess)
- cooperative multitasking, such (twisted.internet.task.) Cooperator and coiterate: basically resumable tasks that are explicit about where they can be paused/resumed
- third party tools such as corotwine, giving stackless-style coroutines, or ampoule, giving remote subprocesses

The more I learn about other stuff the more I see that everything is the same because everything is different :)

On Sat, Oct 27, 2012 at 12:27 PM, Kristján Valur Jónsson <kristjan@ccpgames.com> wrote:

Yes, stacklesslib provides this functionality with the call_on_thread() api, which turns a blocking operation into a non-blocking one. This is also useful for cpu bound operations, btw. For example, in EVE, when we need to do file operations and zipping of local files, we do it using this api.
K

--
cheers

lvh