[Python-Dev] Re: Python multiplexing is too hard (was: Network
Mon, 22 May 2000 09:23:23 -0700 (PDT)
On Mon, 22 May 2000, Cameron Laird wrote:
> I've got a whole list of "higher-level
> abstractions around OS stuff" that I've been
> collecting. Maybe I'll make it fit for
> others to see once we're through this affair
Absolutely! I've thought about this too. A nice "child process
management" module would be very convenient to have -- i've done
such stuff before -- though i don't know enough about Windows
semantics to make one that works on multiple platforms. Some
sort of (hypothetical)
delegate.spawn(function) - return a child object or id
delegate.kill(id) - kill child
etc. could possibly free us from some of the system dependencies
of fork, signal, etc.
I currently have a module called "delegate" which can run a
function in a child process for you. It uses pickle() to send
the return value of the function back to the parent (via an
unnamed pipe). Again, Unix-specific -- but it would be very
cool if we could provide this functionality in a module. My
module provides just two things, but it's already very useful:
delegate.timeout(function, timeout) - run the 'function' in
a child process; if the function doesn't finish in
'timeout' seconds, kill it and raise an exception;
otherwise, return the return value of the function
delegate.parallelize(function, [work, work, work...]) -
fork off many children (you can specify how many if
you want) and set each one to work calling the 'function'
with one of the 'work' items, queueing up work for
each of the children until all the work gets done.
Return the results in a dictionary mapping each 'work'
item to its result.