[Python-Dev] Re: Python multiplexing is too hard (was: Network statistics program)

[Ka-Ping Yee]

On Mon, 22 May 2000, Cameron Laird wrote:
> I've got a whole list of "higher-level
> abstractions around OS stuff" that I've been
> collecting.  Maybe I'll make it fit for
> others to see once we're through this affair

Absolutely!  I've thought about this too.  A nice "child process
management" module would be very convenient to have -- i've done
such stuff before -- though i don't know enough about Windows
semantics to make one that works on multiple platforms.  Some
sort of (hypothetical)

    delegate.spawn(function) - return a child object or id
    delegate.kill(id) - kill child

etc. could possibly free us from some of the system dependencies
of fork, signal, etc.

I currently have a module called "delegate" which can run a
function in a child process for you.  It uses pickle() to send
the return value of the function back to the parent (via an
unnamed pipe).  Again, Unix-specific -- but it would be very
cool if we could provide this functionality in a module.  My
module provides just two things, but it's already very useful:

    delegate.timeout(function, timeout) - run the 'function' in
        a child process; if the function doesn't finish in
        'timeout' seconds, kill it and raise an exception;
        otherwise, return the return value of the function

    delegate.parallelize(function, [work, work, work...]) -
        fork off many children (you can specify how many if
        you want) and set each one to work calling the 'function'
        with one of the 'work' items, queueing up work for
        each of the children until all the work gets done.
        Return the results in a dictionary mapping each 'work'
        item to its result.


-- ?!ng