multiprocessing signal defect

[Antoine Pitrou]

On Fri, 29 Oct 2010 10:08:01 -0400
Adam Tauno Williams <awilliam at whitemice.org> wrote:
> 
> No, I don't think so.  You're asking the module to over generalize
> behavior.  Reaping of the child is important, and that the child needs
> to be reaped may matter to the master child (why? did something go
> wrong?).  Silently reaping them [which would reduce the size of the
> Pool? Or would it dynamically create a new worker?] might have
> unintended side effects.  Maybe since Pool specifically generalizes
> child management you could make an argument it should reap, but I'm not
> sure.  Personally I'd recommend that your worker processes include a
> signal handler to do something smart in the case of a "-15" [for which
> there isn't really a thread equivalent - can you sent a SystemV style
> signal to an individual thread in a process?  I don't think so.]
> 
> How would a 'traditional' thread pool behave if a thread abended?  [of
> course, that depends on the thread-pool implementation]

A thread pool would certainly raise an exception in the caller (the
main thread probably) to signal the error.

I tried Neal's program (on 3.2 under Linux) and I don't witness the
same behaviour. The killed process disappears immediately, but the Pool
immediately spawns another process... which means you can't kill the
whole thing using Control-C (it kills one or both of the children but
they immediately get respawned, ad nauseam).

Another issue is that the respawned child doesn't seem to exhaust all
jobs, which means the parent waits indefinitely.

Regards

Antoine.