waste of resources ?
Arne Mueller
a.mueller at icrf.icnet.uk
Wed Jun 9 11:22:41 EDT 1999
Carey Evans wrote:
>
> Arne Mueller <a.mueller at icrf.icnet.uk> writes:
>
> [snip]
>
> > > def sigchild_handler(signum, frame):
> > > os.wait()
>
> [snip]
>
> > Hm, that doesn't work properly with my program. THe number of zombie
> > processes is steadily at about 20, which means there are 20 unhandled
> > died children.
>
> Unix doesn't guarantee you'll get exactly one signal delivered per
> event. To be sure to get all the children, you need to loop over the
> processes with waitpid().
>
> I think something like this should do it:
>
> ----------
> def sigchld_handler(signum, frame):
> print "Checking for processes..."
> try:
> while 1:
> pid, sts = os.waitpid(-1, os.WNOHANG)
> if pid == 0:
> break
> print "Process %d ended with status %04x." % (pid, sts)
> except OSError, detail:
> if detail.args[0] != errno.ECHILD:
> raise
> else:
> print "No more children."
> else:
> print "Some children yet to finish."
> ----------
Hm, that's similar what I do in my program but I always have as many
zombies as I fork processes, here's a program fragment:
cpu = 4
used = 0
for fn in files:
if cpu:
if used < cpu:
r, w = os.pipe()
pid = os.fork()
used = used + 1
if pid: # parent
os.close(w)
chfds.append(r)
chpids[r] = pid # dict. with filedescriptor as key
# fork children for available processors
if used < cpu and fn != last: continue
else: # child
os.close(r)
param['files'] = [fn]
# some application specific stuff
msa = ProcessMSA(param)
msa.of = None
msa.dbkeys = {}
msa.param = {}
# pickle objecty and send it to parent
p = mpickle.dumps(msa)
r, w, o = select.select([], [w], [])
os.write(w[0], p)
os._exit(0)
# parent reads from ready file descriptors
while used >= cpu or (fn == last and used):
r, w, o = select.select(chfds, [], [])
for i in r:
msa = mpickle.loads(readPickleStream(i))
os.close(i)
stderr.write('number %d\n' % len(stats.ids.keys()))
stats.add(msa) # do something with the received
object
used = used - 1 # adjust number of used processors
chfds.remove(i)
os.waitpid(chpids[i], os.WNOHANG)
del chpids[i]
Why does that code produce zombies? Do I've to send a signal to the
child that it has to die? I mean maybe ther child exits when the parent
hasn't reached the os.waitpid?
Thanks for discussion,
Arne
More information about the Python-list
mailing list