Whether to call Py_Finalize when exiting from the child process of a fork from a spawned thread
Hi all, I'm working on http://bugs.python.org/issue6642 for unladen swallow (because it happens to bite us in a weird way), and Jeff Yasskin told me to ask python-dev what the proper behavior should be at exit from the child process of a fork from a spawned thread. Right now, it seems that there is an assumption that when exiting the process, control will always return through Py_Main, which in turn calls Py_Finalize, to do things like GC and calling atexit handlers. Normally, if you fork a process from the main thread, this assumption will remain true, because main and Py_Main are still at the bottom of the stack in the child process. However, if you fork from a spawned thread, then the root of the stack will be the thread bootstrap routine for the platform's pythreads implementation. On one hand, you may not want to call the user's atexit handlers multiple times from different processes if they have externally visible effects. On the other hand, people seem to assume that Py_Finalize will be called at process exit to do various cleanups. On the third hand, maybe Python could just clear out all the atexit handlers in the child after a fork. So what should the correct behavior be? Thanks, Reid
Standard POSIX fork semantics should be a guidance. IIUC, termination of the last thread is equivalent to calling exit(0) (although return from main() still means that exit is invoked right away, and the return value of main is the exit code - right?). Calling exit means to call all exit handlers. So to match POSIX semantics, we should also call the exit handlers in the child process (along with invoking a final garbage collection etc.) Regards, Martin
On Tue, Sep 1, 2009 at 2:58 PM, "Martin v. Löwis"<martin@v.loewis.de> wrote:
It depends, there is also _exit, which exists solely for the purpose of working around exit handlers being called from a forked child process at exit. Which semantics should Python have? In my opinion, it is more obvious that the user's handlers would be called than not, so I agree with you. Reid
I don't think so. There are other cases where you don't want to run atexit handlers, eg. in a segfault signal handler. _exit has been there "forever", I believe (of course, so did fork()).
POSIX says that ending the last thread is equivalent to exit(0), and this is what Python should also do when the last thread ends. This is independent of somebody calling _exit(), which should have the very effect that calling _exit has (i.e. immediate process termination). Regards, Martin
Reid Kleckner wrote:
Seems to me there's no single answer to that, because some kinds of atexit handlers may need to be called in child processes, while others may need *not* to be. Maybe we need a flag when registering an atexit handler to indicate whether it should be kept across forks? -- Greg
Standard POSIX fork semantics should be a guidance. IIUC, termination of the last thread is equivalent to calling exit(0) (although return from main() still means that exit is invoked right away, and the return value of main is the exit code - right?). Calling exit means to call all exit handlers. So to match POSIX semantics, we should also call the exit handlers in the child process (along with invoking a final garbage collection etc.) Regards, Martin
On Tue, Sep 1, 2009 at 2:58 PM, "Martin v. Löwis"<martin@v.loewis.de> wrote:
It depends, there is also _exit, which exists solely for the purpose of working around exit handlers being called from a forked child process at exit. Which semantics should Python have? In my opinion, it is more obvious that the user's handlers would be called than not, so I agree with you. Reid
I don't think so. There are other cases where you don't want to run atexit handlers, eg. in a segfault signal handler. _exit has been there "forever", I believe (of course, so did fork()).
POSIX says that ending the last thread is equivalent to exit(0), and this is what Python should also do when the last thread ends. This is independent of somebody calling _exit(), which should have the very effect that calling _exit has (i.e. immediate process termination). Regards, Martin
Reid Kleckner wrote:
Seems to me there's no single answer to that, because some kinds of atexit handlers may need to be called in child processes, while others may need *not* to be. Maybe we need a flag when registering an atexit handler to indicate whether it should be kept across forks? -- Greg
participants (3)
-
"Martin v. Löwis" -
Greg Ewing -
Reid Kleckner