[Web-SIG] A 'shutdown' function in WSGI

[Graham Dumpleton]

On 21 February 2012 18:53, Tarek Ziadé <ziade.tarek at gmail.com> wrote:
>
>
> On Tue, Feb 21, 2012 at 2:39 AM, Graham Dumpleton
> <graham.dumpleton at gmail.com> wrote:
>>
>> ...
>>
>> Overall the best chance of being able to do anything is relying on atexit.
>>
>> You are though at the mercy of the WSGI hosting mechanism shutting
>> down the process and so the interpreter, in an orderly manner such
>> that atexit callbacks get called.
>>
>> In Apache/mod_wsgi you get this guarantee, even in sub interpreters
>> where atexit callbacks wouldn't normally be called when they are
>> destroyed.
>>
>> For uWSGI, atexit callbacks will not be called at the moment, by
>> Robert is making changes to it so you get a guarantee there as well.
>> It is possible he is only doing this though for case where main
>> interpreter is being used, as doing it for sub interpreters is a bit
>> fiddly.
>>
>
> But I don't think you can guarantee that everything is still up in memory by
> the time atexit gets called,
> so you can't really call cleanup code there.

The only thing which is done prior to atexit callbacks being called is
waiting on threads which weren't marked as daemonised.

void
Py_Finalize(void)
{
    PyInterpreterState *interp;
    PyThreadState *tstate;

    if (!initialized)
        return;

    wait_for_thread_shutdown();

    /* The interpreter is still entirely intact at this point, and the
     * exit funcs may be relying on that.  In particular, if some thread
     * or exit func is still waiting to do an import, the import machinery
     * expects Py_IsInitialized() to return true.  So don't say the
     * interpreter is uninitialized until after the exit funcs have run.
     * Note that Threading.py uses an exit func to do a join on all the
     * threads created thru it, so this also protects pending imports in
     * the threads created via Threading.
     */
    call_sys_exitfunc();

    ...

>> Any pure Python WSGI servers shouldn't have issues so long as they
>> aren't force exiting the whole process and bypassing normal
>> interpreter destruction.
>
>
> what do you mean by bypassing its destruction ?

Non catchable signal from within process or from a distinct monitoring process.

One of this things I pointed out is being missed.

That is, a WSGI adapter may be running on top of another layer of
abstraction, such as FASTCGI for example, where the lower layer isn't
going to have any callback mechanism of its own to even notify the
WSGI layer to trigger registered cleanup callbacks.

This is why the only mechanism one can universally rely on is the
Python interpreters own atexit mechanism.

Graham