[Web-SIG] again about logging in WSGI

[Graham Dumpleton]

On 23/11/2007, Manlio Perillo <manlio_perillo at libero.it> wrote:
> Hi.
>
> As I have written in a previous thread, I would like to use nginx
> logging system in a WSGI application (of course the same is valid for
> Apache)
>
> A first problem is that the wsgi.errors stream defined in the
> environment dictionary is valid only for the current request, but I want
> to use a stream valid for the entire process lifetime.
>
> I think that there are two solutions:
> 1) call an application supplied `init_application(environ)` callable,
>     where the environ dictionary contains the "right" wsgi.errors stream
>     object
> 2) add to the environ dictionary a `wsgi.global_errors` stream object
>
> Any suggestions?

Getting back to what you were originally asking about, I don't really
see why you need anything extra in the WSGI application environment.

For starters, one would remap sys.stderr to send to the web server log
system as level ERROR. This should include ensuring that output is
flushed after each newline so that it appears promptly in the error
logs. This does mean the wrapper has to do some buffering and be a bit
more intelligent. Doing this remapping of sys.stderr ensures though
that any modules which use it directly to output errors will still
have their output go somewhere sensible.

As for everything else, as suggested in a previous email, you would be
better off basing anything else beyond sys.stderr and wsgi.errors off
the Python 'logging' module. In doing this, nothing would be required
in the WSGI environment. Any code would just log output using the
'logging' module.

The magic missing bit would then be the WSGI adapter somehow making
available a type of LogHandler class, eg., ApacheLogHandler, which can
be referenced in configuration mechanism used to setup the 'logging'
module. As necessary the LogHandler can be tied into the WSGI adapter
dispatch mechanism so that it can know when something is logged in the
context of a request handler, as opposed to at global scope on module
import of WSGI application and correctly log details against the
internal request structure, thereby enabling of client IP details
against the request.

In other words, it should be possible to do it all transparently
without requiring extensions to be added to the WSGI environment. At
most, may need a configuration option in the WSGI adapter to setup
configuration of 'logging' module somehow. The benefit of this
approach is that it is more easily portable to a different WSGI
hosting environment. The most that would be required is a change in
the 'logging' module configuration, no user code would need to be
changed.

Anyway, I'll have to think about it properly. I have been meaning to
basically do just this for Apache mod_wsgi for a while, but just
haven't got around to it.

Graham