[Web-SIG] again about logging in WSGI

[Manlio Perillo]

Graham Dumpleton ha scritto:
> On 23/11/2007, Manlio Perillo <manlio_perillo at libero.it> wrote:
>> Hi.
>>
>> As I have written in a previous thread, I would like to use nginx
>> logging system in a WSGI application (of course the same is valid for
>> Apache)
>>
>> A first problem is that the wsgi.errors stream defined in the
>> environment dictionary is valid only for the current request, but I want
>> to use a stream valid for the entire process lifetime.
>>
>> I think that there are two solutions:
>> 1) call an application supplied `init_application(environ)` callable,
>>     where the environ dictionary contains the "right" wsgi.errors stream
>>     object
>> 2) add to the environ dictionary a `wsgi.global_errors` stream object
>>
>> Any suggestions?
> 
> Getting back to what you were originally asking about, I don't really
> see why you need anything extra in the WSGI application environment.
> 
> For starters, one would remap sys.stderr to send to the web server log
> system as level ERROR. 

I have discarded any solution based on sys.stderr since the WSGI spec 
says nothing about the behaviour of stderr/stdout.

But this seems a reasonable solution, thanks.

> This should include ensuring that output is
> flushed after each newline so that it appears promptly in the error
> logs. This does mean the wrapper has to do some buffering and be a bit
> more intelligent. Doing this remapping of sys.stderr ensures though
> that any modules which use it directly to output errors will still
> have their output go somewhere sensible.
> 
> As for everything else, as suggested in a previous email, you would be
> better off basing anything else beyond sys.stderr and wsgi.errors off
> the Python 'logging' module. In doing this, nothing would be required
> in the WSGI environment. Any code would just log output using the
> 'logging' module.
> 

This is what I want to do.
But instead of setup the 'logging' module in nginx mod_wsgi, I would 
like to offer a minimal support so that this can be done by the WSGI 
application (or middleware).

In this way, the application is not forced to use the standard logging 
system.
Moreover I do not want to make nginx mod_wsgi too "complex".


> The magic missing bit would then be the WSGI adapter somehow making
> available a type of LogHandler class, eg., ApacheLogHandler, which can
> be referenced in configuration mechanism used to setup the 'logging'
> module. As necessary the LogHandler can be tied into the WSGI adapter
> dispatch mechanism so that it can know when something is logged in the
> context of a request handler, as opposed to at global scope on module
> import of WSGI application and correctly log details against the
> internal request structure, thereby enabling of client IP details
> against the request.
> 

Too complex, IMHO.

I think that it is better to have two separate logging objects.
wsgi.errors and sys.stderr, as an example.

With these two objects a middleware/application can:
- setup the global logging object using sys.stderr
- setup(?) a logging object in the wsgi dictionary using wsgi.errors


A first improvement is to add to the wsgi dictionary something like
nginx.log_level (this must be done in the server configuration file, 
using, as an example, the `wsgi_param` directive)

so that the middleware knows the log level used by the server.

In this way, as an example, if the log level is 'NGX_ERROR' and a WSGI 
application do something like:
   log.debug("a debug message")

this message does not will be written to the server log.


A second improvement is to provide support, as you suggest, to a LogHandler.

This too can be done by a middleware (IMHO), if the WSGI middleware 
expose an usable interface.


However the first improvement is all I need, for now.


> In other words, it should be possible to do it all transparently
> without requiring extensions to be added to the WSGI environment. At
> most, may need a configuration option in the WSGI adapter to setup
> configuration of 'logging' module somehow. The benefit of this
> approach is that it is more easily portable to a different WSGI
> hosting environment. The most that would be required is a change in
> the 'logging' module configuration, no user code would need to be
> changed.
> 
> Anyway, I'll have to think about it properly. I have been meaning to
> basically do just this for Apache mod_wsgi for a while, but just
> haven't got around to it.
> 
> Graham
> 

Manlio Perillo