[Python-Dev] Using logging in the stdlib and its unit tests

[Glenn Linderman]

On 12/9/2010 12:26 AM, Vinay Sajip wrote:
> Glenn Linderman<v+python<at>  g.nevcal.com>  writes:
>> >  Or what am I missing?
> That threads are not necessarily dedicated to apps, in a real world setting.
> Depending on the server implementation, a single thread could be asked to handle
> requests for different apps over its lifetime. So the only way of knowing which
> threads are currently servicing a particular app is to maintain a set of them.

Agreed, they are not necessarily dedicated to apps.  But while they are 
running the app, they have the appname in their thread local storage, 
no?    So if a thread has the appname in its thread local storage, is it 
not servicing that app, at that moment?  And if it is, why is that 
insufficient?  That is my question, and you've sidestepped it.


> And one more thing: the filters for*both*  apps are called for a given request.
> One will return True, the other will return False.
>
> Bear in mind that the intention of the post is to be didactic, so it's possible
> there are some improvements that could be made when implementing for production.

OK, I just learned what "didactic" meant, so you've taught me something.

I understood that both filters would be called.  I understood, that in 
production, it would probably not be necessary to log messages to both 
the app log and the global log, that the global log would be there just 
for things that are not app-specific.

But I still don't see a need to maintain a set of threads running the 
app in the app object, if the app keeps track of what app it is running, 
in a spot that is accessible to the filter (which it seems to be).  I 
don't see how it is beneficial to keep track of two separate data 
structures that could serve the same purpose.

I realize you designed this in 10 minutes, and probably took twice that 
long to write it up, so didn't necessarily analyze it for efficiency.  
But I'm asking for that analysis now, if there is any real need for the 
app to track the set of threads, _for the purpose of the problem being 
solved_?  I understand there might be other reasons for which that might 
be useful, but for the logging it simply seems to be inefficient 
redundancy... and if it isn't, then I don't understand the example, yet, 
and I'm trying to.

So since you hand-waved, instead of giving a straight answer, and then 
maybe your second message was an attempt to back-pedal a bit, not sure, 
I went ahead and downloaded your code, made changes to remove the set of 
threads as outlined (see modified functions below), and it seems to run 
just as correctly. I thought it was an obvious question, while trying to 
understand the code, and maybe learn about the logger, and I guess I 
have, a little.  And maybe some other things too.  Please further 
explain if I am still missing something.  Under what conditions of the 
problem you were solving does the code below fail?

class InjectingFilter(logging.Filter):
     def __init__(self, app):
         self.app = app

     def filter(self, record):
         record.method = tlocal.request.method
         record.ip = tlocal.request.ip
         record.appName = tlocal.appName
         return record.appName == self.app.name
         # tname = threading.currentThread().getName()
         # return tname in self.app.threads

class WebApp:
     def __init__(self, name):
         self.name = name
         self.threads = set()
         handler = logging.FileHandler(name + '.log', 'w')
         f = InjectingFilter(self)
         handler.setFormatter(formatter)
         handler.addFilter(f)
         root.addHandler(handler)
         self.num_requests = 0

     def process_request(self, request):
         tlocal.request = request
         tlocal.appName = self.name
         # tname = threading.currentThread().getName()
         # self.threads.add(tname)
         self.num_requests += 1
         try:
             logger.debug('Request processing started')
             webapplib.useful()
             logger.debug('Request processing finished')
         finally:
             pass
             # self.threads.remove(tname)

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20101209/0097cd7e/attachment-0001.html>