[Web-SIG] Alternative to threading.local, based on the stack

[Donovan Preston]

On Jul 4, 2008, at 12:10 PM, Ian Bicking wrote:

> Iwan Vosloo wrote:
>> Many web frameworks and ORM tools have the need to propagate data
>> depending on some or other context within which a request is dealt  
>> with.
>> Passing it all via parameters to every nook of your code is  
>> cumbersome.
>> A lot of the frameworks use a thread local context to solve this
>> problem. I'm assuming these are based on threading.local.  (See,  
>> for example:
>> http://www.sqlalchemy.org/docs/05/ 
>> session.html#unitofwork_contextual )
>> Such usage assumes that one request is served per thread.
>> This is not necessarily the case.  (Twisted would perhaps be an  
>> example,
>> but I have not checked how the twisted people deal with the issue.)
>
> The Spawning server (http://ulaluma.com/pyx/archives/2008/06/spawning_01_rel.html 
> ) would indeed get things mixed up this way, as uses greenlets to  
> make (at least some) blocking calls async.  So it would encounter  
> this problem full-force.

With the latest version of Spawning (http://pypi.python.org/pypi/Spawning/0.6 
) this is only true if specifically configured to do so (by passing -- 
threads=0 or including num_threads = 0 in the ini file). In this case  
Spawning monkey-patches the threadlocal module with a version that  
stores things in greenlet-local storage. This makes Pylons  
applications and other applications that use thread-local storage work  
as long as the application does not do any blocking database operations.

However, by default Spawning now uses a threadpool to execute wsgi  
applications, since the vast majority of wsgi applications probably  
block. This makes it functionally identical to the Twisted server  
which executes the actual wsgi application in a threadpool.

> To throw another wrench in things, with the Paste/WebError  
> evalexception interactive exception handler, it restores this thread- 
> local context so you can later execute expressions in the same  
> context.

It seems to me that what is really needed here is an extension of wsgi  
that specifies how to get, set, and list request local storage, and  
for people to use that instead of the threadlocal module. Of course,  
for threaded servers, they will just use the threadlocal module, but  
for Spawning running in single-threaded cooperative mode it would use  
a greenlet-local implementation, and for a hypothetical Twisted server  
running a hypothetical asynchronous wsgi application it would just use  
a random request id.

Donovan