Trying to use threading.local()
Cameron Simpson
cs at cskk.id.au
Thu Sep 13 06:42:38 EDT 2018
On 13Sep2018 12:21, Antoon Pardon <antoon.pardon at vub.be> wrote:
>On 12-09-18 22:14, Peter Otten wrote:
>> As I understand it you need one local() instance that is shared by all
>> workers. Every thead will then see thread-specific values.
>
>It has always puzzled me how this is useful. The times I work with threads,
>I just put thread specific values in the local variables of the thread function.
That works if your thread function is self contained (only that function needs
access to the thread specific values) or you can easily pass the thread
specific values on to subsidiary functions if they're needed.
>What is gained by using threading.local instances?
Supposing you've got some longer lived objects which existed before your thread
started and survive after it completes, which want to maintain some state for
client threads, by which I mean your subsequent threads which may make use of
these objects.
For this purpose the long lived object wants to keep some distinct state for
each client thread, because they may use it concurrently. Consider an object
looking after a database, which automatically opens a database connection to
serve each client thread when it is accessed. You'd make a threading.local
instance to manage this connection, and each thread would see its own distinct
connection by accessing that threading.local instance.
There's any number of other examples, but they probably all exist around
objects essentially external to your thread function which need to provide
thread specific state which isn't constructed by the thread function itself but
instead as a consequence of the function accessing the long lived object.
I've got an example here where I maintain a stack of data source contexts for
some data access. Calling code looks like this:
with S:
... call functions which make use of "S" implicitly ...
with S2:
... here we use "S2" implicitly ...
... back to using "S" ...
... back to whatever was in use previously ...
The "S" context manager pushes itself onto a stack, and the access code makes
use of whatever is on the top of that stack. This supports generic code which
doesn't have to pass "S" to every single function call, including _through_
functions which never use "S" but themselve call other functions which do.
So the current "S" is effectively a piece of global context, like the UNIX
working directory (it is implicitly used when you open a file with a relative
pathname).
Now imagine that I have multiple threads working this way. I now need thread
specific context stacks. I do this with a threading.local instance, with the
stack an attribute of the threading.local. Multiple threads can now freely push
and pop from the context stack without interfering with each other.
Hoping this clarifies the use case.
Cheers,
Cameron Simpson <cs at cskk.id.au>
More information about the Python-list
mailing list