[Python-Dev] Version 3 Proposal: thread-local data

[Jim Fulton]

(Changes from previous version:

   - Changed subject line

   - Added subclassability

   - Added cleanup of thread data on destruction

)

We often want to associate data with threads.  Python
manages a dictionary per-tread for this purpose, but there's
currently no way to get to this dictionary from Python.

I propose, for 2.4, to add a "local" class to the threading
module (and dummy_threading module) for creating
thread-local objects.  I'll provide both Python and
C implementations.

Here is proposed documentation (and test:) for the class:

"""Thread-local objects

Thread-local objects support the management of thread-local data.
If you have data that you want to be local to a thread, simply create
a thread-local object and use it's attributes:

   >>> import threading
   >>> mydata = threading.local()
   >>> mydata.__class__.__name__
   'local'
   >>> mydata.number = 42
   >>> mydata.number
   42

You can also access the local-object's dictionary:

   >>> mydata.__dict__
   {'number': 42}
   >>> mydata.__dict__.setdefault('widgets', [])
   []
   >>> mydata.widgets
   []

What's important about thread-local objects is that their data are
local to a thread. If we access the data in a different thread:

   >>> log = []
   >>> def f():
   ...     log.append(mydata.__dict__.items())
   ...     mydata.number = 11
   ...     log.append(mydata.number)

   >>> import threading
   >>> thread = threading.Thread(target=f)
   >>> thread.start()
   >>> thread.join()
   >>> log
   [[], 11]

we get different data.  Furthermore, changes made in the other thread
don't affect data seen in this thread:

   >>> mydata.number
   42

Of course, values you get from a local object, including a __dict__
attribute, are for whatever thread was current at the time the
attribute was read.  For that reason, you generally don't want to save
these values across threads, as they apply only to the thread they
came from.

You can create custom local objects by subclassing the local class:

   >>> class MyLocal(threading.local):
   ...     number = 2
   ...     def __init__(self, **kw):
   ...         self.__dict__.update(kw)
   ...     def squared(self):
   ...         return self.number ** 2

This can be useful to support default values, methods and
initialization.  Note that if you define an __init__ method, it will be
called each time the local object is used in a separate thread.  This
is necessary to initialize each thread's dictionary.

Now if we create a local object:

   >>> mydata = MyLocal(color='red')

Now we have a default number:

   >>> mydata.number
   2

an initial color:

   >>> mydata.color
   'red'
   >>> del mydata.color

And a method that operates on the data:

   >>> mydata.squared()
   4

As before, we can access the data in a separate thread:

   >>> log = []
   >>> thread = threading.Thread(target=f)
   >>> thread.start()
   >>> thread.join()
   >>> log
   [[('color', 'red')], 11]

without effecting this threads data:

   >>> mydata.number
   2
   >>> mydata.color
   Traceback (most recent call last):
   ...
   AttributeError: 'MyLocal' object has no attribute 'color'

Note that subclasses can define slots, but slots are shared across
threads. Only the instance dictionary is thread local.

"""


Here is a demonstration Python implementation:

     from threading import currentThread, enumerate

     class _localbase(object):
         __slots__ = '_local__key', '_local__args'

         def __new__(cls, *args, **kw):
             self = object.__new__(cls)
             key = '_local__key', 'thread.local.' + str(id(self))
             object.__setattr__(self, '_local__key', key)
             object.__setattr__(self, '_local__args', (args, kw))

             if args or kw and (cls.__init__ is object.__init__):
                 raise TypeError("Initialization arguments are not supported")

             return self

     def _patch(self):
         key = object.__getattribute__(self, '_local__key')
         d = currentThread().__dict__.get(key)
         if d is None:
             d = {}
             currentThread().__dict__[key] = d
             object.__setattr__(self, '__dict__', d)

             # we have a new instance dict, so call out __init__ if we have
             # one
             cls = type(self)
             if cls.__init__ is not object.__init__:
                 args, kw = object.__getattribute__(self, '_local__args')
                 cls.__init__(self, *args, **kw)
         else:
             object.__setattr__(self, '__dict__', d)

     class local(_localbase):

         def __getattribute__(self, name):
             _patch(self)
             return object.__getattribute__(self, name)

         def __setattr__(self, name, value):
             _patch(self)
             return object.__setattr__(self, name, value)

         def __delattr__(self, name):
             _patch(self)
             return object.__delattr__(self, name)

         def __del__(self):
             key = object.__getattribute__(self, '_local__key')
             for thread in enumerate():
                 if key in thread.__dict__:
                     del thread.__dict__[key]

Any objections? Any more tricks up your sleeve Armin? :)

Jim

-- 
Jim Fulton           mailto:jim at zope.com       Python Powered!
CTO                  (540) 361-1714            http://www.python.org
Zope Corporation     http://www.zope.com       http://www.zope.org