[Python-Dev] New PEP: 319

[Skip Montanaro]

    >> I have a multi-threaded XML-RPC server which, among lots of other
    >> bits of data maintains some "top 50" data (top 50 cities searched
    >> for, top 50 performers searched for, etc).  Update time for that data
    >> is very fast relative to much of the other data maintained by the
    >> server.

    Michel>     synchronize item:

    Michel> would create a (hidden) lock for each item for you.  Wouldn't
    Michel> this solve your problem of no two threads changing one item?  or
    Michel> do changes to any one top 50 item *require* locking all 50?  If
    Michel> they are independent that this is exactly the purpose PEP 319
    Michel> serves.

I can see I wasn't clear in my original post.  Let me be more concrete.  I
have a class with several objects with store information about the top 50
searches for musicians and cities on the Mojam and Musi-Cal websites:

    class DBServer(genericserver.GenericServer):
        doratings = 1
        poolsize = 5

        def __init__(self, address, handlerclass):
            genericserver.GenericServer.__init__(self, address, handlerclass)

            self.conn_pool = Queue.Queue(self.poolsize)
            self.init_locks()
            ...
            self.top_50_perfs = {}
            self.top_50_cities = {}
            self.top_50_mojam = {}
            self.top_50_musi_cal = {}
            ...

Instead of creating a lock to protect each of those four objects I create a
single lock for that purpose:

        def init_locks(self):
            self.sql_lock = threading.RLock()
            self.query_lock = threading.RLock()
            self.top_50_lock = threading.RLock()
            self.namemap_lock = threading.RLock()
            self.dump_lock = threading.RLock()

Your proposal would suggest I (implicitly) create a lock for each of those
top_50_* dictionaries.  I think my code would be more complex.  I think this
is precisely the sort of case Jack was alluding to with his "one lock,
multiple objects" case.  In this case it's overkill for me to create
separate locks for each object because access times for those data are
fast.  There isn't likely to be any contention for that lock.

For my applications I would be more than happy with a more succinct (and
safe) way to write:

    lock.acquire()
    try:
        block
    finally:
        lock.release()

I don't really care what the syntax is, but I think implicit per-object
locks are unnecessary.

    >> Still, I agree with Jack that there are plenty of situations where
    >> you use one lock to lock multiple objects.  (Consider the Python GIL
    >> as another example. ;-)

    Michel> Isn't the interpreter one object in this case?  Does the GIL
    Michel> lock anything else other than the interpreter?

Depends on what level you look at it.  Sure, the interpreter is a single
object, but it's a very complex object which contains lots of other
subobjects.  There are lots of places in the Python code where assumptions
are made that because the GIL is being held, the (in)consistency of a
particular object at that point in time isn't crucial.  You know no other
thread can access that object right then because the GIL is held by the
currently executing thread.  As long as the object's state is made
consistent by the time you release the GIL you're golden.  In essence, the
GIL is Jack's "one lock, multiple objects" case taken to the extreme.

Skip