Read-Write Lock vs primitive Lock()

[Gabriel Genellina]

En Tue, 30 Dec 2008 06:16:23 -0200, k3xji <sumerc at gmail.com> escribió:

> As GLOBAL_LOOP_COUNT is 10000000, now this is making a bottleneck on
> the readers. I had assumed that as everythread is given only 100
> bytecodes to execute, that it will be enough to have a 10000 value for
> this number to let other rthread start() and try to acquire the lock.
> But, Python undocumentedly, does not grab the GIL easily if a Lock is
> held. This is strange for me.

This has nothing to do with the GIL, nor undocumented behavior. If all  
threads are blocked waiting for the same lock, yielding the CPU has no  
effect, because nobody can progress until the lock is released.

I'll ask you again: what is your use case? what do you want to do? what is  
a "reader" in your problem? what is a "writer"? what is the resource you  
want to protect using a lock?
Different problems have different behaviors; for example, for  
reading/writing files, your model is basically wrong.

> That may be the point. That is why I am trying to test this. When will
> ReadWrite() lock permforms better over the primitive lock. By the way,
> for the reference, the code in the recipe is used in CherryPy and
> based on the Tanenbaum's book Operating Systems Design.

Ah, good to know at least it has some background. Anyway, isn't a  
guarantee of correctness. By exampe, Tanenbaum's algorithm for the dining  
philosophers problem were (is?) flawed (according to W. Stallings).

> I understand your point completely, but let's change anything inside
> the loop. Just assume that we a thread that is supposed to read
> something, and a thread that is supposed to write. If I create
> thousands of readers, and if the operation is enormously huge
> calculation(as Python does not grab GIL easily inside a Lock), then
> this will create a bottlencek on readers. But, with ReadWrite Lock
> this problem *SHOULD* be fixed and in my tests I see it is fixed, when
> I increase the time spent in the loop in huge amounts.

But why would you protect the whole long calculation with a lock? And why  
do you say "Python does not grab GIL easily inside a Lock", whatever that  
means?
If you hold a scarce resource for a long time, this is likely to cause a  
bottleneck, and the language (be it Python or other) is mostly irrelevant  
here.

> I had
> assumed that Python will release GIL for other threads after 100
> bytecodes are executed by default.

Yes. You can alter how often this occurs, using sys.setcheckinterval (the  
code I posted does that, just to see how results vary)

> However, as I stated, when a Lock()
> is held this is changing. I think this is because to avoid simple MT
> problems for new programmers. Not sure. I will read threads.c for
> information.

No, it's always the same thing, locks don't affect this behavior. But as I  
said above, if no other thread can progress because all of them are  
waiting to acquire the same lock, you gain nothing by releasing the GIL.

-- 
Gabriel Genellina