[issue7946] Convoy effect with I/O bound threads and New GIL

[David Beazley]

David Beazley <dave at dabeaz.com> added the comment:

Here's a short benchmark for everyone who thinks that my original benchmark was somehow related to TCP behavior.   This one doesn't even involve sockets:

from threading import Thread
import time

def writenums(f,n):
    start = time.time()
    for x in range(n):
        f.write("%d\n" % x)
    end = time.time()
    print(end-start)

def spin():
    while True:
        pass

# Uncomment to add a thread
#t1 = Thread(target=spin)
#t1.daemon=True
#t1.start()

writenums(open("/tmp/nums","w"),1000000)


If I run this on my Macbook with no threads, it takes about 1.05 seconds.  If the one spinning thread is turned on, the time jumps to about 4.5 seconds.  What you're seeing is that the spinning thread unfairly hogs the CPU.

If I use my own patched version (new GIL with priorities), the threaded version drops back down to about 1.10 seconds.   I have not tried it with Antoine's latest patch, but would expect similar results as he is also using priorities.

Just to be clear, this issue is not specific to sockets or TCP.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue7946>
_______________________________________