[issue11408] python locks: blocking acquire calls useless gettimeofday
Charles-Francois Natali
report at bugs.python.org
Sat Mar 5 15:09:08 CET 2011
New submission from Charles-Francois Natali <neologix at free.fr>:
While tracing a program using multiprocessing queues, I noticed that there were many calls to gettimeofday.
It turns out that acquire_timed, used by lock_PyThread_acquire_lock and rlock_acquire, always call gettimeofday, even if no timeout argument is given.
Here's an example of the performance impact (I know it's a contrived example :-):
$ cat /tmp/test_lock.py
import threading
lock = threading.Lock()
i = 0
def do_loop():
global i
for j in range(500000):
lock.acquire()
i += 1
lock.release()
t1 = threading.Thread(target=do_loop)
t2 = threading.Thread(target=do_loop)
t1.start()
t2.start()
t1.join()
t2.join()
With current code:
$ time ./python /tmp/test_lock.py
real 0m5.200s
user 0m3.288s
sys 0m1.896s
Without useless calls to gettimeofday:
$ time ./python /tmp/test_lock.py
real 0m3.091s
user 0m3.056s
sys 0m0.020s
Note that the actual gain depends on the kernel, hardware and clocksource in use (the above measurements are on a Linux 2.6.32 kernel, using acpi_pm as clocksource).
Attached is a patch removing useless calls to gettimeofday.
Note that I also removed the check for expired timeout following trylock in case of PY_LOCK_INTR, since according to http://pubs.opengroup.org/onlinepubs/009695399/functions/sem_wait.html, it seems that only sem_wait is interruptible, not sem_trywait (e.g. on Linux, sem_trywait is implemented using futex which handle non-contended case in user-space). Windows locking primitives can't return PY_LOCK_INTR. Anyway, even if it happend once in a blue moon, we would just retry a trylock, which kind of makes sense.
----------
files: lock_gettimeofday_py3k.diff
keywords: patch
messages: 130121
nosy: neologix, pitrou
priority: normal
severity: normal
status: open
title: python locks: blocking acquire calls useless gettimeofday
type: performance
Added file: http://bugs.python.org/file21007/lock_gettimeofday_py3k.diff
_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue11408>
_______________________________________
More information about the Python-bugs-list
mailing list