[issue7946] Convoy effect with I/O bound threads and New GIL

[Antoine Pitrou]

Antoine Pitrou <pitrou at free.fr> added the comment:

> I use clock_gettime() to get the thread running time to calculate slice depletion.

Ouch. CLOCK_THREAD_CPUTIME_ID is not a required part of the standard. Only CLOCK_REALTIME is guaranteed to exist.

By the way, it's not obvious cpued tests anything meaningful. I understand the bias you are trying to avoid but constructing artificial test cases is not very useful, because we are playing with heuristics and it's always possible to defeat some expectations. That's why benchmarks should try to model/represent real-world situations.


I've tried ccbench with your patch and there's a clear regression in latency numbers:

Background CPU task: Pi calculation (Python)

CPU threads=0: 0 ms. (std dev: 0 ms.)
CPU threads=1: 1 ms. (std dev: 0 ms.)
CPU threads=2: 0 ms. (std dev: 2 ms.)
CPU threads=3: 0 ms. (std dev: 2 ms.)
CPU threads=4: 2 ms. (std dev: 8 ms.)

Background CPU task: regular expression (C)

CPU threads=0: 0 ms. (std dev: 0 ms.)
CPU threads=1: 969 ms. (std dev: 577 ms.)
CPU threads=2: 1050 ms. (std dev: 577 ms.)
CPU threads=3: 1217 ms. (std dev: 577 ms.)
CPU threads=4: 1416 ms. (std dev: 577 ms.)

Vanilla py3k and py3k+gilinter both don't have this problem (on the regular expression workload).

IO bandwidth numbers are similar between the BFS patch:

Background CPU task: Pi calculation (Python)

CPU threads=0: 6952.7 packets/s.
CPU threads=1: 4350.3 ( 62 %)
CPU threads=2: 3332.2 ( 47 %)
CPU threads=3: 2632.0 ( 37 %)
CPU threads=4: 2052.0 ( 29 %)

and py3k + gilinter:

Background CPU task: Pi calculation (Python)

CPU threads=0: 7023.4 packets/s.
CPU threads=1: 5019.4 ( 71 %)
CPU threads=2: 4474.6 ( 63 %)
CPU threads=3: 2728.5 ( 38 %)
CPU threads=4: 2244.9 ( 31 %)

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue7946>
_______________________________________