Hi.

Disabling ASLR means you get more repeatable benchmarks, of course, but also means that on another identical machine (or a bit different circumstances), you can get different results, hence you moved statistical error to a more systematic one. I don't think that's a win

On Fri, Feb 12, 2016 at 8:42 AM, Patrascu, Alecsandru <alecsandru.patrascu@intel.com> wrote:

Hi,

Some of the things we do here at Intel, in our Languages Performance Lab [1,2], is to disable ASLR as you get more reliable results. This can be achieved on Linux by running echo 0 > /proc/sys/kernel/randomize_va_space. Also, setting the CPU frequency at a fixed frequency, disabling Turbo Boost and Hyper Threading, also helps for benchmark stability.

From my experience, the isolcpus feature is useful when you have a lot of cores on your machine because the kernel will have other cores on which it can schedule its work; furthermore, it is a best effort situation and it is not an absolute guarantee that the kernel will not use the cores specified if you have a lot of processes running (for example, if you benchmark on a machine with 2 physical cores and you isolate one of the cores, there is a big chance that the kernel will schedule processes on this core also, even it is for a small amount of time). Nevertheless, for machines with more physical cores, it can be good to have dedicated core(s) on which we do benchmarking.

[1] http://languagesperformance.intel.com/ [2] https://lists.01.org/pipermail/langperf/

Thank you, Alecsandru

...

-----Original Message----- From: Speed [mailto:speed- bounces+alecsandru.patrascu=intel.com@python.org] On Behalf Of Victor Stinner Sent: Friday, February 12, 2016 12:54 AM To: speed@python.org Subject: [Speed] Linux tip: use isolcpus to have (more) reliable benchmark

Hi,

I'm sharing with you my notes (tricks) to get more reliable benchmarks on Linux if your CPU have multiple cores:

https://haypo-notes.readthedocs.org/microbenchmark.html#reliable-micro- benchmarks

FYI perf.py recently got a new --affinity= optional parameter. I plan to send a patch to automatically use /sys/devices/system/cpu/isolated if it's not empty.

What are your "tricks" to get reliable benchmarks?

Victor

---

Speed mailing list Speed@python.org https://mail.python.org/mailman/listinfo/speed

---

Speed mailing list Speed@python.org https://mail.python.org/mailman/listinfo/speed