[Python-Dev] Python Benchmarks

[M.-A. Lemburg]

M.-A. Lemburg wrote:
>>> That's why the timers being used by pybench will become a
>>> parameter that you can then select to adapt pybench it to
>>> the OS your running pybench on.
>> Wasn't that decision a consequence of the problems found during
>> the sprint?
> 
> It's a consequence of a discussion I had with Steve Holden
> and Tim Peters:
> 
> I believe that using wall-clock timers
> for benchmarking is not a good approach due to the high
> noise level. Process time timers typically have a lower
> resolution, but give a better picture of the actual
> run-time of your code and also don't exhibit as much noise
> as the wall-clock timer approach. Of course, you have
> to run the tests somewhat longer to get reasonable
> accuracy of the timings.
> 
> Tim thinks that it's better to use short running tests and
> an accurate timer, accepting the added noise and counting
> on the user making sure that the noise level is at a
> minimum.

I just had an idea: if we could get each test to run
inside a single time slice assigned by the OS scheduler,
then we could benefit from the better resolution of the
hardware timers while still keeping the noise to a
minimum.

I suppose this could be achieved by:

* making sure that each tests needs less than 10ms to run

* calling time.sleep(0) after each test run

Here's some documentation on the Linux scheduler:

http://www.samspublishing.com/articles/article.asp?p=101760&seqNum=2&rl=1

Table 3.1 has the minimum time slice: 10ms.

What do you think ? Would this work ?

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Jun 02 2006)
>>> Python/Zope Consulting and Support ...        http://www.egenix.com/
>>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________
2006-07-03: EuroPython 2006, CERN, Switzerland              30 days left

::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! ::::