[Python-ideas] Timing hefty or long-running blocks of code

[Masklinn]

On 2013-06-01, at 11:24 , Steven D'Aprano wrote:

> The timeit module is great for timing small code snippets, but it is rather inconvenient for timing larger blocks of code. It's also over-kill: under normal circumstances, there is little need for the heroic measures timeit goes through to accurately time long-running code.

Does timeit really go through much heroics when using it in code? The
CLI version does a bunch of testing and adaptation to try to get
accurate measures but as far as I know the Python API has no such
facilities, it runs what you gave it the number of time you tell it.

The only "heroics" it goes through is using timeit.default_timer(),
which is supposed to provide the most accurate clock available.

> When people want to time something that doesn't require timeit, they have to write their own timer boilerplate:
> 
> from time import time
> t = time()
> do_this()
> do_that()
> print(time() - t)
> 
> 
> Although only a few lines of boilerplate, this has a few problems, the biggest two being:
> 
> * the user has to decide which of the many clocks provided by Python
>  is suitable (hint: time.time is not necessarily the best);

But that's what timeit.default_timer() is for right? This seems to be a
problem of awareness more than availability.

> * this is inconvenient at the interactive interpreter, since the timer
>  is started before the code being typed has been entered.
> 
> I propose a context manager in the timeit module:
> 
> with Stopwatch():
>    do_this()
>    do_that()
> 
> => results are automatically printed, or available for programmatic access
> 
> I have had a recipe for this on ActiveState for about 18 months, where it is relatively popular:
> 
> http://code.activestate.com/recipes/577896-benchmark-code-with-the-with-statement
> 
> and I have extensively used it interactively, in Python 2.6 through 3.3 inclusive. The latest version can be found here:
> 
> https://code.google.com/p/my-startup-file/source/browse/timer.py
> 
> This solves the two problems listed above:
> 
> * the Stopwatch can use the most appropriate timer by default, and
>  allow the user to override it if they so choose;
> 
> * the Stopwatch will not start until the with-block is actually
>  entered.
> 
> 
> Is there interest in seeing this in the standard library?

A more general version can be achieved through timeit.Timer, though it's
got more boilerplate and as far as I know there's no example of it in
the documentation:

@timeit.Timer
def foo():
    do_this()
    do_that()
print foo.timeit(1)

On the other hand it allows repeated benching to try and refine the
iterations count.

Maybe context-management could simply be added to timeit.Timer,
rather than be a separate object?