Hi, Perhaps I am the only one bothered by the timeit module, but it seems poorly designed to me. First of all, it should use a geometric series with a timeout value to detect how many iterations it should perform. Currently, the user is required to manually specify the number of iterations (the default is 1 million). If the user optimizes his or her code, then the number of iterations must be changed. If the user moves to a slower or faster computer, then the number of iterations must be changed again. This is annoying. Secondly, there should be a way to time a callable directly. That is, without finding the string name of the callable and using a string "import X" statement. These contortions violate rules #1 and #3 of the Zen of Python. Yes, there is function call overhead in timing a callable as opposed to a fragment of code. However, when I'm benchmarking code I am often timing functions, so it seems logical to make this easy to do. I have three (mutually exclusive) ideas for improving the current situation: * Add a time_callable() or time_func() function to the timeit module. These would take the callable, args, kwargs, and maximum time* to spend timing the function as arguments. * Add a FuncTimer class (function/callable timer) to the existing timeit module. The constructor takes a callable, args, kwargs as arguments. The class has instance methods timeit(max_time=4.0) and repeat(repeat=3, max_time=4.0). * Create a new timeit module (e.g. timeit2) which uses geometric series to implement both code fragment timing (which timeit does) and callable timing. I have a simple implementation of geometric series timing routines for callables and code fragments. This is recipe 440657. http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/440657 This recipe leaves the GC enabled. Clearly, this is simple to implement. If python-dev likes the idea of improving Python's timeit functionality, I volunteer to implement whichever interface is decided upon. Opinions? Thanks, Connelly Barnes * The maximum time will be exceeded only if a single call to the callable takes more than max_time seconds. __________________________________________________ Do You Yahoo!? Tired of spam? Yahoo! Mail has the best spam protection around http://mail.yahoo.com
Connelly Barnes wrote:
Hi,
Perhaps I am the only one bothered by the timeit module, but it seems poorly designed to me.
First of all, it should use a geometric series with a timeout value to detect how many iterations it should perform. Currently, the user is required to manually specify the number of iterations (the default is 1
The provision of a default value generally seems to chop the legs from your argument that the number of iterations is required.
million). If the user optimizes his or her code, then the number of iterations must be changed. If the user moves to a slower or faster computer, then the number of iterations must be changed again. This is annoying.
What? The purpose of timeit is to give an approximation to the length of time to run a specific piece ofcode. Why must the number of iterations be changed when moving to a slower or faster computer?
Secondly, there should be a way to time a callable directly. That is, without finding the string name of the callable and using a string "import X" statement. These contortions violate rules #1 and #3 of the Zen of Python.
Presumably for benchmarking purposes the function call overhead would be present for all compaered techniques. Do you mean rules #0 and #2?
[...] regards Steve -- Steve Holden +44 150 684 7255 +1 800 494 3119 Holden Web LLC www.holdenweb.com PyCon TX 2006 www.python.org/pycon/
Steve Holden wrote:
Connelly Barnes wrote:
Hi,
Perhaps I am the only one bothered by the timeit module, but it seems poorly designed to me.
First of all, it should use a geometric series with a timeout value to detect how many iterations it should perform. Currently, the user is required to manually specify the number of iterations (the default is 1
The provision of a default value generally seems to chop the legs from your argument that the number of iterations is required.
From the command line, timeit autocalibrates to use the smallest power of ten that results in taking at least 0.2 seconds to execute all iterations. IMO, making this the default behaviour of the timer objects themselves would be *much* more useful behaviour than the current default to 10e6 iterations. I've been caught a few times hammering on Ctrl-C when I realised that I'd just told the interpreter to go do a million iterations of something that took a few seconds for each attempt.
million). If the user optimizes his or her code, then the number of iterations must be changed. If the user moves to a slower or faster computer, then the number of iterations must be changed again. This is annoying.
What? The purpose of timeit is to give an approximation to the length of time to run a specific piece ofcode.
Why must the number of iterations be changed when moving to a slower or faster computer?
Because the default of 10e6 may be entirely inappropriate depending on how long an individual iteration is. If a single iteration takes a microsecond, great, but if it takes a second, better send out for pizza (or maybe go for a trip around the world in the 4 months that sucker is going to take to execute!) If Timer was changed as follows, it would "just work", even when each iteration took a few seconds (6 seconds per iteration would give 1 minute for each execution of the timeit method): class Timer: def __init__(self, stmt="pass", setup="pass", timer=default_timer): """Constructor. See class doc string.""" # ... existing constructor self.default_number = None def timeit(self, number=None): # Note changed default if number is None: if self.default_number is None: self.default_number = self.calibrate() number = self.default_number # ... existing timeit method def calibrate(self): # New method, taken from script code at end of module # find smallest power of 10 >= 10 so that 0.2 seconds <= total time # capped at 10 billion (10e10) iterations, as reaching that limit # implies fewer than 20 picoseconds (2e-11) per iteration (i.e. Fast!) for i in range(1, 10): number = 10**i x = self.timeit(number) if x >= 0.2: break self.default_number = number
Secondly, there should be a way to time a callable directly. That is, without finding the string name of the callable and using a string "import X" statement. These contortions violate rules #1 and #3 of the Zen of Python.
Presumably for benchmarking purposes the function call overhead would be present for all compaered techniques. Do you mean rules #0 and #2?
Timing an existing function really is a pain - timeit expects source code it can plug into a code template, so timing an existing function from an interactive session isn't as easy as it could be (you have to figure out a way to give strings to Timer that it can plug into its string template, rather than just giving it the callable directly). A subclass would deal with that quite handily: class CallTimer(Timer): # Use lambda or functional.partial to pass arguments to the callable def __init__(self, callable, timer=default_timer): def inner(_it, _timer): _t0 = _timer() for _i in _it: callable() _t1 = _timer() return _t1 - _t0 self.timer = timer self.inner = inner self.src = None # Using a real function, so leave linecache alone when printing exceptions def print_exc(self, file=None): import traceback traceback.print_exc(file=file) Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia --------------------------------------------------------------- http://www.boredomandlaziness.org
participants (3)
-
Connelly Barnes -
Nick Coghlan -
Steve Holden