adding a timeit.Timer context manager and decorator to time code/functions
When optimizing code, I often need to timeit a part of code or a function. I am using then the following class ``` import logging import time from functools import wraps from typing import Optional _logger = logging.getLogger("Timer") class Timer: def __init__(self, description: Optional[str] = None, logger: Optional[logging.Logger] = None): self.description = "<anonymous code block>" if description is None else description self.logger = _logger if logger is None else logger self.msg = "'%(description)s' ran in %(time).3f seconds" def __enter__(self): self.start = time.perf_counter() return self def __exit__(self, *args): self.end = time.perf_counter() self.interval = self.end - self.start self.logger.debug( self.msg, {"description": self.description, "time": self.interval} # f"Thread: '{get_ident()}' '{self.description}' ran in {round(self.interval, 3)} seconds" ) def __call__(self, f): # decorator call @wraps(f) def wrapper(*args, **kw): with Timer(description=f.__qualname__): return f(*args, **kw) return wrapper ``` that I can use either as a context manager in ``` with Timer("how long does this take?") as t: time.sleep(1) # output: DEBUG:Timer:'how long does this take?' ran in 1.000 seconds ``` or as a decorator ``` @Timer() def my_function(): """this is my function""" time.sleep(2) my_function() # output: DEBUG:Timer:'my_function' ran in 2.000 seconds ``` This class could be added to the timeit module as it fits well its functional scope.
On Fri, 27 Nov 2020 at 17:11, <sdementen@gmail.com> wrote:
When optimizing code, I often need to timeit a part of code or a function. I am using then the following class [...] that I can use either as a context manager in ``` with Timer("how long does this take?") as t: time.sleep(1) # output: DEBUG:Timer:'how long does this take?' ran in 1.000 seconds ``` or as a decorator ``` @Timer() def my_function(): """this is my function""" time.sleep(2) my_function() # output: DEBUG:Timer:'my_function' ran in 2.000 seconds ```
This class could be added to the timeit module as it fits well its functional scope.
timeit already has a Timer class that runs the code n times, and other features. I suppose a decorator for timeit should have all the features of timeit.Timer and the same overhead (not more).
The feature I am looking for is more about timing a code block or a function while running the code in dev/production. Not about timing it for benchmarking purposes. To use timeit (or the current Timer class), one has to write the stmt as a string which is not convenient (yet I understand that if you want to time a code snippet by running it more than once there may be not alternative than using stmt as strings)
29.11.20 22:44, sdementen@gmail.com пише:
The feature I am looking for is more about timing a code block or a function while running the code in dev/production. Not about timing it for benchmarking purposes. To use timeit (or the current Timer class), one has to write the stmt as a string which is not convenient (yet I understand that if you want to time a code snippet by running it more than once there may be not alternative than using stmt as strings)
You can pass a function.
I have read the documentation of the timeit module and also you can pass a function (without arguments) to it (e.g. https://stackoverflow.com/questions/5086430/how-to-pass-parameters-of-a-func...). My needs are: - simple way to instrument my code to log time for a function or a code block (I am not writing a specific script to time/benchmark a specific code) - have the timing sent to some logger - be the less intrusive possible With the Timer proposed here above (bad naming as Timer already exists in the timeit module, maybe Clocker ?), I can simply: - decorate a function with it and I am done - wrap an existing code with it as context manager I thank you for the suggestions already made but they do not fit in my need/use case in a nice/pythonic way.
On Sun, Nov 29, 2020 at 10:36 PM <sdementen@gmail.com> wrote:
My needs are: - simple way to instrument my code to log time for a function or a code block (I am not writing a specific script to time/benchmark a specific code) - have the timing sent to some logger - be the less intrusive possible
With the Timer proposed here above (bad naming as Timer already exists in the timeit module, maybe Clocker ?), I can simply: - decorate a function with it and I am done - wrap an existing code with it as context manager
timeit is indeed pretty awkward. Personally, I use iPython's @timeit for quick an dirty timing of a single function, or a few lines of code. But I'm not sure a context manager or decorator is needed, either. They are not hard to write. IN fact, I've seen both used as examples in tutorials, etc, including some of my own :-) https://uwpce-pythoncert.github.io/ProgrammingInPython/modules/Decorators.ht... and https://uwpce-pythoncert.github.io/ProgrammingInPython/exercises/context-man... (those are very simple, more robust versions would be good) I'd be very surprised if there wasn't something on PyPi as well. -- have you looked? I think one reason this hasn't been made more standard is that for inline timing, it's not clear how the results should be stored / presented -- everyone will want to do that their own way (or do a proper profile). But if you do figure out exactly what you want, and can't find it on PyPi, you can write it, and then put it on PyPi, and/or propose it be added to the std lib. -CHB -- Christopher Barker, PhD Python Language Consulting - Teaching - Scientific Software Development - Desktop GUI and Web Development - wxPython, numpy, scipy, Cython
On Sun, 29 Nov 2020 at 21:45, <sdementen@gmail.com> wrote:
To use timeit (or the current Timer class), one has to write the stmt as a string which is not convenient (yet I understand that if you want to time a code snippet by running it more than once there may be not alternative than using stmt as strings)
You can get the code of a function as a string using `inspect`. Don't know about generic code, maybe with `ast`? Or you can use the `globals` parameter of timeit and pass the function name, if I understood what Serhiy meant: def timefunc(func, *args, name="f", stmt=None, **kwargs): try: globs = kwargs.pop("globals") except KeyError: globs = {} globs[name] = func if stmt is None: stmt = f"{name}()" timeit.timeit(stmt, *args, globals=globs, **kwargs) (not tested)
On Mon, Nov 30, 2020 at 10:14 AM Marco Sulla <Marco.Sulla.Python@gmail.com> wrote:
On Sun, 29 Nov 2020 at 21:45, <sdementen@gmail.com> wrote:
To use timeit (or the current Timer class), one has to write the stmt as a string which is not convenient (yet I understand that if you want to time a code snippet by running it more than once there may be not alternative than using stmt as strings)
You can get the code of a function as a string using `inspect`. Don't know about generic code, maybe with `ast`? Or you can use the `globals` parameter of timeit and pass the function name, if I understood what Serhiy meant:
No, Serhiy meant that you can pass a function to timeit.
import hashlib def f(): ... h = hashlib.sha256(b"This is a test").hexdigest() ... import timeit timeit.timeit(f) 0.6120481300167739
ChrisA
On Mon, 30 Nov 2020 at 03:39, Chris Angelico <rosuav@gmail.com> wrote:
No, Serhiy meant that you can pass a function to timeit.
Aaah, didn't know this. On Mon, 30 Nov 2020 at 12:21, Steven D'Aprano <steve@pearwood.info> wrote:
On Mon, Nov 30, 2020 at 12:11:01AM +0100, Marco Sulla wrote:
You can get the code of a function as a string using `inspect`.
*Sometimes*.
>>> import inspect >>> def func(): ... return 1 ... >>> inspect.getsource(func) Traceback (most recent call last): [...] OSError: could not get source code
So `inspect.getsource()` does not work in the REPL. It does make sense. Anyway, usually I use cProfile, I find the "most slow" function and then I measure the more suspicious lines. So personally I'm more interested in a context manager for timeit than a decorator. I know that PyCharm can report you the speed line per line, but not the free version. On Mon, 30 Nov 2020 at 12:21, Steven D'Aprano <steve@pearwood.info> wrote:
I have been using a variant of this for years now:
https://code.activestate.com/recipes/577896-benchmark-code-with-the-with-sta...
I think it's not a bad idea to have it in timeit.
On Mon, Nov 30, 2020 at 12:11:01AM +0100, Marco Sulla wrote:
You can get the code of a function as a string using `inspect`.
*Sometimes*. >>> import inspect >>> def func(): ... return 1 ... >>> inspect.getsource(func) Traceback (most recent call last): [...] OSError: could not get source code I have been using a variant of this for years now: https://code.activestate.com/recipes/577896-benchmark-code-with-the-with-sta... It's not suitable for timing microbenchmarks, since the overhead of the with statement could be greater than the code you are timing, but sometimes you want a quick time measurement for something, and this is much better than import time t = time.time() # run code here t = time.time() - t *especially* when you're typing code in the interactive interpreter. -- Steve
participants (6)
-
Chris Angelico -
Christopher Barker -
Marco Sulla -
sdementen＠gmail.com -
Serhiy Storchaka -
Steven D'Aprano