Best way to report progress at fixed intervals
Slaunger at gmail.com
Tue Dec 9 22:27:37 CET 2008
On 9 Dec., 19:35, rdmur... at bitdance.com wrote:
> I felt like a little lunchtime challenge, so I wrote something that
> I think matches your spec, based on your sample code. This is not
> necessarily the best implementation, but I think it is simpler and
> clearer than yours. The biggest change is that the work is being
> done in the subthread, while the main thread does the monitoring.
Well, thank you for spending your lunch time break on my little
> It would be fairly simple to enhance this so that you could pass
> arbitrary arguments in to the worker function, in addition to
> or instead of the loop counter.
Yes, I agree
> Test module for testing generic ways of displaying progress
> information at regular intervals.
> import random
> import threading
> import time
> def work(i):
> Dummy process function, which takes a random time in the interval
> 0.0-0.5 secs to execute
> print "Work step %d" % i
> time.sleep(0.5 * random.random())
> class Monitor(object):
> This class creates an object that will execute a worker function
> in a loop and at regular intervals emit a progress report on
> how many times the function has been called.
> def dowork(self):
> Call the worker function in a loop, keeping track of how
> many times it was called in self.no
> for self.no in xrange(self.max_iter):
> def __call__(self, func, verbose=True, max_iter=20, progress_interval=1.0):
I had to look up the meaning of __call__, to grasp this, but I get
> Repeatedly call 'func', passing it the loop count, for max_iter
> iterations, and every progress_interval seconds report how
> many times the function has been called.
> # Not all of these need to be instance variables, but they might
> # as well be in case we want to reference them in an enhanced
> # dowork function.
> self.func = func
> self.verbose = verbose
> if self.verbose:
> print ("Work through all %d steps reporting progress every "
> "%3.1f secs...") % (self.max_iter, self.progress_interval)
> # Create a thread to run the loop, and start it going.
> worker = threading.Thread(target=self.dowork)
> # Monitoring loop.
> loops = 0
> # We're going to loop ten times per second using an integer count,
> # so multiply the seconds parameter by 10 to give it the same
> # magnitude.
> intint = int(self.progress_interval*10)
Is this not an unnecessary complication?
> # isAlive will be false after dowork returns
> while worker.isAlive():
> loops += 1
> # Wait 0.1 seconds between checks so that we aren't chewing
> # CPU in a spin loop.
Why not just call this with progress_interval directly?
> # when the modulus (second element of divmod tuple) is zero,
> # then we have hit a new progress_interval, so emit the report.
And then avoid this if expression?
> if not divmod(loops, intint):
> print "Processed %d of %d" % (self.no, self.max_iter)
> if verbose:
> print "Finished working through %d steps" % max_iter
> if __name__ == "__main__":
> #Create the monitor.
> monitor = Monitor()
> #Run the work function under monitoring.
I was unfamiliar with this notation, but now I understand it simply
invokes __call__. Thank you for showing me that!
OK. I agree this is a more elegant implementation, although I my mind,
I find it more natural if the reporting goes on in a subthread, but
that is a matter of taste, I guess. Anyway: Thank you again for
spending your lunch break on this!
More information about the Python-list