Hi, What do we think is the best way to go about adding a very basic benchmarking tool to yt? Right now my thought is to simply add to mylog 'benchmark'. For each part you wanted to benchmark you'd wrap it with two statements: mylog.benchmark('start complicated mess') ... mylog.benchmark('end complicated mess') Instead of printing this to stderr/out, it would go to a standard file, or something that could be easily analyzed by a simple function. The nice thing about using mylog is Matt has already got processor labels working, and it has timestamps. It would be therefore also trivial to have nested benchmarked sections, etc... Thanks for your comments! _______________________________________________________ sskory@physics.ucsd.edu o__ Stephen Skory http://physics.ucsd.edu/~sskory/ _.>/ _Graduate Student ________________________________(_)_\(_)_______________
Hi Stephen,
I've written up some code and stuck it in the yt branch of the hg
repo. The changeset is here:
http://hg.enzotools.org/yt/rev/87dcd3182260
So far it's kind of rudimentary, but it exposes two functions for
decorating. The slightly more sophisticated example you have proposed
would be better handled through an extension to the logging module,
which would require adding a new filter and a new loglevel to the
yt/logger.py module. You could then have it transparently call the
yt_counters object as necessary.
Here's a script to demonstrate the simple stuff I put in:
http://paste.enzotools.org/show/164/
It would also be nice to hook into atexit to have it print these on exit:
http://docs.python.org/library/atexit.html
-Matt
On Sat, Jul 25, 2009 at 1:31 PM, Stephen Skory
Hi,
What do we think is the best way to go about adding a very basic benchmarking tool to yt?
Right now my thought is to simply add to mylog 'benchmark'. For each part you wanted to benchmark you'd wrap it with two statements:
mylog.benchmark('start complicated mess') ... mylog.benchmark('end complicated mess')
Instead of printing this to stderr/out, it would go to a standard file, or something that could be easily analyzed by a simple function. The nice thing about using mylog is Matt has already got processor labels working, and it has timestamps. It would be therefore also trivial to have nested benchmarked sections, etc...
Thanks for your comments!
_______________________________________________________ sskory@physics.ucsd.edu o__ Stephen Skory http://physics.ucsd.edu/~sskory/ _.>/ _Graduate Student ________________________________(_)_\(_)_______________
_______________________________________________ Yt-dev mailing list Yt-dev@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
Thanks Matt, I'll see if I can get something working using what you've started. _______________________________________________________ sskory@physics.ucsd.edu o__ Stephen Skory http://physics.ucsd.edu/~sskory/ _.>/ _Graduate Student ________________________________(_)_\(_)_______________ ----- Original Message ----
From: Matthew Turk
To: yt-dev@lists.spacepope.org Sent: Saturday, July 25, 2009 2:38:56 PM Subject: Re: [Yt-dev] yt timing function Hi Stephen,
I've written up some code and stuck it in the yt branch of the hg repo. The changeset is here:
http://hg.enzotools.org/yt/rev/87dcd3182260
So far it's kind of rudimentary, but it exposes two functions for decorating. The slightly more sophisticated example you have proposed would be better handled through an extension to the logging module, which would require adding a new filter and a new loglevel to the yt/logger.py module. You could then have it transparently call the yt_counters object as necessary.
Here's a script to demonstrate the simple stuff I put in:
http://paste.enzotools.org/show/164/
It would also be nice to hook into atexit to have it print these on exit:
http://docs.python.org/library/atexit.html
-Matt
On Sat, Jul 25, 2009 at 1:31 PM, Stephen Skorywrote:
Hi,
What do we think is the best way to go about adding a very basic benchmarking
tool to yt?
Right now my thought is to simply add to mylog 'benchmark'. For each part you
wanted to benchmark you'd wrap it with two statements:
mylog.benchmark('start complicated mess') ... mylog.benchmark('end complicated mess')
Instead of printing this to stderr/out, it would go to a standard file, or
something that could be easily analyzed by a simple function. The nice thing about using mylog is Matt has already got processor labels working, and it has timestamps. It would be therefore also trivial to have nested benchmarked sections, etc...
Thanks for your comments!
_______________________________________________________ sskory@physics.ucsd.edu o__ Stephen Skory http://physics.ucsd.edu/~sskory/ _.>/ _Graduate Student ________________________________(_)_\(_)_______________
_______________________________________________ Yt-dev mailing list Yt-dev@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
_______________________________________________ Yt-dev mailing list Yt-dev@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
Hi all, I improved the counters to be able to handle nested counters. It isn't designed to handle bridging counters. By that I mean counters should go strictly in order in and out: OK: start1, start2, end2, end1 not OK: start1, start2, end1, end2 With a script like this: http://paste.enzotools.org/show/165/ This gets printed out automatically at exit: Current counter status: 0 : hello : 1.113e-01 1 : this and that : 9.792e-02 2 : boom : 5.903e-03 1 : Hey you : 3.597e-03 0 : blah : 6.301e-04 I haven't decided if the level number at the beginning is a good or bad thing. See changelog here: http://hg.enzotools.org/yt/rev/e5fc71364f7c Any comments? _______________________________________________________ sskory@physics.ucsd.edu o__ Stephen Skory http://physics.ucsd.edu/~sskory/ _.>/ _Graduate Student ________________________________(_)_\(_)_______________
This is great. You can export the two changesets 87dcd3182260 and
e5fc71364f7c as patches, apply them to trunk and commit.
-Matt
On Mon, Jul 27, 2009 at 12:21 PM, Stephen Skory
Hi all,
I improved the counters to be able to handle nested counters. It isn't designed to handle bridging counters. By that I mean counters should go strictly in order in and out:
OK: start1, start2, end2, end1
not OK: start1, start2, end1, end2
With a script like this:
http://paste.enzotools.org/show/165/
This gets printed out automatically at exit:
Current counter status:
0 : hello : 1.113e-01 1 : this and that : 9.792e-02 2 : boom : 5.903e-03 1 : Hey you : 3.597e-03
0 : blah : 6.301e-04
I haven't decided if the level number at the beginning is a good or bad thing.
See changelog here:
http://hg.enzotools.org/yt/rev/e5fc71364f7c
Any comments?
_______________________________________________________ sskory@physics.ucsd.edu o__ Stephen Skory http://physics.ucsd.edu/~sskory/ _.>/ _Graduate Student ________________________________(_)_\(_)_______________ _______________________________________________ Yt-dev mailing list Yt-dev@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
This is great. You can export the two changesets 87dcd3182260 and e5fc71364f7c as patches, apply them to trunk and commit.
Do you mean the trunk hg branch, or svn trunk? _______________________________________________________ sskory@physics.ucsd.edu o__ Stephen Skory http://physics.ucsd.edu/~sskory/ _.>/ _Graduate Student ________________________________(_)_\(_)_______________
I meant the svn trunk -- things will get confused if you commit to
either "yt-1.5" or "trunk" branches in hg. (It's a flaw in my
workflow.)
On Mon, Jul 27, 2009 at 12:31 PM, Stephen Skory
This is great. You can export the two changesets 87dcd3182260 and e5fc71364f7c as patches, apply them to trunk and commit.
Do you mean the trunk hg branch, or svn trunk?
_______________________________________________________ sskory@physics.ucsd.edu o__ Stephen Skory http://physics.ucsd.edu/~sskory/ _.>/ _Graduate Student ________________________________(_)_\(_)_______________ _______________________________________________ Yt-dev mailing list Yt-dev@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
participants (2)
-
Matthew Turk
-
Stephen Skory