The halo profiling script in the cookbook should work fine. The only thing
I would change is the halo finding to work with the ParallelHop or
something else configured for larger datasets.
On Fri, Mar 1, 2013 at 4:34 PM, Matthew Turk
That sounds great. I have the Santa Fe Light Cone. I think focusing on Enzo data (since FLASH data in 2.x has poor performance) would be best for now.
Would you be able to provide a relatively simple, but compute-heavy, halo profiling script? Would the cookbook one work pretty well for this? I can handle the other aspects.
On Fri, Mar 1, 2013 at 3:48 PM, Britton Smith
wrote: If we do want to go with some big datasets, I can provide some 1024^3 and 1536^3 unigrids. I'm sure others have better AMR data than I do.
On Fri, Mar 1, 2013 at 3:41 PM, Matthew Turk
wrote: Hey Sam,
On Fri, Mar 1, 2013 at 2:38 PM, Sam Skillman
wrote: I think in terms of providing a test for the cluster, the benchmark should not require the shipment of data. For the benchmarking of yt for yt's own sake, then we do need to test the IO performance.
I'm not really sure that we want to avoid IO...
I think it'd be great to make an in-memory dataset that can then be
used
for testing performance. For example, set up a bunch of refined spheres and then project/slice/profile/render/halo find?
Perhaps, but will our current IC generators work at really, really big scale? I didn't think they would. Plus it's hard to get them to subdivide -- you end up with ltos of nested grids.
Sam
On Fri, Mar 1, 2013 at 12:31 PM, Matthew Turk
wrote: Hi Jeff,
Sure. I got asked for a standard yt performance benchmark to be run to figure out performance of an analysis cluster and evaluate its readiness. :) I think it's safe to sya that we should be pushing things like IO, memory capacity and communication performance.
Adding such a set of scripts (and their results!) would be very
useful
going forward, along with a little description of why we chose the routines we did.
-Matt
On Fri, Mar 1, 2013 at 2:17 PM, j s oishi
wrote: Matt,
It might be helpful if you shared some more details of this particular benchmarking exercise, if you can. It would be helpful for making sure we present the most usefil information to the people asking for it as well as info that is useful to us going forwards.
j
On Fri, Mar 1, 2013 at 2:05 PM, Matthew Turk < matthewturk@gmail.com> wrote: > Definitely, yes. For what it's worth, the specific use case I have > in > mind is for performance testing a system, so I think scalability > will > be important. > > On Fri, Mar 1, 2013 at 2:01 PM, Cameron Hummels < chummels@gmail.com> > wrote: >> It might be worth showing how well this works on a single proc as >> well >> as >> how well it works using parallel mode on a few different numbers of >> processors. >> >> >> On Fri, Mar 1, 2013 at 1:57 PM, Christopher Moody >>
>> wrote: >>> >>> Perhaps a benchmark for simply calculating a derived field? This >>> would >>> then profile a much smaller codebase. >>> >>> Also benchmarking projections/fields for AMR/octree/particles of >>> similar >>> resolutions would be pretty cool. >>> >>> chris >>> >>> >>> On Fri, Mar 1, 2013 at 10:53 AM, Matthew Turk >>> >>> wrote: >>>> >>>> Okay, I like that idea. So a unified script with timing for in >>>> each >>>> section might include: >>>> >>>> * Halo profiling >>>> * Global projection >>>> * Global profiles >>>> * Global VR (CTF and OffAxisProj) >>>> >>>> I think we probably want a large unigrid and a large AMR dataset >>>> to >>>> run these on, too. >>>> >>>> On Fri, Mar 1, 2013 at 1:50 PM, Nathan Goldbaum >>>> >>>> wrote: >>>> > Probably a good idea to try some off axis projections or simple >>>> > volume >>>> > renderings to test the VR code. >>>> > >>>> > >>>> > On Fri, Mar 1, 2013 at 10:48 AM, Matthew Turk >>>> > >>>> > wrote: >>>> >> >>>> >> Hi all, >>>> >> >>>> >> I got an email asking for a benchmark of yt. I think this is >>>> >> very, >>>> >> very valuable to have going forward. I was wondering if >>>> >> anyone >>>> >> had >>>> >> any suggestions? >>>> >> >>>> >> I'm thinking that we probably want to test things like the >>>> >> halo >>>> >> finder, projections, and phase plots. Would a medium (1536^3) >>>> >> halo >>>> >> profiling run do that? Do we want to add on some global >>>> >> projections >>>> >> and phase plots as well? >>>> >> >>>> >> -Matt >>>> >> _______________________________________________ >>>> >> yt-dev mailing list >>>> >> yt-dev@lists.spacepope.org >>>> >> http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org >>>> > >>>> > >>>> > >>>> > _______________________________________________ >>>> > yt-dev mailing list >>>> > yt-dev@lists.spacepope.org >>>> > http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org >>>> > >>>> _______________________________________________ >>>> yt-dev mailing list >>>> yt-dev@lists.spacepope.org >>>> http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org >>> >>> >>> >>> _______________________________________________ >>> yt-dev mailing list >>> yt-dev@lists.spacepope.org >>> http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org >>> >> >> >> >> -- >> Cameron Hummels >> Postdoctoral Researcher >> Steward Observatory >> University of Arizona >> http://chummels.org >> >> _______________________________________________ >> yt-dev mailing list >> yt-dev@lists.spacepope.org >> http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org >> > _______________________________________________ > yt-dev mailing list > yt-dev@lists.spacepope.org > http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org _______________________________________________ yt-dev mailing list yt-dev@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
yt-dev mailing list yt-dev@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
_______________________________________________ yt-dev mailing list yt-dev@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
_______________________________________________ yt-dev mailing list yt-dev@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
_______________________________________________ yt-dev mailing list yt-dev@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
_______________________________________________ yt-dev mailing list yt-dev@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org