Okay, I like that idea. So a unified script with timing for in each section might include: * Halo profiling * Global projection * Global profiles * Global VR (CTF and OffAxisProj) I think we probably want a large unigrid and a large AMR dataset to run these on, too. On Fri, Mar 1, 2013 at 1:50 PM, Nathan Goldbaum wrote:

Probably a good idea to try some off axis projections or simple volume renderings to test the VR code.

On Fri, Mar 1, 2013 at 10:48 AM, Matthew Turk wrote:

...

Hi all,

I got an email asking for a benchmark of yt. I think this is very, very valuable to have going forward. I was wondering if anyone had any suggestions?

I'm thinking that we probably want to test things like the halo finder, projections, and phase plots. Would a medium (1536^3) halo profiling run do that? Do we want to add on some global projections and phase plots as well?

-Matt _______________________________________________ yt-dev mailing list yt-dev@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org

_______________________________________________ yt-dev mailing list yt-dev@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org

It might be worth showing how well this works on a single proc as well as how well it works using parallel mode on a few different numbers of processors. On Fri, Mar 1, 2013 at 1:57 PM, Christopher Moody wrote:

Perhaps a benchmark for simply calculating a derived field? This would then profile a much smaller codebase.

Also benchmarking projections/fields for AMR/octree/particles of similar resolutions would be pretty cool.

chris

On Fri, Mar 1, 2013 at 10:53 AM, Matthew Turk wrote:

...

Okay, I like that idea. So a unified script with timing for in each section might include:

* Halo profiling * Global projection * Global profiles * Global VR (CTF and OffAxisProj)

I think we probably want a large unigrid and a large AMR dataset to run these on, too.

On Fri, Mar 1, 2013 at 1:50 PM, Nathan Goldbaum wrote:

...

Probably a good idea to try some off axis projections or simple volume renderings to test the VR code.

On Fri, Mar 1, 2013 at 10:48 AM, Matthew Turk wrote:

...

Hi all,

I got an email asking for a benchmark of yt. I think this is very, very valuable to have going forward. I was wondering if anyone had any suggestions?

I'm thinking that we probably want to test things like the halo finder, projections, and phase plots. Would a medium (1536^3) halo profiling run do that? Do we want to add on some global projections and phase plots as well?

-Matt _______________________________________________ yt-dev mailing list yt-dev@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org

_______________________________________________ yt-dev mailing list yt-dev@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org

_______________________________________________ yt-dev mailing list yt-dev@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org

_______________________________________________ yt-dev mailing list yt-dev@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org

-- Cameron Hummels Postdoctoral Researcher Steward Observatory University of Arizona http://chummels.org

Matt, It might be helpful if you shared some more details of this particular benchmarking exercise, if you can. It would be helpful for making sure we present the most usefil information to the people asking for it as well as info that is useful to us going forwards. j On Fri, Mar 1, 2013 at 2:05 PM, Matthew Turk wrote:

Definitely, yes. For what it's worth, the specific use case I have in mind is for performance testing a system, so I think scalability will be important.

On Fri, Mar 1, 2013 at 2:01 PM, Cameron Hummels wrote:

...

It might be worth showing how well this works on a single proc as well as how well it works using parallel mode on a few different numbers of processors.

On Fri, Mar 1, 2013 at 1:57 PM, Christopher Moody wrote:

...

Perhaps a benchmark for simply calculating a derived field? This would then profile a much smaller codebase.

Also benchmarking projections/fields for AMR/octree/particles of similar resolutions would be pretty cool.

chris

On Fri, Mar 1, 2013 at 10:53 AM, Matthew Turk wrote:

...

Okay, I like that idea. So a unified script with timing for in each section might include:

* Halo profiling * Global projection * Global profiles * Global VR (CTF and OffAxisProj)

I think we probably want a large unigrid and a large AMR dataset to run these on, too.

On Fri, Mar 1, 2013 at 1:50 PM, Nathan Goldbaum wrote:

...

Probably a good idea to try some off axis projections or simple volume renderings to test the VR code.

On Fri, Mar 1, 2013 at 10:48 AM, Matthew Turk wrote:

...

Hi all,

I got an email asking for a benchmark of yt. I think this is very, very valuable to have going forward. I was wondering if anyone had any suggestions?

I'm thinking that we probably want to test things like the halo finder, projections, and phase plots. Would a medium (1536^3) halo profiling run do that? Do we want to add on some global projections and phase plots as well?

-Matt _______________________________________________ yt-dev mailing list yt-dev@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org

_______________________________________________ yt-dev mailing list yt-dev@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org

_______________________________________________ yt-dev mailing list yt-dev@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org

_______________________________________________ yt-dev mailing list yt-dev@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org

-- Cameron Hummels Postdoctoral Researcher Steward Observatory University of Arizona http://chummels.org

_______________________________________________ yt-dev mailing list yt-dev@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org

_______________________________________________ yt-dev mailing list yt-dev@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org

I think in terms of providing a test for the cluster, the benchmark should not require the shipment of data. For the benchmarking of yt for yt's own sake, then we do need to test the IO performance. I think it'd be great to make an in-memory dataset that can then be used for testing performance. For example, set up a bunch of refined spheres and then project/slice/profile/render/halo find? Sam On Fri, Mar 1, 2013 at 12:31 PM, Matthew Turk wrote:

Hi Jeff,

Sure. I got asked for a standard yt performance benchmark to be run to figure out performance of an analysis cluster and evaluate its readiness. :) I think it's safe to sya that we should be pushing things like IO, memory capacity and communication performance.

Adding such a set of scripts (and their results!) would be very useful going forward, along with a little description of why we chose the routines we did.

-Matt

...

Matt,

It might be helpful if you shared some more details of this particular benchmarking exercise, if you can. It would be helpful for making sure we present the most usefil information to the people asking for it as well as info that is useful to us going forwards.

j

On Fri, Mar 1, 2013 at 2:05 PM, Matthew Turk wrote:

...

Definitely, yes. For what it's worth, the specific use case I have in mind is for performance testing a system, so I think scalability will be important.

On Fri, Mar 1, 2013 at 2:01 PM, Cameron Hummels wrote:

...

It might be worth showing how well this works on a single proc as well as how well it works using parallel mode on a few different numbers of processors.

On Fri, Mar 1, 2013 at 1:57 PM, Christopher Moody < chrisemoody@gmail.com> wrote:

...

Perhaps a benchmark for simply calculating a derived field? This would then profile a much smaller codebase.

Also benchmarking projections/fields for AMR/octree/particles of

similar

...

resolutions would be pretty cool.

chris

On Fri, Mar 1, 2013 at 10:53 AM, Matthew Turk wrote:

...

Okay, I like that idea. So a unified script with timing for in each section might include:

* Halo profiling * Global projection * Global profiles * Global VR (CTF and OffAxisProj)

I think we probably want a large unigrid and a large AMR dataset to run these on, too.

On Fri, Mar 1, 2013 at 1:50 PM, Nathan Goldbaum <

goldbaum@ucolick.org>

...

wrote: > Probably a good idea to try some off axis projections or simple volume > renderings to test the VR code. > > > On Fri, Mar 1, 2013 at 10:48 AM, Matthew Turk < matthewturk@gmail.com> > wrote: >> >> Hi all, >> >> I got an email asking for a benchmark of yt. I think this is very, >> very valuable to have going forward. I was wondering if anyone had >> any suggestions? >> >> I'm thinking that we probably want to test things like the halo >> finder, projections, and phase plots. Would a medium (1536^3) halo >> profiling run do that? Do we want to add on some global

On Fri, Mar 1, 2013 at 2:17 PM, j s oishi wrote: projections

...

...

...

...

...

>> and phase plots as well? >> >> -Matt >> _______________________________________________ >> yt-dev mailing list >> yt-dev@lists.spacepope.org >> http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org > > > > _______________________________________________ > yt-dev mailing list > yt-dev@lists.spacepope.org > http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org > _______________________________________________ yt-dev mailing list yt-dev@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org

_______________________________________________ yt-dev mailing list yt-dev@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org

-- Cameron Hummels Postdoctoral Researcher Steward Observatory University of Arizona http://chummels.org

_______________________________________________ yt-dev mailing list yt-dev@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org

_______________________________________________ yt-dev mailing list yt-dev@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org

---

yt-dev mailing list yt-dev@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org

---

yt-dev mailing list yt-dev@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org

If we do want to go with some big datasets, I can provide some 1024^3 and 1536^3 unigrids. I'm sure others have better AMR data than I do. On Fri, Mar 1, 2013 at 3:41 PM, Matthew Turk wrote:

Hey Sam,

On Fri, Mar 1, 2013 at 2:38 PM, Sam Skillman wrote:

...

I think in terms of providing a test for the cluster, the benchmark should not require the shipment of data. For the benchmarking of yt for yt's own sake, then we do need to test the IO performance.

I'm not really sure that we want to avoid IO...

...

I think it'd be great to make an in-memory dataset that can then be used

for

...

testing performance. For example, set up a bunch of refined spheres and then project/slice/profile/render/halo find?

Perhaps, but will our current IC generators work at really, really big scale? I didn't think they would. Plus it's hard to get them to subdivide -- you end up with ltos of nested grids.

...

Sam

On Fri, Mar 1, 2013 at 12:31 PM, Matthew Turk

wrote:

...

...

Hi Jeff,

Sure. I got asked for a standard yt performance benchmark to be run to figure out performance of an analysis cluster and evaluate its readiness. :) I think it's safe to sya that we should be pushing things like IO, memory capacity and communication performance.

Adding such a set of scripts (and their results!) would be very useful going forward, along with a little description of why we chose the routines we did.

-Matt

On Fri, Mar 1, 2013 at 2:17 PM, j s oishi wrote:

...

Matt,

It might be helpful if you shared some more details of this particular benchmarking exercise, if you can. It would be helpful for making sure we present the most usefil information to the people asking for it as well as info that is useful to us going forwards.

j

On Fri, Mar 1, 2013 at 2:05 PM, Matthew Turk wrote:

...

Definitely, yes. For what it's worth, the specific use case I have

in

...

...

...

mind is for performance testing a system, so I think scalability will be important.

On Fri, Mar 1, 2013 at 2:01 PM, Cameron Hummels wrote:

...

It might be worth showing how well this works on a single proc as well as how well it works using parallel mode on a few different numbers of processors.

On Fri, Mar 1, 2013 at 1:57 PM, Christopher Moody wrote: > > Perhaps a benchmark for simply calculating a derived field? This > would > then profile a much smaller codebase. > > Also benchmarking projections/fields for AMR/octree/particles of > similar > resolutions would be pretty cool. > > chris > > > On Fri, Mar 1, 2013 at 10:53 AM, Matthew Turk < matthewturk@gmail.com> > wrote: >> >> Okay, I like that idea. So a unified script with timing for in each >> section might include: >> >> * Halo profiling >> * Global projection >> * Global profiles >> * Global VR (CTF and OffAxisProj) >> >> I think we probably want a large unigrid and a large AMR dataset to >> run these on, too. >> >> On Fri, Mar 1, 2013 at 1:50 PM, Nathan Goldbaum >> >> wrote: >> > Probably a good idea to try some off axis projections or simple >> > volume >> > renderings to test the VR code. >> > >> > >> > On Fri, Mar 1, 2013 at 10:48 AM, Matthew Turk >> > >> > wrote: >> >> >> >> Hi all, >> >> >> >> I got an email asking for a benchmark of yt. I think this is >> >> very, >> >> very valuable to have going forward. I was wondering if anyone >> >> had >> >> any suggestions? >> >> >> >> I'm thinking that we probably want to test things like the halo >> >> finder, projections, and phase plots. Would a medium (1536^3) >> >> halo >> >> profiling run do that? Do we want to add on some global >> >> projections >> >> and phase plots as well? >> >> >> >> -Matt >> >> _______________________________________________ >> >> yt-dev mailing list >> >> yt-dev@lists.spacepope.org >> >> http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org >> > >> > >> > >> > _______________________________________________ >> > yt-dev mailing list >> > yt-dev@lists.spacepope.org >> > http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org >> > >> _______________________________________________ >> yt-dev mailing list >> yt-dev@lists.spacepope.org >> http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org > > > > _______________________________________________ > yt-dev mailing list > yt-dev@lists.spacepope.org > http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org >

-- Cameron Hummels Postdoctoral Researcher Steward Observatory University of Arizona http://chummels.org

_______________________________________________ yt-dev mailing list yt-dev@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org

_______________________________________________ yt-dev mailing list yt-dev@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org

---

yt-dev mailing list yt-dev@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org

---

yt-dev mailing list yt-dev@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org

_______________________________________________ yt-dev mailing list yt-dev@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org

_______________________________________________ yt-dev mailing list yt-dev@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org

The halo profiling script in the cookbook should work fine. The only thing I would change is the halo finding to work with the ParallelHop or something else configured for larger datasets. On Fri, Mar 1, 2013 at 4:34 PM, Matthew Turk wrote:

That sounds great. I have the Santa Fe Light Cone. I think focusing on Enzo data (since FLASH data in 2.x has poor performance) would be best for now.

Would you be able to provide a relatively simple, but compute-heavy, halo profiling script? Would the cookbook one work pretty well for this? I can handle the other aspects.

On Fri, Mar 1, 2013 at 3:48 PM, Britton Smith wrote:

...

If we do want to go with some big datasets, I can provide some 1024^3 and 1536^3 unigrids. I'm sure others have better AMR data than I do.

On Fri, Mar 1, 2013 at 3:41 PM, Matthew Turk wrote:

...

Hey Sam,

On Fri, Mar 1, 2013 at 2:38 PM, Sam Skillman wrote:

...

I think in terms of providing a test for the cluster, the benchmark should not require the shipment of data. For the benchmarking of yt for yt's own sake, then we do need to test the IO performance.

I'm not really sure that we want to avoid IO...

...

I think it'd be great to make an in-memory dataset that can then be

used

...

...

for testing performance. For example, set up a bunch of refined spheres and then project/slice/profile/render/halo find?

Perhaps, but will our current IC generators work at really, really big scale? I didn't think they would. Plus it's hard to get them to subdivide -- you end up with ltos of nested grids.

...

Sam

On Fri, Mar 1, 2013 at 12:31 PM, Matthew Turk wrote:

...

Hi Jeff,

Sure. I got asked for a standard yt performance benchmark to be run to figure out performance of an analysis cluster and evaluate its readiness. :) I think it's safe to sya that we should be pushing things like IO, memory capacity and communication performance.

Adding such a set of scripts (and their results!) would be very

useful

...

...

going forward, along with a little description of why we chose the routines we did.

-Matt

On Fri, Mar 1, 2013 at 2:17 PM, j s oishi wrote:

...

Matt,

It might be helpful if you shared some more details of this particular benchmarking exercise, if you can. It would be helpful for making sure we present the most usefil information to the people asking for it as well as info that is useful to us going forwards.

j

On Fri, Mar 1, 2013 at 2:05 PM, Matthew Turk < matthewturk@gmail.com> wrote: > Definitely, yes. For what it's worth, the specific use case I have > in > mind is for performance testing a system, so I think scalability > will > be important. > > On Fri, Mar 1, 2013 at 2:01 PM, Cameron Hummels < chummels@gmail.com> > wrote: >> It might be worth showing how well this works on a single proc as >> well >> as >> how well it works using parallel mode on a few different numbers of >> processors. >> >> >> On Fri, Mar 1, 2013 at 1:57 PM, Christopher Moody >> >> wrote: >>> >>> Perhaps a benchmark for simply calculating a derived field? This >>> would >>> then profile a much smaller codebase. >>> >>> Also benchmarking projections/fields for AMR/octree/particles of >>> similar >>> resolutions would be pretty cool. >>> >>> chris >>> >>> >>> On Fri, Mar 1, 2013 at 10:53 AM, Matthew Turk >>> >>> wrote: >>>> >>>> Okay, I like that idea. So a unified script with timing for in >>>> each >>>> section might include: >>>> >>>> * Halo profiling >>>> * Global projection >>>> * Global profiles >>>> * Global VR (CTF and OffAxisProj) >>>> >>>> I think we probably want a large unigrid and a large AMR dataset >>>> to >>>> run these on, too. >>>> >>>> On Fri, Mar 1, 2013 at 1:50 PM, Nathan Goldbaum >>>> >>>> wrote: >>>> > Probably a good idea to try some off axis projections or simple >>>> > volume >>>> > renderings to test the VR code. >>>> > >>>> > >>>> > On Fri, Mar 1, 2013 at 10:48 AM, Matthew Turk >>>> > >>>> > wrote: >>>> >> >>>> >> Hi all, >>>> >> >>>> >> I got an email asking for a benchmark of yt. I think this is >>>> >> very, >>>> >> very valuable to have going forward. I was wondering if >>>> >> anyone >>>> >> had >>>> >> any suggestions? >>>> >> >>>> >> I'm thinking that we probably want to test things like the >>>> >> halo >>>> >> finder, projections, and phase plots. Would a medium (1536^3) >>>> >> halo >>>> >> profiling run do that? Do we want to add on some global >>>> >> projections >>>> >> and phase plots as well? >>>> >> >>>> >> -Matt >>>> >> _______________________________________________ >>>> >> yt-dev mailing list >>>> >> yt-dev@lists.spacepope.org >>>> >> http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org >>>> > >>>> > >>>> > >>>> > _______________________________________________ >>>> > yt-dev mailing list >>>> > yt-dev@lists.spacepope.org >>>> > http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org >>>> > >>>> _______________________________________________ >>>> yt-dev mailing list >>>> yt-dev@lists.spacepope.org >>>> http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org >>> >>> >>> >>> _______________________________________________ >>> yt-dev mailing list >>> yt-dev@lists.spacepope.org >>> http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org >>> >> >> >> >> -- >> Cameron Hummels >> Postdoctoral Researcher >> Steward Observatory >> University of Arizona >> http://chummels.org >> >> _______________________________________________ >> yt-dev mailing list >> yt-dev@lists.spacepope.org >> http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org >> > _______________________________________________ > yt-dev mailing list > yt-dev@lists.spacepope.org > http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org _______________________________________________ yt-dev mailing list yt-dev@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org

---

yt-dev mailing list yt-dev@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org

_______________________________________________ yt-dev mailing list yt-dev@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org

_______________________________________________ yt-dev mailing list yt-dev@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org

_______________________________________________ yt-dev mailing list yt-dev@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org

_______________________________________________ yt-dev mailing list yt-dev@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org