Covering Grid Dimensions: multiple refinement levels
Dear YT Users, To better understand covering_grid (or smoothed_covering_grid), can someone explain what dimensions I should use when I have multiple levels of refinement? If I have 1 level of refinement from AMR data, the dimensions should be the same, I believe, but what if I have 2 or 3 levels of refinement? Should the fixed resolution region then have dimensions of dims*2^2 and dims*3^2 respectively? Here is one example from a tutorial <http://yt-project.org/doc/examining/low_level_inspection.html#examining-grid-data-in-a-fixed-resolution-array> if that helps isolate my question: all_data_level_2 = ds.covering_grid(level=2, left_edge=[0,0.0,0.0], dims=ds.domain_dimensions * 2**2) Thanks in advance, Carla
I've done this in the past: ref = int(np.product(ds.ref_factors[0:max_level])) # allocate for our uniformly-gridded result dims = ds.domain_dimensions*ref this will work for a more general case when the jump between levels can change as a function of level. On Thu, Nov 19, 2015 at 9:10 AM, Carla Bernhardt < carla.j.bernhardt@gmail.com> wrote:
Dear YT Users,
To better understand covering_grid (or smoothed_covering_grid), can someone explain what dimensions I should use when I have multiple levels of refinement? If I have 1 level of refinement from AMR data, the dimensions should be the same, I believe, but what if I have 2 or 3 levels of refinement? Should the fixed resolution region then have dimensions of dims*2^2 and dims*3^2 respectively?
Here is one example from a tutorial <http://yt-project.org/doc/examining/low_level_inspection.html#examining-grid-data-in-a-fixed-resolution-array> if that helps isolate my question: all_data_level_2 = ds.covering_grid(level=2, left_edge=[0,0.0,0.0], dims=ds.domain_dimensions * 2**2)
Thanks in advance, Carla
_______________________________________________ yt-users mailing list yt-users@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
-- Michael Zingale Associate Professor Dept. of Physics & Astronomy • Stony Brook University • Stony Brook, NY 11794-3800 *phone*: 631-632-8225 *e-mail*: Michael.Zingale@stonybrook.edu *web*: http://www.astro.sunysb.edu/mzingale github: http://github.com/zingale
Thank you for your quick response. However, when I used: ref = int(np.product(ds.ref_factors[0:3])), I got this error: AttributeError: 'EnzoDataset' object has no attribute 'ref_factors' Did I misunderstand your suggestion? Or do I need to import something? Thanks, Carla 2015-11-19 15:39 GMT+01:00 Michael Zingale <michael.zingale@stonybrook.edu>:
I've done this in the past:
ref = int(np.product(ds.ref_factors[0:max_level]))
# allocate for our uniformly-gridded result
dims = ds.domain_dimensions*ref
this will work for a more general case when the jump between levels can change as a function of level.
On Thu, Nov 19, 2015 at 9:10 AM, Carla Bernhardt < carla.j.bernhardt@gmail.com> wrote:
Dear YT Users,
To better understand covering_grid (or smoothed_covering_grid), can someone explain what dimensions I should use when I have multiple levels of refinement? If I have 1 level of refinement from AMR data, the dimensions should be the same, I believe, but what if I have 2 or 3 levels of refinement? Should the fixed resolution region then have dimensions of dims*2^2 and dims*3^2 respectively?
Here is one example from a tutorial <http://yt-project.org/doc/examining/low_level_inspection.html#examining-grid-data-in-a-fixed-resolution-array> if that helps isolate my question: all_data_level_2 = ds.covering_grid(level=2, left_edge=[0,0.0,0.0], dims=ds.domain_dimensions * 2**2)
Thanks in advance, Carla
_______________________________________________ yt-users mailing list yt-users@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
-- Michael Zingale Associate Professor
Dept. of Physics & Astronomy • Stony Brook University • Stony Brook, NY 11794-3800 *phone*: 631-632-8225 *e-mail*: Michael.Zingale@stonybrook.edu *web*: http://www.astro.sunysb.edu/mzingale github: http://github.com/zingale
_______________________________________________ yt-users mailing list yt-users@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
Hi Carla, So it looks like ref_factors is only defined in the boxlib frontend, which is why it doesn't work for your Enzo data. Boxlib needs a concept of ref_factors because it allows AMR refinement jumps larger than a factor of two on a single level (i.e. level 2 might be four times higher resolution as level 1). Enzo and many other AMR codes do not allow this, instead only allowing a factor of two increase in linear resolution per AMR level. The dimensions you want to use depend on what fraction of the full domain you want to interpolate to uniform resolution. It looks like you want to create a covering grid covering your full domain, so to create a covering grid at the same resolution as AMR level 3 you would do: all_data_level_3 = ds.covering_grid(level=2, left_edge=[0,0.0,0.0], dims=ds.domain_dimensions * 2**3) This is because each refinement level jump in an Enzo simulation corresponds to a factor of two increase in spatial resolution. Hope that makes sense, Nathan On Thu, Nov 19, 2015 at 8:58 AM, Carla Bernhardt < carla.j.bernhardt@gmail.com> wrote:
Thank you for your quick response. However, when I used:
ref = int(np.product(ds.ref_factors[0:3])),
I got this error:
AttributeError: 'EnzoDataset' object has no attribute 'ref_factors'
Did I misunderstand your suggestion? Or do I need to import something?
Thanks,
Carla
2015-11-19 15:39 GMT+01:00 Michael Zingale <michael.zingale@stonybrook.edu
:
I've done this in the past:
ref = int(np.product(ds.ref_factors[0:max_level]))
# allocate for our uniformly-gridded result
dims = ds.domain_dimensions*ref
this will work for a more general case when the jump between levels can change as a function of level.
On Thu, Nov 19, 2015 at 9:10 AM, Carla Bernhardt < carla.j.bernhardt@gmail.com> wrote:
Dear YT Users,
To better understand covering_grid (or smoothed_covering_grid), can someone explain what dimensions I should use when I have multiple levels of refinement? If I have 1 level of refinement from AMR data, the dimensions should be the same, I believe, but what if I have 2 or 3 levels of refinement? Should the fixed resolution region then have dimensions of dims*2^2 and dims*3^2 respectively?
Here is one example from a tutorial <http://yt-project.org/doc/examining/low_level_inspection.html#examining-grid-data-in-a-fixed-resolution-array> if that helps isolate my question: all_data_level_2 = ds.covering_grid(level=2, left_edge=[0,0.0,0.0], dims=ds.domain_dimensions * 2**2)
Thanks in advance, Carla
_______________________________________________ yt-users mailing list yt-users@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
-- Michael Zingale Associate Professor
Dept. of Physics & Astronomy • Stony Brook University • Stony Brook, NY 11794-3800 *phone*: 631-632-8225 *e-mail*: Michael.Zingale@stonybrook.edu *web*: http://www.astro.sunysb.edu/mzingale github: http://github.com/zingale
_______________________________________________ yt-users mailing list yt-users@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
_______________________________________________ yt-users mailing list yt-users@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
Hi Nathan, That makes sense. Thanks! Carla 2015-11-19 16:11 GMT+01:00 Nathan Goldbaum <nathan12343@gmail.com>:
Hi Carla,
So it looks like ref_factors is only defined in the boxlib frontend, which is why it doesn't work for your Enzo data. Boxlib needs a concept of ref_factors because it allows AMR refinement jumps larger than a factor of two on a single level (i.e. level 2 might be four times higher resolution as level 1). Enzo and many other AMR codes do not allow this, instead only allowing a factor of two increase in linear resolution per AMR level.
The dimensions you want to use depend on what fraction of the full domain you want to interpolate to uniform resolution. It looks like you want to create a covering grid covering your full domain, so to create a covering grid at the same resolution as AMR level 3 you would do:
all_data_level_3 = ds.covering_grid(level=2, left_edge=[0,0.0,0.0], dims=ds.domain_dimensions * 2**3)
This is because each refinement level jump in an Enzo simulation corresponds to a factor of two increase in spatial resolution.
Hope that makes sense,
Nathan
On Thu, Nov 19, 2015 at 8:58 AM, Carla Bernhardt < carla.j.bernhardt@gmail.com> wrote:
Thank you for your quick response. However, when I used:
ref = int(np.product(ds.ref_factors[0:3])),
I got this error:
AttributeError: 'EnzoDataset' object has no attribute 'ref_factors'
Did I misunderstand your suggestion? Or do I need to import something?
Thanks,
Carla
2015-11-19 15:39 GMT+01:00 Michael Zingale < michael.zingale@stonybrook.edu>:
I've done this in the past:
ref = int(np.product(ds.ref_factors[0:max_level]))
# allocate for our uniformly-gridded result
dims = ds.domain_dimensions*ref
this will work for a more general case when the jump between levels can change as a function of level.
On Thu, Nov 19, 2015 at 9:10 AM, Carla Bernhardt < carla.j.bernhardt@gmail.com> wrote:
Dear YT Users,
To better understand covering_grid (or smoothed_covering_grid), can someone explain what dimensions I should use when I have multiple levels of refinement? If I have 1 level of refinement from AMR data, the dimensions should be the same, I believe, but what if I have 2 or 3 levels of refinement? Should the fixed resolution region then have dimensions of dims*2^2 and dims*3^2 respectively?
Here is one example from a tutorial <http://yt-project.org/doc/examining/low_level_inspection.html#examining-grid-data-in-a-fixed-resolution-array> if that helps isolate my question: all_data_level_2 = ds.covering_grid(level=2, left_edge=[0,0.0,0.0], dims=ds.domain_dimensions * 2**2)
Thanks in advance, Carla
_______________________________________________ yt-users mailing list yt-users@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
-- Michael Zingale Associate Professor
Dept. of Physics & Astronomy • Stony Brook University • Stony Brook, NY 11794-3800 *phone*: 631-632-8225 *e-mail*: Michael.Zingale@stonybrook.edu *web*: http://www.astro.sunysb.edu/mzingale github: http://github.com/zingale
_______________________________________________ yt-users mailing list yt-users@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
_______________________________________________ yt-users mailing list yt-users@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
_______________________________________________ yt-users mailing list yt-users@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
Thanks again for the help. Once I implemented that, I have a follow-up problem: Once I have this much larger grid covering, it seems possibly too large for yt/python to handle. When I run: cg=ds.smoothed_covering_grid(3,[0,0,0], dims*2**3) data=cg['density'] (where before I had dims*2**1), the variable data does not load and gives a few errors <http://pastebin.com/JVhenc4z> which lead me to believe that python or yt will not let me use that much space. Can I force yt to let me use more space? Thanks, Carla 2015-11-19 16:11 GMT+01:00 Nathan Goldbaum <nathan12343@gmail.com>:
Hi Carla,
So it looks like ref_factors is only defined in the boxlib frontend, which is why it doesn't work for your Enzo data. Boxlib needs a concept of ref_factors because it allows AMR refinement jumps larger than a factor of two on a single level (i.e. level 2 might be four times higher resolution as level 1). Enzo and many other AMR codes do not allow this, instead only allowing a factor of two increase in linear resolution per AMR level.
The dimensions you want to use depend on what fraction of the full domain you want to interpolate to uniform resolution. It looks like you want to create a covering grid covering your full domain, so to create a covering grid at the same resolution as AMR level 3 you would do:
all_data_level_3 = ds.covering_grid(level=2, left_edge=[0,0.0,0.0], dims=ds.domain_dimensions * 2**3)
This is because each refinement level jump in an Enzo simulation corresponds to a factor of two increase in spatial resolution.
Hope that makes sense,
Nathan
On Thu, Nov 19, 2015 at 8:58 AM, Carla Bernhardt < carla.j.bernhardt@gmail.com> wrote:
Thank you for your quick response. However, when I used:
ref = int(np.product(ds.ref_factors[0:3])),
I got this error:
AttributeError: 'EnzoDataset' object has no attribute 'ref_factors'
Did I misunderstand your suggestion? Or do I need to import something?
Thanks,
Carla
2015-11-19 15:39 GMT+01:00 Michael Zingale < michael.zingale@stonybrook.edu>:
I've done this in the past:
ref = int(np.product(ds.ref_factors[0:max_level]))
# allocate for our uniformly-gridded result
dims = ds.domain_dimensions*ref
this will work for a more general case when the jump between levels can change as a function of level.
On Thu, Nov 19, 2015 at 9:10 AM, Carla Bernhardt < carla.j.bernhardt@gmail.com> wrote:
Dear YT Users,
To better understand covering_grid (or smoothed_covering_grid), can someone explain what dimensions I should use when I have multiple levels of refinement? If I have 1 level of refinement from AMR data, the dimensions should be the same, I believe, but what if I have 2 or 3 levels of refinement? Should the fixed resolution region then have dimensions of dims*2^2 and dims*3^2 respectively?
Here is one example from a tutorial <http://yt-project.org/doc/examining/low_level_inspection.html#examining-grid-data-in-a-fixed-resolution-array> if that helps isolate my question: all_data_level_2 = ds.covering_grid(level=2, left_edge=[0,0.0,0.0], dims=ds.domain_dimensions * 2**2)
Thanks in advance, Carla
_______________________________________________ yt-users mailing list yt-users@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
-- Michael Zingale Associate Professor
Dept. of Physics & Astronomy • Stony Brook University • Stony Brook, NY 11794-3800 *phone*: 631-632-8225 *e-mail*: Michael.Zingale@stonybrook.edu *web*: http://www.astro.sunysb.edu/mzingale github: http://github.com/zingale
_______________________________________________ yt-users mailing list yt-users@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
_______________________________________________ yt-users mailing list yt-users@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
_______________________________________________ yt-users mailing list yt-users@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
What is dims*2**3? Depending on your data this might be a very large array that won't fit in memory on your machine. You will need to find a machine that has more RAM to do this or create a covering grid that only covers a subset of the domain. On Thursday, November 19, 2015, Carla Bernhardt <carla.j.bernhardt@gmail.com> wrote:
Thanks again for the help. Once I implemented that, I have a follow-up problem: Once I have this much larger grid covering, it seems possibly too large for yt/python to handle. When I run:
cg=ds.smoothed_covering_grid(3,[0,0,0], dims*2**3) data=cg['density']
(where before I had dims*2**1), the variable data does not load and gives a few errors <http://pastebin.com/JVhenc4z> which lead me to believe that python or yt will not let me use that much space. Can I force yt to let me use more space?
Thanks, Carla
2015-11-19 16:11 GMT+01:00 Nathan Goldbaum <nathan12343@gmail.com <javascript:_e(%7B%7D,'cvml','nathan12343@gmail.com');>>:
Hi Carla,
So it looks like ref_factors is only defined in the boxlib frontend, which is why it doesn't work for your Enzo data. Boxlib needs a concept of ref_factors because it allows AMR refinement jumps larger than a factor of two on a single level (i.e. level 2 might be four times higher resolution as level 1). Enzo and many other AMR codes do not allow this, instead only allowing a factor of two increase in linear resolution per AMR level.
The dimensions you want to use depend on what fraction of the full domain you want to interpolate to uniform resolution. It looks like you want to create a covering grid covering your full domain, so to create a covering grid at the same resolution as AMR level 3 you would do:
all_data_level_3 = ds.covering_grid(level=2, left_edge=[0,0.0,0.0], dims=ds.domain_dimensions * 2**3)
This is because each refinement level jump in an Enzo simulation corresponds to a factor of two increase in spatial resolution.
Hope that makes sense,
Nathan
On Thu, Nov 19, 2015 at 8:58 AM, Carla Bernhardt < carla.j.bernhardt@gmail.com <javascript:_e(%7B%7D,'cvml','carla.j.bernhardt@gmail.com');>> wrote:
Thank you for your quick response. However, when I used:
ref = int(np.product(ds.ref_factors[0:3])),
I got this error:
AttributeError: 'EnzoDataset' object has no attribute 'ref_factors'
Did I misunderstand your suggestion? Or do I need to import something?
Thanks,
Carla
2015-11-19 15:39 GMT+01:00 Michael Zingale < michael.zingale@stonybrook.edu <javascript:_e(%7B%7D,'cvml','michael.zingale@stonybrook.edu');>>:
I've done this in the past:
ref = int(np.product(ds.ref_factors[0:max_level]))
# allocate for our uniformly-gridded result
dims = ds.domain_dimensions*ref
this will work for a more general case when the jump between levels can change as a function of level.
On Thu, Nov 19, 2015 at 9:10 AM, Carla Bernhardt < carla.j.bernhardt@gmail.com <javascript:_e(%7B%7D,'cvml','carla.j.bernhardt@gmail.com');>> wrote:
Dear YT Users,
To better understand covering_grid (or smoothed_covering_grid), can someone explain what dimensions I should use when I have multiple levels of refinement? If I have 1 level of refinement from AMR data, the dimensions should be the same, I believe, but what if I have 2 or 3 levels of refinement? Should the fixed resolution region then have dimensions of dims*2^2 and dims*3^2 respectively?
Here is one example from a tutorial <http://yt-project.org/doc/examining/low_level_inspection.html#examining-grid-data-in-a-fixed-resolution-array> if that helps isolate my question: all_data_level_2 = ds.covering_grid(level=2, left_edge=[0,0.0,0.0], dims=ds.domain_dimensions * 2**2)
Thanks in advance, Carla
_______________________________________________ yt-users mailing list yt-users@lists.spacepope.org <javascript:_e(%7B%7D,'cvml','yt-users@lists.spacepope.org');> http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
-- Michael Zingale Associate Professor
Dept. of Physics & Astronomy • Stony Brook University • Stony Brook, NY 11794-3800 *phone*: 631-632-8225 *e-mail*: Michael.Zingale@stonybrook.edu <javascript:_e(%7B%7D,'cvml','Michael.Zingale@stonybrook.edu');> *web*: http://www.astro.sunysb.edu/mzingale github: http://github.com/zingale
_______________________________________________ yt-users mailing list yt-users@lists.spacepope.org <javascript:_e(%7B%7D,'cvml','yt-users@lists.spacepope.org');> http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
_______________________________________________ yt-users mailing list yt-users@lists.spacepope.org <javascript:_e(%7B%7D,'cvml','yt-users@lists.spacepope.org');> http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
_______________________________________________ yt-users mailing list yt-users@lists.spacepope.org <javascript:_e(%7B%7D,'cvml','yt-users@lists.spacepope.org');> http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
Nathan, Dims*2**3 is 256*2^3 or 2048. With the data I am working with, I think this should be on the order of 1 GB. I don't think that should overload the memory... Is it possible there is something incorrectly defined? Thanks, Carla Am Donnerstag, 19. November 2015 schrieb Nathan Goldbaum :
What is dims*2**3? Depending on your data this might be a very large array that won't fit in memory on your machine.
You will need to find a machine that has more RAM to do this or create a covering grid that only covers a subset of the domain.
On Thursday, November 19, 2015, Carla Bernhardt < carla.j.bernhardt@gmail.com <javascript:_e(%7B%7D,'cvml','carla.j.bernhardt@gmail.com');>> wrote:
Thanks again for the help. Once I implemented that, I have a follow-up problem: Once I have this much larger grid covering, it seems possibly too large for yt/python to handle. When I run:
cg=ds.smoothed_covering_grid(3,[0,0,0], dims*2**3) data=cg['density']
(where before I had dims*2**1), the variable data does not load and gives a few errors <http://pastebin.com/JVhenc4z> which lead me to believe that python or yt will not let me use that much space. Can I force yt to let me use more space?
Thanks, Carla
2015-11-19 16:11 GMT+01:00 Nathan Goldbaum <nathan12343@gmail.com>:
Hi Carla,
So it looks like ref_factors is only defined in the boxlib frontend, which is why it doesn't work for your Enzo data. Boxlib needs a concept of ref_factors because it allows AMR refinement jumps larger than a factor of two on a single level (i.e. level 2 might be four times higher resolution as level 1). Enzo and many other AMR codes do not allow this, instead only allowing a factor of two increase in linear resolution per AMR level.
The dimensions you want to use depend on what fraction of the full domain you want to interpolate to uniform resolution. It looks like you want to create a covering grid covering your full domain, so to create a covering grid at the same resolution as AMR level 3 you would do:
all_data_level_3 = ds.covering_grid(level=2, left_edge=[0,0.0,0.0], dims=ds.domain_dimensions * 2**3)
This is because each refinement level jump in an Enzo simulation corresponds to a factor of two increase in spatial resolution.
Hope that makes sense,
Nathan
On Thu, Nov 19, 2015 at 8:58 AM, Carla Bernhardt < carla.j.bernhardt@gmail.com> wrote:
Thank you for your quick response. However, when I used:
ref = int(np.product(ds.ref_factors[0:3])),
I got this error:
AttributeError: 'EnzoDataset' object has no attribute 'ref_factors'
Did I misunderstand your suggestion? Or do I need to import something?
Thanks,
Carla
2015-11-19 15:39 GMT+01:00 Michael Zingale < michael.zingale@stonybrook.edu>:
I've done this in the past:
ref = int(np.product(ds.ref_factors[0:max_level]))
# allocate for our uniformly-gridded result
dims = ds.domain_dimensions*ref
this will work for a more general case when the jump between levels can change as a function of level.
On Thu, Nov 19, 2015 at 9:10 AM, Carla Bernhardt < carla.j.bernhardt@gmail.com> wrote:
Dear YT Users,
To better understand covering_grid (or smoothed_covering_grid), can someone explain what dimensions I should use when I have multiple levels of refinement? If I have 1 level of refinement from AMR data, the dimensions should be the same, I believe, but what if I have 2 or 3 levels of refinement? Should the fixed resolution region then have dimensions of dims*2^2 and dims*3^2 respectively?
Here is one example from a tutorial <http://yt-project.org/doc/examining/low_level_inspection.html#examining-grid-data-in-a-fixed-resolution-array> if that helps isolate my question: all_data_level_2 = ds.covering_grid(level=2, left_edge=[0,0.0,0.0], dims=ds.domain_dimensions * 2**2)
Thanks in advance, Carla
_______________________________________________ yt-users mailing list yt-users@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
-- Michael Zingale Associate Professor
Dept. of Physics & Astronomy • Stony Brook University • Stony Brook, NY 11794-3800 *phone*: 631-632-8225 *e-mail*: Michael.Zingale@stonybrook.edu *web*: http://www.astro.sunysb.edu/mzingale github: http://github.com/zingale
_______________________________________________ yt-users mailing list yt-users@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
_______________________________________________ yt-users mailing list yt-users@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
_______________________________________________ yt-users mailing list yt-users@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
If I understand this, you are saying it will be a 2048**3 uniform grid. At 8 bytes per #, this is 64 GB of memory to hold the data. On Thu, Nov 19, 2015 at 12:45 PM, Carla Bernhardt < carla.j.bernhardt@gmail.com> wrote:
Nathan,
Dims*2**3 is 256*2^3 or 2048. With the data I am working with, I think this should be on the order of 1 GB. I don't think that should overload the memory... Is it possible there is something incorrectly defined?
Thanks, Carla
Am Donnerstag, 19. November 2015 schrieb Nathan Goldbaum :
What is dims*2**3? Depending on your data this might be a very large array that won't fit in memory on your machine.
You will need to find a machine that has more RAM to do this or create a covering grid that only covers a subset of the domain.
On Thursday, November 19, 2015, Carla Bernhardt < carla.j.bernhardt@gmail.com> wrote:
Thanks again for the help. Once I implemented that, I have a follow-up problem: Once I have this much larger grid covering, it seems possibly too large for yt/python to handle. When I run:
cg=ds.smoothed_covering_grid(3,[0,0,0], dims*2**3) data=cg['density']
(where before I had dims*2**1), the variable data does not load and gives a few errors <http://pastebin.com/JVhenc4z> which lead me to believe that python or yt will not let me use that much space. Can I force yt to let me use more space?
Thanks, Carla
2015-11-19 16:11 GMT+01:00 Nathan Goldbaum <nathan12343@gmail.com>:
Hi Carla,
So it looks like ref_factors is only defined in the boxlib frontend, which is why it doesn't work for your Enzo data. Boxlib needs a concept of ref_factors because it allows AMR refinement jumps larger than a factor of two on a single level (i.e. level 2 might be four times higher resolution as level 1). Enzo and many other AMR codes do not allow this, instead only allowing a factor of two increase in linear resolution per AMR level.
The dimensions you want to use depend on what fraction of the full domain you want to interpolate to uniform resolution. It looks like you want to create a covering grid covering your full domain, so to create a covering grid at the same resolution as AMR level 3 you would do:
all_data_level_3 = ds.covering_grid(level=2, left_edge=[0,0.0,0.0], dims=ds.domain_dimensions * 2**3)
This is because each refinement level jump in an Enzo simulation corresponds to a factor of two increase in spatial resolution.
Hope that makes sense,
Nathan
On Thu, Nov 19, 2015 at 8:58 AM, Carla Bernhardt < carla.j.bernhardt@gmail.com> wrote:
Thank you for your quick response. However, when I used:
ref = int(np.product(ds.ref_factors[0:3])),
I got this error:
AttributeError: 'EnzoDataset' object has no attribute 'ref_factors'
Did I misunderstand your suggestion? Or do I need to import something?
Thanks,
Carla
2015-11-19 15:39 GMT+01:00 Michael Zingale < michael.zingale@stonybrook.edu>:
I've done this in the past:
ref = int(np.product(ds.ref_factors[0:max_level]))
# allocate for our uniformly-gridded result
dims = ds.domain_dimensions*ref
this will work for a more general case when the jump between levels can change as a function of level.
On Thu, Nov 19, 2015 at 9:10 AM, Carla Bernhardt < carla.j.bernhardt@gmail.com> wrote:
> Dear YT Users, > > To better understand covering_grid (or smoothed_covering_grid), can > someone explain what dimensions I should use when I have multiple levels of > refinement? If I have 1 level of refinement from AMR data, the dimensions > should be the same, I believe, but what if I have 2 or 3 levels of > refinement? Should the fixed resolution region then have dimensions of > dims*2^2 and dims*3^2 respectively? > > Here is one example from a tutorial > <http://yt-project.org/doc/examining/low_level_inspection.html#examining-grid-data-in-a-fixed-resolution-array> > if that helps isolate my question: > all_data_level_2 = ds.covering_grid(level=2, left_edge=[0,0.0,0.0], > dims=ds.domain_dimensions * > 2**2) > > Thanks in advance, > Carla > > _______________________________________________ > yt-users mailing list > yt-users@lists.spacepope.org > http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org > >
-- Michael Zingale Associate Professor
Dept. of Physics & Astronomy • Stony Brook University • Stony Brook, NY 11794-3800 *phone*: 631-632-8225 *e-mail*: Michael.Zingale@stonybrook.edu *web*: http://www.astro.sunysb.edu/mzingale github: http://github.com/zingale
_______________________________________________ yt-users mailing list yt-users@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
_______________________________________________ yt-users mailing list yt-users@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
_______________________________________________ yt-users mailing list yt-users@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
yt-users mailing list yt-users@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
-- Michael Zingale Associate Professor Dept. of Physics & Astronomy • Stony Brook University • Stony Brook, NY 11794-3800 *phone*: 631-632-8225 *e-mail*: Michael.Zingale@stonybrook.edu *web*: http://www.astro.sunysb.edu/mzingale github: http://github.com/zingale
Dear yt users, Thanks again for the help on this topic. To make this work with the available memory, I want to take a cut region from the data set and put it into the smoothed_covering_grid(). I know smoothed_covering_grid() uses a left_edge, but why not a right_edge? Can I cut it after it is made? I have tried cut_region(), box() and region() but none of these support smoothed_covering_grid() after application. I have a center and a width in code_length units. Cheers, Carla 2015-11-19 18:48 GMT+01:00 Michael Zingale <michael.zingale@stonybrook.edu>:
If I understand this, you are saying it will be a 2048**3 uniform grid. At 8 bytes per #, this is 64 GB of memory to hold the data.
On Thu, Nov 19, 2015 at 12:45 PM, Carla Bernhardt < carla.j.bernhardt@gmail.com> wrote:
Nathan,
Dims*2**3 is 256*2^3 or 2048. With the data I am working with, I think this should be on the order of 1 GB. I don't think that should overload the memory... Is it possible there is something incorrectly defined?
Thanks, Carla
Am Donnerstag, 19. November 2015 schrieb Nathan Goldbaum :
What is dims*2**3? Depending on your data this might be a very large array that won't fit in memory on your machine.
You will need to find a machine that has more RAM to do this or create a covering grid that only covers a subset of the domain.
On Thursday, November 19, 2015, Carla Bernhardt < carla.j.bernhardt@gmail.com> wrote:
Thanks again for the help. Once I implemented that, I have a follow-up problem: Once I have this much larger grid covering, it seems possibly too large for yt/python to handle. When I run:
cg=ds.smoothed_covering_grid(3,[0,0,0], dims*2**3) data=cg['density']
(where before I had dims*2**1), the variable data does not load and gives a few errors <http://pastebin.com/JVhenc4z> which lead me to believe that python or yt will not let me use that much space. Can I force yt to let me use more space?
Thanks, Carla
2015-11-19 16:11 GMT+01:00 Nathan Goldbaum <nathan12343@gmail.com>:
Hi Carla,
So it looks like ref_factors is only defined in the boxlib frontend, which is why it doesn't work for your Enzo data. Boxlib needs a concept of ref_factors because it allows AMR refinement jumps larger than a factor of two on a single level (i.e. level 2 might be four times higher resolution as level 1). Enzo and many other AMR codes do not allow this, instead only allowing a factor of two increase in linear resolution per AMR level.
The dimensions you want to use depend on what fraction of the full domain you want to interpolate to uniform resolution. It looks like you want to create a covering grid covering your full domain, so to create a covering grid at the same resolution as AMR level 3 you would do:
all_data_level_3 = ds.covering_grid(level=2, left_edge=[0,0.0,0.0], dims=ds.domain_dimensions * 2**3)
This is because each refinement level jump in an Enzo simulation corresponds to a factor of two increase in spatial resolution.
Hope that makes sense,
Nathan
On Thu, Nov 19, 2015 at 8:58 AM, Carla Bernhardt < carla.j.bernhardt@gmail.com> wrote:
Thank you for your quick response. However, when I used:
ref = int(np.product(ds.ref_factors[0:3])),
I got this error:
AttributeError: 'EnzoDataset' object has no attribute 'ref_factors'
Did I misunderstand your suggestion? Or do I need to import something?
Thanks,
Carla
2015-11-19 15:39 GMT+01:00 Michael Zingale < michael.zingale@stonybrook.edu>:
> I've done this in the past: > > ref = int(np.product(ds.ref_factors[0:max_level])) > > > > # allocate for our uniformly-gridded result > > dims = ds.domain_dimensions*ref > > this will work for a more general case when the jump between levels > can change as a function of level. > > On Thu, Nov 19, 2015 at 9:10 AM, Carla Bernhardt < > carla.j.bernhardt@gmail.com> wrote: > >> Dear YT Users, >> >> To better understand covering_grid (or smoothed_covering_grid), >> can someone explain what dimensions I should use when I have multiple >> levels of refinement? If I have 1 level of refinement from AMR data, the >> dimensions should be the same, I believe, but what if I have 2 or 3 levels >> of refinement? Should the fixed resolution region then have dimensions of >> dims*2^2 and dims*3^2 respectively? >> >> Here is one example from a tutorial >> <http://yt-project.org/doc/examining/low_level_inspection.html#examining-grid-data-in-a-fixed-resolution-array> >> if that helps isolate my question: >> all_data_level_2 = ds.covering_grid(level=2, left_edge=[0,0.0,0.0], >> dims=ds.domain_dimensions * >> 2**2) >> >> Thanks in advance, >> Carla >> >> _______________________________________________ >> yt-users mailing list >> yt-users@lists.spacepope.org >> http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org >> >> > > > -- > Michael Zingale > Associate Professor > > Dept. of Physics & Astronomy • Stony Brook University • Stony Brook, > NY 11794-3800 > *phone*: 631-632-8225 > *e-mail*: Michael.Zingale@stonybrook.edu > *web*: http://www.astro.sunysb.edu/mzingale > github: http://github.com/zingale > > > _______________________________________________ > yt-users mailing list > yt-users@lists.spacepope.org > http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org > >
_______________________________________________ yt-users mailing list yt-users@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
_______________________________________________ yt-users mailing list yt-users@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
yt-users mailing list yt-users@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
-- Michael Zingale Associate Professor
Dept. of Physics & Astronomy • Stony Brook University • Stony Brook, NY 11794-3800 *phone*: 631-632-8225 *e-mail*: Michael.Zingale@stonybrook.edu *web*: http://www.astro.sunysb.edu/mzingale github: http://github.com/zingale
_______________________________________________ yt-users mailing list yt-users@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
On Thu, Nov 26, 2015 at 7:19 AM, Carla Bernhardt < carla.j.bernhardt@gmail.com> wrote:
Dear yt users,
Thanks again for the help on this topic. To make this work with the available memory, I want to take a cut region from the data set and put it into the smoothed_covering_grid(). I know smoothed_covering_grid() uses a left_edge, but why not a right_edge? Can I cut it after it is made? I have tried cut_region(), box() and region() but none of these support smoothed_covering_grid() after application. I have a center and a width in code_length units.
Both covering_grid and smoothed_covering_grid work the same way, so below I'll only refer to covering_grid, even though this applies equally to both. The job of covering_grid is to create a uniform resolution representation of an AMR simulation, at the same resolution of one of the AMR levels. This explains why you need to specify the AMR level you want to sample at - an AMR simulation has *multiple* spatial resolutions, one for each AMR level, so covering_grid needs a user to tell it which resolution to sample at. The AMR level determines the linear cell spacing of the resulting uniform resolution grid. In general, for a simulation with grid dimensions on level 0 given by "domain_dimensions", and a box size in code units given by box_size, the linear resolution on AMR level "level" is (in python pseudo-code, where both box_size and domain_dimensions act like numpy arrays): dx = box_size / (domain_dimensions * 2**level) In addition, covering_grid needs to know where in the simulation to begin sampling. This is left_edge, the bottom-left hand corner of the region to sample over. In general left_edge needs to be carefully chosen so it falls exactly at a zone boundary at the AMR level you're sampling at. I believe that if this isn't the case, it will either floor or ceiling left_edge to the nearest zone boundary. You might also expect to be able to to specify a right_edge, however, this would be very misleading and possibly surprising, so it is not done this way. This is because an arbitrary right_edge will most likely not happen to fall an exact integer number of zones away from the left_edge. Instead, covering_grid lets users specify the integer *dimensions* of the resulting grid. To calculate right_edge from the dims, you can do (again interpreting the arrays below as numpy arrays): right_edge = left_edge + dims*dx where dx is from the first equation I gave above. This is definitely not intuitive (I was confused by it at first) but it is a sensible way of doing it once you realize the limitations imposed by the problem. Using covering_grid is "nice" because it's a "natural" way to represent AMR data: you are doing the minimum possible amount of resampling or interpolating, and are using the "real" simulation data on the AMR level you are sampling at. I should also mention that yt also provides arbitrary_grid, which *does* take a left_edge, right_edge, and dims. On the stable branch all extant stable releases of yt, arbitrary_grid can only be used for deposited particle fields. However, in the upcoming yt 3.3. release and in the current development arbitrary_grid can be used for AMR mesh fields like ('gas', 'density'). While arbitrary_grid has an easier to understand API, it is also not as natural a representation of AMR data as covering_grid, since unless left_edge, right_edge, and the dimensions of your arbitrary_grid are carefully chosen, every zone in the result will be affected by resampling or interpolating. Hope that explanation helped! -Nathan
Cheers, Carla
2015-11-19 18:48 GMT+01:00 Michael Zingale <michael.zingale@stonybrook.edu
:
If I understand this, you are saying it will be a 2048**3 uniform grid. At 8 bytes per #, this is 64 GB of memory to hold the data.
On Thu, Nov 19, 2015 at 12:45 PM, Carla Bernhardt < carla.j.bernhardt@gmail.com> wrote:
Nathan,
Dims*2**3 is 256*2^3 or 2048. With the data I am working with, I think this should be on the order of 1 GB. I don't think that should overload the memory... Is it possible there is something incorrectly defined?
Thanks, Carla
Am Donnerstag, 19. November 2015 schrieb Nathan Goldbaum :
What is dims*2**3? Depending on your data this might be a very large array that won't fit in memory on your machine.
You will need to find a machine that has more RAM to do this or create a covering grid that only covers a subset of the domain.
On Thursday, November 19, 2015, Carla Bernhardt < carla.j.bernhardt@gmail.com> wrote:
Thanks again for the help. Once I implemented that, I have a follow-up problem: Once I have this much larger grid covering, it seems possibly too large for yt/python to handle. When I run:
cg=ds.smoothed_covering_grid(3,[0,0,0], dims*2**3) data=cg['density']
(where before I had dims*2**1), the variable data does not load and gives a few errors <http://pastebin.com/JVhenc4z> which lead me to believe that python or yt will not let me use that much space. Can I force yt to let me use more space?
Thanks, Carla
2015-11-19 16:11 GMT+01:00 Nathan Goldbaum <nathan12343@gmail.com>:
Hi Carla,
So it looks like ref_factors is only defined in the boxlib frontend, which is why it doesn't work for your Enzo data. Boxlib needs a concept of ref_factors because it allows AMR refinement jumps larger than a factor of two on a single level (i.e. level 2 might be four times higher resolution as level 1). Enzo and many other AMR codes do not allow this, instead only allowing a factor of two increase in linear resolution per AMR level.
The dimensions you want to use depend on what fraction of the full domain you want to interpolate to uniform resolution. It looks like you want to create a covering grid covering your full domain, so to create a covering grid at the same resolution as AMR level 3 you would do:
all_data_level_3 = ds.covering_grid(level=2, left_edge=[0,0.0,0.0], dims=ds.domain_dimensions * 2**3)
This is because each refinement level jump in an Enzo simulation corresponds to a factor of two increase in spatial resolution.
Hope that makes sense,
Nathan
On Thu, Nov 19, 2015 at 8:58 AM, Carla Bernhardt < carla.j.bernhardt@gmail.com> wrote:
> Thank you for your quick response. However, when I used: > > ref = int(np.product(ds.ref_factors[0:3])), > > I got this error: > > AttributeError: 'EnzoDataset' object has no attribute 'ref_factors' > > Did I misunderstand your suggestion? Or do I need to import > something? > > Thanks, > > Carla > > 2015-11-19 15:39 GMT+01:00 Michael Zingale < > michael.zingale@stonybrook.edu>: > >> I've done this in the past: >> >> ref = int(np.product(ds.ref_factors[0:max_level])) >> >> >> >> # allocate for our uniformly-gridded result >> >> dims = ds.domain_dimensions*ref >> >> this will work for a more general case when the jump between levels >> can change as a function of level. >> >> On Thu, Nov 19, 2015 at 9:10 AM, Carla Bernhardt < >> carla.j.bernhardt@gmail.com> wrote: >> >>> Dear YT Users, >>> >>> To better understand covering_grid (or smoothed_covering_grid), >>> can someone explain what dimensions I should use when I have multiple >>> levels of refinement? If I have 1 level of refinement from AMR data, the >>> dimensions should be the same, I believe, but what if I have 2 or 3 levels >>> of refinement? Should the fixed resolution region then have dimensions of >>> dims*2^2 and dims*3^2 respectively? >>> >>> Here is one example from a tutorial >>> <http://yt-project.org/doc/examining/low_level_inspection.html#examining-grid-data-in-a-fixed-resolution-array> >>> if that helps isolate my question: >>> all_data_level_2 = ds.covering_grid(level=2, left_edge=[0,0.0,0.0], >>> dims=ds.domain_dimensions * >>> 2**2) >>> >>> Thanks in advance, >>> Carla >>> >>> _______________________________________________ >>> yt-users mailing list >>> yt-users@lists.spacepope.org >>> http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org >>> >>> >> >> >> -- >> Michael Zingale >> Associate Professor >> >> Dept. of Physics & Astronomy • Stony Brook University • Stony >> Brook, NY 11794-3800 >> *phone*: 631-632-8225 >> *e-mail*: Michael.Zingale@stonybrook.edu >> *web*: http://www.astro.sunysb.edu/mzingale >> github: http://github.com/zingale >> >> >> _______________________________________________ >> yt-users mailing list >> yt-users@lists.spacepope.org >> http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org >> >> > > _______________________________________________ > yt-users mailing list > yt-users@lists.spacepope.org > http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org > >
_______________________________________________ yt-users mailing list yt-users@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
yt-users mailing list yt-users@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
-- Michael Zingale Associate Professor
Dept. of Physics & Astronomy • Stony Brook University • Stony Brook, NY 11794-3800 *phone*: 631-632-8225 *e-mail*: Michael.Zingale@stonybrook.edu *web*: http://www.astro.sunysb.edu/mzingale github: http://github.com/zingale
_______________________________________________ yt-users mailing list yt-users@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
_______________________________________________ yt-users mailing list yt-users@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
Nathan, Thank you for your explanation. I had looked at arbitrary_grid, but I want to confirm that the data is the same as my AMR data and I thought this would present a problem. You mention that I "might also expect to be able to specify a right_edge", but where is this right_edge implemented? I tried just putting in right_edge=left_edge + dims*dx in the smoothed_covering_grid command, but it throws an error that there is no right_edge (which I knew from reading the documentation, I was just hoping I would get lucky). Thanks, Carla 2015-11-26 15:13 GMT+01:00 Nathan Goldbaum <nathan12343@gmail.com>:
On Thu, Nov 26, 2015 at 7:19 AM, Carla Bernhardt < carla.j.bernhardt@gmail.com> wrote:
Dear yt users,
Thanks again for the help on this topic. To make this work with the available memory, I want to take a cut region from the data set and put it into the smoothed_covering_grid(). I know smoothed_covering_grid() uses a left_edge, but why not a right_edge? Can I cut it after it is made? I have tried cut_region(), box() and region() but none of these support smoothed_covering_grid() after application. I have a center and a width in code_length units.
Both covering_grid and smoothed_covering_grid work the same way, so below I'll only refer to covering_grid, even though this applies equally to both.
The job of covering_grid is to create a uniform resolution representation of an AMR simulation, at the same resolution of one of the AMR levels. This explains why you need to specify the AMR level you want to sample at - an AMR simulation has *multiple* spatial resolutions, one for each AMR level, so covering_grid needs a user to tell it which resolution to sample at.
The AMR level determines the linear cell spacing of the resulting uniform resolution grid. In general, for a simulation with grid dimensions on level 0 given by "domain_dimensions", and a box size in code units given by box_size, the linear resolution on AMR level "level" is (in python pseudo-code, where both box_size and domain_dimensions act like numpy arrays):
dx = box_size / (domain_dimensions * 2**level)
In addition, covering_grid needs to know where in the simulation to begin sampling. This is left_edge, the bottom-left hand corner of the region to sample over. In general left_edge needs to be carefully chosen so it falls exactly at a zone boundary at the AMR level you're sampling at. I believe that if this isn't the case, it will either floor or ceiling left_edge to the nearest zone boundary.
You might also expect to be able to to specify a right_edge, however, this would be very misleading and possibly surprising, so it is not done this way. This is because an arbitrary right_edge will most likely not happen to fall an exact integer number of zones away from the left_edge. Instead, covering_grid lets users specify the integer *dimensions* of the resulting grid. To calculate right_edge from the dims, you can do (again interpreting the arrays below as numpy arrays):
right_edge = left_edge + dims*dx
where dx is from the first equation I gave above.
This is definitely not intuitive (I was confused by it at first) but it is a sensible way of doing it once you realize the limitations imposed by the problem. Using covering_grid is "nice" because it's a "natural" way to represent AMR data: you are doing the minimum possible amount of resampling or interpolating, and are using the "real" simulation data on the AMR level you are sampling at.
I should also mention that yt also provides arbitrary_grid, which *does* take a left_edge, right_edge, and dims. On the stable branch all extant stable releases of yt, arbitrary_grid can only be used for deposited particle fields. However, in the upcoming yt 3.3. release and in the current development arbitrary_grid can be used for AMR mesh fields like ('gas', 'density'). While arbitrary_grid has an easier to understand API, it is also not as natural a representation of AMR data as covering_grid, since unless left_edge, right_edge, and the dimensions of your arbitrary_grid are carefully chosen, every zone in the result will be affected by resampling or interpolating.
Hope that explanation helped!
-Nathan
Cheers, Carla
2015-11-19 18:48 GMT+01:00 Michael Zingale < michael.zingale@stonybrook.edu>:
If I understand this, you are saying it will be a 2048**3 uniform grid. At 8 bytes per #, this is 64 GB of memory to hold the data.
On Thu, Nov 19, 2015 at 12:45 PM, Carla Bernhardt < carla.j.bernhardt@gmail.com> wrote:
Nathan,
Dims*2**3 is 256*2^3 or 2048. With the data I am working with, I think this should be on the order of 1 GB. I don't think that should overload the memory... Is it possible there is something incorrectly defined?
Thanks, Carla
Am Donnerstag, 19. November 2015 schrieb Nathan Goldbaum :
What is dims*2**3? Depending on your data this might be a very large array that won't fit in memory on your machine.
You will need to find a machine that has more RAM to do this or create a covering grid that only covers a subset of the domain.
On Thursday, November 19, 2015, Carla Bernhardt < carla.j.bernhardt@gmail.com> wrote:
Thanks again for the help. Once I implemented that, I have a follow-up problem: Once I have this much larger grid covering, it seems possibly too large for yt/python to handle. When I run:
cg=ds.smoothed_covering_grid(3,[0,0,0], dims*2**3) data=cg['density']
(where before I had dims*2**1), the variable data does not load and gives a few errors <http://pastebin.com/JVhenc4z> which lead me to believe that python or yt will not let me use that much space. Can I force yt to let me use more space?
Thanks, Carla
2015-11-19 16:11 GMT+01:00 Nathan Goldbaum <nathan12343@gmail.com>:
> Hi Carla, > > So it looks like ref_factors is only defined in the boxlib frontend, > which is why it doesn't work for your Enzo data. Boxlib needs a concept of > ref_factors because it allows AMR refinement jumps larger than a factor of > two on a single level (i.e. level 2 might be four times higher resolution > as level 1). Enzo and many other AMR codes do not allow this, instead only > allowing a factor of two increase in linear resolution per AMR level. > > The dimensions you want to use depend on what fraction of the full > domain you want to interpolate to uniform resolution. It looks like you > want to create a covering grid covering your full domain, so to create a > covering grid at the same resolution as AMR level 3 you would do: > > all_data_level_3 = ds.covering_grid(level=2, left_edge=[0,0.0,0.0], > dims=ds.domain_dimensions * 2**3) > > This is because each refinement level jump in an Enzo simulation > corresponds to a factor of two increase in spatial resolution. > > Hope that makes sense, > > Nathan > > On Thu, Nov 19, 2015 at 8:58 AM, Carla Bernhardt < > carla.j.bernhardt@gmail.com> wrote: > >> Thank you for your quick response. However, when I used: >> >> ref = int(np.product(ds.ref_factors[0:3])), >> >> I got this error: >> >> AttributeError: 'EnzoDataset' object has no attribute 'ref_factors' >> >> Did I misunderstand your suggestion? Or do I need to import >> something? >> >> Thanks, >> >> Carla >> >> 2015-11-19 15:39 GMT+01:00 Michael Zingale < >> michael.zingale@stonybrook.edu>: >> >>> I've done this in the past: >>> >>> ref = int(np.product(ds.ref_factors[0:max_level])) >>> >>> >>> >>> # allocate for our uniformly-gridded result >>> >>> dims = ds.domain_dimensions*ref >>> >>> this will work for a more general case when the jump between >>> levels can change as a function of level. >>> >>> On Thu, Nov 19, 2015 at 9:10 AM, Carla Bernhardt < >>> carla.j.bernhardt@gmail.com> wrote: >>> >>>> Dear YT Users, >>>> >>>> To better understand covering_grid (or smoothed_covering_grid), >>>> can someone explain what dimensions I should use when I have multiple >>>> levels of refinement? If I have 1 level of refinement from AMR data, the >>>> dimensions should be the same, I believe, but what if I have 2 or 3 levels >>>> of refinement? Should the fixed resolution region then have dimensions of >>>> dims*2^2 and dims*3^2 respectively? >>>> >>>> Here is one example from a tutorial >>>> <http://yt-project.org/doc/examining/low_level_inspection.html#examining-grid-data-in-a-fixed-resolution-array> >>>> if that helps isolate my question: >>>> all_data_level_2 = ds.covering_grid(level=2, >>>> left_edge=[0,0.0,0.0], >>>> dims=ds.domain_dimensions * >>>> 2**2) >>>> >>>> Thanks in advance, >>>> Carla >>>> >>>> _______________________________________________ >>>> yt-users mailing list >>>> yt-users@lists.spacepope.org >>>> http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org >>>> >>>> >>> >>> >>> -- >>> Michael Zingale >>> Associate Professor >>> >>> Dept. of Physics & Astronomy • Stony Brook University • Stony >>> Brook, NY 11794-3800 >>> *phone*: 631-632-8225 >>> *e-mail*: Michael.Zingale@stonybrook.edu >>> *web*: http://www.astro.sunysb.edu/mzingale >>> github: http://github.com/zingale >>> >>> >>> _______________________________________________ >>> yt-users mailing list >>> yt-users@lists.spacepope.org >>> http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org >>> >>> >> >> _______________________________________________ >> yt-users mailing list >> yt-users@lists.spacepope.org >> http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org >> >> > > _______________________________________________ > yt-users mailing list > yt-users@lists.spacepope.org > http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org > >
yt-users mailing list yt-users@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
-- Michael Zingale Associate Professor
Dept. of Physics & Astronomy • Stony Brook University • Stony Brook, NY 11794-3800 *phone*: 631-632-8225 *e-mail*: Michael.Zingale@stonybrook.edu *web*: http://www.astro.sunysb.edu/mzingale github: http://github.com/zingale
_______________________________________________ yt-users mailing list yt-users@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
_______________________________________________ yt-users mailing list yt-users@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
_______________________________________________ yt-users mailing list yt-users@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
On Thursday, November 26, 2015, Carla Bernhardt <carla.j.bernhardt@gmail.com> wrote:
Nathan,
Thank you for your explanation. I had looked at arbitrary_grid, but I want to confirm that the data is the same as my AMR data and I thought this would present a problem. You mention that I "might also expect to be able to specify a right_edge", but where is this right_edge implemented? I tried just putting in right_edge=left_edge + dims*dx in the smoothed_covering_grid command, but it throws an error that there is no right_edge (which I knew from reading the documentation, I was just hoping I would get lucky).
I was trying to explain why you can't specify a right_edge directly. You'll likely want to calculate what it is, but can't directly specify it when creating the covering grid.
Thanks, Carla
2015-11-26 15:13 GMT+01:00 Nathan Goldbaum <nathan12343@gmail.com <javascript:_e(%7B%7D,'cvml','nathan12343@gmail.com');>>:
On Thu, Nov 26, 2015 at 7:19 AM, Carla Bernhardt < carla.j.bernhardt@gmail.com <javascript:_e(%7B%7D,'cvml','carla.j.bernhardt@gmail.com');>> wrote:
Dear yt users,
Thanks again for the help on this topic. To make this work with the available memory, I want to take a cut region from the data set and put it into the smoothed_covering_grid(). I know smoothed_covering_grid() uses a left_edge, but why not a right_edge? Can I cut it after it is made? I have tried cut_region(), box() and region() but none of these support smoothed_covering_grid() after application. I have a center and a width in code_length units.
Both covering_grid and smoothed_covering_grid work the same way, so below I'll only refer to covering_grid, even though this applies equally to both.
The job of covering_grid is to create a uniform resolution representation of an AMR simulation, at the same resolution of one of the AMR levels. This explains why you need to specify the AMR level you want to sample at - an AMR simulation has *multiple* spatial resolutions, one for each AMR level, so covering_grid needs a user to tell it which resolution to sample at.
The AMR level determines the linear cell spacing of the resulting uniform resolution grid. In general, for a simulation with grid dimensions on level 0 given by "domain_dimensions", and a box size in code units given by box_size, the linear resolution on AMR level "level" is (in python pseudo-code, where both box_size and domain_dimensions act like numpy arrays):
dx = box_size / (domain_dimensions * 2**level)
In addition, covering_grid needs to know where in the simulation to begin sampling. This is left_edge, the bottom-left hand corner of the region to sample over. In general left_edge needs to be carefully chosen so it falls exactly at a zone boundary at the AMR level you're sampling at. I believe that if this isn't the case, it will either floor or ceiling left_edge to the nearest zone boundary.
You might also expect to be able to to specify a right_edge, however, this would be very misleading and possibly surprising, so it is not done this way. This is because an arbitrary right_edge will most likely not happen to fall an exact integer number of zones away from the left_edge. Instead, covering_grid lets users specify the integer *dimensions* of the resulting grid. To calculate right_edge from the dims, you can do (again interpreting the arrays below as numpy arrays):
right_edge = left_edge + dims*dx
where dx is from the first equation I gave above.
This is definitely not intuitive (I was confused by it at first) but it is a sensible way of doing it once you realize the limitations imposed by the problem. Using covering_grid is "nice" because it's a "natural" way to represent AMR data: you are doing the minimum possible amount of resampling or interpolating, and are using the "real" simulation data on the AMR level you are sampling at.
I should also mention that yt also provides arbitrary_grid, which *does* take a left_edge, right_edge, and dims. On the stable branch all extant stable releases of yt, arbitrary_grid can only be used for deposited particle fields. However, in the upcoming yt 3.3. release and in the current development arbitrary_grid can be used for AMR mesh fields like ('gas', 'density'). While arbitrary_grid has an easier to understand API, it is also not as natural a representation of AMR data as covering_grid, since unless left_edge, right_edge, and the dimensions of your arbitrary_grid are carefully chosen, every zone in the result will be affected by resampling or interpolating.
Hope that explanation helped!
-Nathan
Cheers, Carla
2015-11-19 18:48 GMT+01:00 Michael Zingale < michael.zingale@stonybrook.edu <javascript:_e(%7B%7D,'cvml','michael.zingale@stonybrook.edu');>>:
If I understand this, you are saying it will be a 2048**3 uniform grid. At 8 bytes per #, this is 64 GB of memory to hold the data.
On Thu, Nov 19, 2015 at 12:45 PM, Carla Bernhardt < carla.j.bernhardt@gmail.com <javascript:_e(%7B%7D,'cvml','carla.j.bernhardt@gmail.com');>> wrote:
Nathan,
Dims*2**3 is 256*2^3 or 2048. With the data I am working with, I think this should be on the order of 1 GB. I don't think that should overload the memory... Is it possible there is something incorrectly defined?
Thanks, Carla
Am Donnerstag, 19. November 2015 schrieb Nathan Goldbaum :
What is dims*2**3? Depending on your data this might be a very large array that won't fit in memory on your machine.
You will need to find a machine that has more RAM to do this or create a covering grid that only covers a subset of the domain.
On Thursday, November 19, 2015, Carla Bernhardt < carla.j.bernhardt@gmail.com> wrote:
> Thanks again for the help. Once I implemented that, I have a > follow-up problem: Once I have this much larger grid covering, it seems > possibly too large for yt/python to handle. When I run: > > cg=ds.smoothed_covering_grid(3,[0,0,0], dims*2**3) > data=cg['density'] > > (where before I had dims*2**1), the variable data does not load and > gives a few errors <http://pastebin.com/JVhenc4z> which lead me to > believe that python or yt will not let me use that much space. Can I force > yt to let me use more space? > > Thanks, > Carla > > 2015-11-19 16:11 GMT+01:00 Nathan Goldbaum <nathan12343@gmail.com>: > >> Hi Carla, >> >> So it looks like ref_factors is only defined in the boxlib >> frontend, which is why it doesn't work for your Enzo data. Boxlib needs a >> concept of ref_factors because it allows AMR refinement jumps larger than a >> factor of two on a single level (i.e. level 2 might be four times higher >> resolution as level 1). Enzo and many other AMR codes do not allow this, >> instead only allowing a factor of two increase in linear resolution per AMR >> level. >> >> The dimensions you want to use depend on what fraction of the full >> domain you want to interpolate to uniform resolution. It looks like you >> want to create a covering grid covering your full domain, so to create a >> covering grid at the same resolution as AMR level 3 you would do: >> >> all_data_level_3 = ds.covering_grid(level=2, left_edge=[0,0.0,0.0], >> dims=ds.domain_dimensions * >> 2**3) >> >> This is because each refinement level jump in an Enzo simulation >> corresponds to a factor of two increase in spatial resolution. >> >> Hope that makes sense, >> >> Nathan >> >> On Thu, Nov 19, 2015 at 8:58 AM, Carla Bernhardt < >> carla.j.bernhardt@gmail.com> wrote: >> >>> Thank you for your quick response. However, when I used: >>> >>> ref = int(np.product(ds.ref_factors[0:3])), >>> >>> I got this error: >>> >>> AttributeError: 'EnzoDataset' object has no attribute 'ref_factors' >>> >>> Did I misunderstand your suggestion? Or do I need to import >>> something? >>> >>> Thanks, >>> >>> Carla >>> >>> 2015-11-19 15:39 GMT+01:00 Michael Zingale < >>> michael.zingale@stonybrook.edu>: >>> >>>> I've done this in the past: >>>> >>>> ref = int(np.product(ds.ref_factors[0:max_level])) >>>> >>>> >>>> >>>> # allocate for our uniformly-gridded result >>>> >>>> dims = ds.domain_dimensions*ref >>>> >>>> this will work for a more general case when the jump between >>>> levels can change as a function of level. >>>> >>>> On Thu, Nov 19, 2015 at 9:10 AM, Carla Bernhardt < >>>> carla.j.bernhardt@gmail.com> wrote: >>>> >>>>> Dear YT Users, >>>>> >>>>> To better understand covering_grid (or smoothed_covering_grid), >>>>> can someone explain what dimensions I should use when I have multiple >>>>> levels of refinement? If I have 1 level of refinement from AMR data, the >>>>> dimensions should be the same, I believe, but what if I have 2 or 3 levels >>>>> of refinement? Should the fixed resolution region then have dimensions of >>>>> dims*2^2 and dims*3^2 respectively? >>>>> >>>>> Here is one example from a tutorial >>>>> <http://yt-project.org/doc/examining/low_level_inspection.html#examining-grid-data-in-a-fixed-resolution-array> >>>>> if that helps isolate my question: >>>>> all_data_level_2 = ds.covering_grid(level=2, >>>>> left_edge=[0,0.0,0.0], >>>>> dims=ds.domain_dimensions >>>>> * 2**2) >>>>> >>>>> Thanks in advance, >>>>> Carla >>>>> >>>>> _______________________________________________ >>>>> yt-users mailing list >>>>> yt-users@lists.spacepope.org >>>>> http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org >>>>> >>>>> >>>> >>>> >>>> -- >>>> Michael Zingale >>>> Associate Professor >>>> >>>> Dept. of Physics & Astronomy • Stony Brook University • Stony >>>> Brook, NY 11794-3800 >>>> *phone*: 631-632-8225 >>>> *e-mail*: Michael.Zingale@stonybrook.edu >>>> *web*: http://www.astro.sunysb.edu/mzingale >>>> github: http://github.com/zingale >>>> >>>> >>>> _______________________________________________ >>>> yt-users mailing list >>>> yt-users@lists.spacepope.org >>>> http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org >>>> >>>> >>> >>> _______________________________________________ >>> yt-users mailing list >>> yt-users@lists.spacepope.org >>> http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org >>> >>> >> >> _______________________________________________ >> yt-users mailing list >> yt-users@lists.spacepope.org >> http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org >> >> >
yt-users mailing list yt-users@lists.spacepope.org <javascript:_e(%7B%7D,'cvml','yt-users@lists.spacepope.org');> http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
-- Michael Zingale Associate Professor
Dept. of Physics & Astronomy • Stony Brook University • Stony Brook, NY 11794-3800 *phone*: 631-632-8225 *e-mail*: Michael.Zingale@stonybrook.edu <javascript:_e(%7B%7D,'cvml','Michael.Zingale@stonybrook.edu');> *web*: http://www.astro.sunysb.edu/mzingale github: http://github.com/zingale
_______________________________________________ yt-users mailing list yt-users@lists.spacepope.org <javascript:_e(%7B%7D,'cvml','yt-users@lists.spacepope.org');> http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
_______________________________________________ yt-users mailing list yt-users@lists.spacepope.org <javascript:_e(%7B%7D,'cvml','yt-users@lists.spacepope.org');> http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
_______________________________________________ yt-users mailing list yt-users@lists.spacepope.org <javascript:_e(%7B%7D,'cvml','yt-users@lists.spacepope.org');> http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
Hi Nathan, Back in November you said "in the upcoming yt 3.3. release and in the current development arbitrary_grid can be used for AMR mesh fields like ('gas', 'density')." Is this now available? I have come back to this problem and still need to find a way to produce unigrid data from a specified region (so the left_edge and the right_edge attribute would be great) as well as with dimensions that can be processed by a computer. Thanks for your help. Best, Carla 2015-11-26 15:13 GMT+01:00 Nathan Goldbaum <nathan12343@gmail.com>:
On Thu, Nov 26, 2015 at 7:19 AM, Carla Bernhardt < carla.j.bernhardt@gmail.com> wrote:
Dear yt users,
Thanks again for the help on this topic. To make this work with the available memory, I want to take a cut region from the data set and put it into the smoothed_covering_grid(). I know smoothed_covering_grid() uses a left_edge, but why not a right_edge? Can I cut it after it is made? I have tried cut_region(), box() and region() but none of these support smoothed_covering_grid() after application. I have a center and a width in code_length units.
Both covering_grid and smoothed_covering_grid work the same way, so below I'll only refer to covering_grid, even though this applies equally to both.
The job of covering_grid is to create a uniform resolution representation of an AMR simulation, at the same resolution of one of the AMR levels. This explains why you need to specify the AMR level you want to sample at - an AMR simulation has *multiple* spatial resolutions, one for each AMR level, so covering_grid needs a user to tell it which resolution to sample at.
The AMR level determines the linear cell spacing of the resulting uniform resolution grid. In general, for a simulation with grid dimensions on level 0 given by "domain_dimensions", and a box size in code units given by box_size, the linear resolution on AMR level "level" is (in python pseudo-code, where both box_size and domain_dimensions act like numpy arrays):
dx = box_size / (domain_dimensions * 2**level)
In addition, covering_grid needs to know where in the simulation to begin sampling. This is left_edge, the bottom-left hand corner of the region to sample over. In general left_edge needs to be carefully chosen so it falls exactly at a zone boundary at the AMR level you're sampling at. I believe that if this isn't the case, it will either floor or ceiling left_edge to the nearest zone boundary.
You might also expect to be able to to specify a right_edge, however, this would be very misleading and possibly surprising, so it is not done this way. This is because an arbitrary right_edge will most likely not happen to fall an exact integer number of zones away from the left_edge. Instead, covering_grid lets users specify the integer *dimensions* of the resulting grid. To calculate right_edge from the dims, you can do (again interpreting the arrays below as numpy arrays):
right_edge = left_edge + dims*dx
where dx is from the first equation I gave above.
This is definitely not intuitive (I was confused by it at first) but it is a sensible way of doing it once you realize the limitations imposed by the problem. Using covering_grid is "nice" because it's a "natural" way to represent AMR data: you are doing the minimum possible amount of resampling or interpolating, and are using the "real" simulation data on the AMR level you are sampling at.
I should also mention that yt also provides arbitrary_grid, which *does* take a left_edge, right_edge, and dims. On the stable branch all extant stable releases of yt, arbitrary_grid can only be used for deposited particle fields. However, in the upcoming yt 3.3. release and in the current development arbitrary_grid can be used for AMR mesh fields like ('gas', 'density'). While arbitrary_grid has an easier to understand API, it is also not as natural a representation of AMR data as covering_grid, since unless left_edge, right_edge, and the dimensions of your arbitrary_grid are carefully chosen, every zone in the result will be affected by resampling or interpolating.
Hope that explanation helped!
-Nathan
Cheers, Carla
2015-11-19 18:48 GMT+01:00 Michael Zingale < michael.zingale@stonybrook.edu>:
If I understand this, you are saying it will be a 2048**3 uniform grid. At 8 bytes per #, this is 64 GB of memory to hold the data.
On Thu, Nov 19, 2015 at 12:45 PM, Carla Bernhardt < carla.j.bernhardt@gmail.com> wrote:
Nathan,
Dims*2**3 is 256*2^3 or 2048. With the data I am working with, I think this should be on the order of 1 GB. I don't think that should overload the memory... Is it possible there is something incorrectly defined?
Thanks, Carla
Am Donnerstag, 19. November 2015 schrieb Nathan Goldbaum :
What is dims*2**3? Depending on your data this might be a very large array that won't fit in memory on your machine.
You will need to find a machine that has more RAM to do this or create a covering grid that only covers a subset of the domain.
On Thursday, November 19, 2015, Carla Bernhardt < carla.j.bernhardt@gmail.com> wrote:
Thanks again for the help. Once I implemented that, I have a follow-up problem: Once I have this much larger grid covering, it seems possibly too large for yt/python to handle. When I run:
cg=ds.smoothed_covering_grid(3,[0,0,0], dims*2**3) data=cg['density']
(where before I had dims*2**1), the variable data does not load and gives a few errors <http://pastebin.com/JVhenc4z> which lead me to believe that python or yt will not let me use that much space. Can I force yt to let me use more space?
Thanks, Carla
2015-11-19 16:11 GMT+01:00 Nathan Goldbaum <nathan12343@gmail.com>:
> Hi Carla, > > So it looks like ref_factors is only defined in the boxlib frontend, > which is why it doesn't work for your Enzo data. Boxlib needs a concept of > ref_factors because it allows AMR refinement jumps larger than a factor of > two on a single level (i.e. level 2 might be four times higher resolution > as level 1). Enzo and many other AMR codes do not allow this, instead only > allowing a factor of two increase in linear resolution per AMR level. > > The dimensions you want to use depend on what fraction of the full > domain you want to interpolate to uniform resolution. It looks like you > want to create a covering grid covering your full domain, so to create a > covering grid at the same resolution as AMR level 3 you would do: > > all_data_level_3 = ds.covering_grid(level=2, left_edge=[0,0.0,0.0], > dims=ds.domain_dimensions * 2**3) > > This is because each refinement level jump in an Enzo simulation > corresponds to a factor of two increase in spatial resolution. > > Hope that makes sense, > > Nathan > > On Thu, Nov 19, 2015 at 8:58 AM, Carla Bernhardt < > carla.j.bernhardt@gmail.com> wrote: > >> Thank you for your quick response. However, when I used: >> >> ref = int(np.product(ds.ref_factors[0:3])), >> >> I got this error: >> >> AttributeError: 'EnzoDataset' object has no attribute 'ref_factors' >> >> Did I misunderstand your suggestion? Or do I need to import >> something? >> >> Thanks, >> >> Carla >> >> 2015-11-19 15:39 GMT+01:00 Michael Zingale < >> michael.zingale@stonybrook.edu>: >> >>> I've done this in the past: >>> >>> ref = int(np.product(ds.ref_factors[0:max_level])) >>> >>> >>> >>> # allocate for our uniformly-gridded result >>> >>> dims = ds.domain_dimensions*ref >>> >>> this will work for a more general case when the jump between >>> levels can change as a function of level. >>> >>> On Thu, Nov 19, 2015 at 9:10 AM, Carla Bernhardt < >>> carla.j.bernhardt@gmail.com> wrote: >>> >>>> Dear YT Users, >>>> >>>> To better understand covering_grid (or smoothed_covering_grid), >>>> can someone explain what dimensions I should use when I have multiple >>>> levels of refinement? If I have 1 level of refinement from AMR data, the >>>> dimensions should be the same, I believe, but what if I have 2 or 3 levels >>>> of refinement? Should the fixed resolution region then have dimensions of >>>> dims*2^2 and dims*3^2 respectively? >>>> >>>> Here is one example from a tutorial >>>> <http://yt-project.org/doc/examining/low_level_inspection.html#examining-grid-data-in-a-fixed-resolution-array> >>>> if that helps isolate my question: >>>> all_data_level_2 = ds.covering_grid(level=2, >>>> left_edge=[0,0.0,0.0], >>>> dims=ds.domain_dimensions * >>>> 2**2) >>>> >>>> Thanks in advance, >>>> Carla >>>> >>>> _______________________________________________ >>>> yt-users mailing list >>>> yt-users@lists.spacepope.org >>>> http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org >>>> >>>> >>> >>> >>> -- >>> Michael Zingale >>> Associate Professor >>> >>> Dept. of Physics & Astronomy • Stony Brook University • Stony >>> Brook, NY 11794-3800 >>> *phone*: 631-632-8225 >>> *e-mail*: Michael.Zingale@stonybrook.edu >>> *web*: http://www.astro.sunysb.edu/mzingale >>> github: http://github.com/zingale >>> >>> >>> _______________________________________________ >>> yt-users mailing list >>> yt-users@lists.spacepope.org >>> http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org >>> >>> >> >> _______________________________________________ >> yt-users mailing list >> yt-users@lists.spacepope.org >> http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org >> >> > > _______________________________________________ > yt-users mailing list > yt-users@lists.spacepope.org > http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org > >
yt-users mailing list yt-users@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
-- Michael Zingale Associate Professor
Dept. of Physics & Astronomy • Stony Brook University • Stony Brook, NY 11794-3800 *phone*: 631-632-8225 *e-mail*: Michael.Zingale@stonybrook.edu *web*: http://www.astro.sunysb.edu/mzingale github: http://github.com/zingale
_______________________________________________ yt-users mailing list yt-users@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
_______________________________________________ yt-users mailing list yt-users@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
_______________________________________________ yt-users mailing list yt-users@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
On Tue, Apr 5, 2016 at 8:49 AM, Carla Bernhardt <carla.j.bernhardt@gmail.com
wrote:
Hi Nathan,
Back in November you said "in the upcoming yt 3.3. release and in the current development arbitrary_grid can be used for AMR mesh fields like ('gas', 'density')." Is this now available? I have come back to this problem and still need to find a way to produce unigrid data from a specified region (so the left_edge and the right_edge attribute would be great) as well as with dimensions that can be processed by a computer.
yt 3.3 hasn't been released yet, but you can try installing from the development branch. http://yt-project.org/doc/installing.html#installing-yt-using-pip-or-from-so... specifically the steps beginning with "hg clone" at the page I linked to ^
Thanks for your help.
Best, Carla
2015-11-26 15:13 GMT+01:00 Nathan Goldbaum <nathan12343@gmail.com>:
On Thu, Nov 26, 2015 at 7:19 AM, Carla Bernhardt < carla.j.bernhardt@gmail.com> wrote:
Dear yt users,
Thanks again for the help on this topic. To make this work with the available memory, I want to take a cut region from the data set and put it into the smoothed_covering_grid(). I know smoothed_covering_grid() uses a left_edge, but why not a right_edge? Can I cut it after it is made? I have tried cut_region(), box() and region() but none of these support smoothed_covering_grid() after application. I have a center and a width in code_length units.
Both covering_grid and smoothed_covering_grid work the same way, so below I'll only refer to covering_grid, even though this applies equally to both.
The job of covering_grid is to create a uniform resolution representation of an AMR simulation, at the same resolution of one of the AMR levels. This explains why you need to specify the AMR level you want to sample at - an AMR simulation has *multiple* spatial resolutions, one for each AMR level, so covering_grid needs a user to tell it which resolution to sample at.
The AMR level determines the linear cell spacing of the resulting uniform resolution grid. In general, for a simulation with grid dimensions on level 0 given by "domain_dimensions", and a box size in code units given by box_size, the linear resolution on AMR level "level" is (in python pseudo-code, where both box_size and domain_dimensions act like numpy arrays):
dx = box_size / (domain_dimensions * 2**level)
In addition, covering_grid needs to know where in the simulation to begin sampling. This is left_edge, the bottom-left hand corner of the region to sample over. In general left_edge needs to be carefully chosen so it falls exactly at a zone boundary at the AMR level you're sampling at. I believe that if this isn't the case, it will either floor or ceiling left_edge to the nearest zone boundary.
You might also expect to be able to to specify a right_edge, however, this would be very misleading and possibly surprising, so it is not done this way. This is because an arbitrary right_edge will most likely not happen to fall an exact integer number of zones away from the left_edge. Instead, covering_grid lets users specify the integer *dimensions* of the resulting grid. To calculate right_edge from the dims, you can do (again interpreting the arrays below as numpy arrays):
right_edge = left_edge + dims*dx
where dx is from the first equation I gave above.
This is definitely not intuitive (I was confused by it at first) but it is a sensible way of doing it once you realize the limitations imposed by the problem. Using covering_grid is "nice" because it's a "natural" way to represent AMR data: you are doing the minimum possible amount of resampling or interpolating, and are using the "real" simulation data on the AMR level you are sampling at.
I should also mention that yt also provides arbitrary_grid, which *does* take a left_edge, right_edge, and dims. On the stable branch all extant stable releases of yt, arbitrary_grid can only be used for deposited particle fields. However, in the upcoming yt 3.3. release and in the current development arbitrary_grid can be used for AMR mesh fields like ('gas', 'density'). While arbitrary_grid has an easier to understand API, it is also not as natural a representation of AMR data as covering_grid, since unless left_edge, right_edge, and the dimensions of your arbitrary_grid are carefully chosen, every zone in the result will be affected by resampling or interpolating.
Hope that explanation helped!
-Nathan
Cheers, Carla
2015-11-19 18:48 GMT+01:00 Michael Zingale < michael.zingale@stonybrook.edu>:
If I understand this, you are saying it will be a 2048**3 uniform grid. At 8 bytes per #, this is 64 GB of memory to hold the data.
On Thu, Nov 19, 2015 at 12:45 PM, Carla Bernhardt < carla.j.bernhardt@gmail.com> wrote:
Nathan,
Dims*2**3 is 256*2^3 or 2048. With the data I am working with, I think this should be on the order of 1 GB. I don't think that should overload the memory... Is it possible there is something incorrectly defined?
Thanks, Carla
Am Donnerstag, 19. November 2015 schrieb Nathan Goldbaum :
What is dims*2**3? Depending on your data this might be a very large array that won't fit in memory on your machine.
You will need to find a machine that has more RAM to do this or create a covering grid that only covers a subset of the domain.
On Thursday, November 19, 2015, Carla Bernhardt < carla.j.bernhardt@gmail.com> wrote:
> Thanks again for the help. Once I implemented that, I have a > follow-up problem: Once I have this much larger grid covering, it seems > possibly too large for yt/python to handle. When I run: > > cg=ds.smoothed_covering_grid(3,[0,0,0], dims*2**3) > data=cg['density'] > > (where before I had dims*2**1), the variable data does not load and > gives a few errors <http://pastebin.com/JVhenc4z> which lead me to > believe that python or yt will not let me use that much space. Can I force > yt to let me use more space? > > Thanks, > Carla > > 2015-11-19 16:11 GMT+01:00 Nathan Goldbaum <nathan12343@gmail.com>: > >> Hi Carla, >> >> So it looks like ref_factors is only defined in the boxlib >> frontend, which is why it doesn't work for your Enzo data. Boxlib needs a >> concept of ref_factors because it allows AMR refinement jumps larger than a >> factor of two on a single level (i.e. level 2 might be four times higher >> resolution as level 1). Enzo and many other AMR codes do not allow this, >> instead only allowing a factor of two increase in linear resolution per AMR >> level. >> >> The dimensions you want to use depend on what fraction of the full >> domain you want to interpolate to uniform resolution. It looks like you >> want to create a covering grid covering your full domain, so to create a >> covering grid at the same resolution as AMR level 3 you would do: >> >> all_data_level_3 = ds.covering_grid(level=2, left_edge=[0,0.0,0.0], >> dims=ds.domain_dimensions * >> 2**3) >> >> This is because each refinement level jump in an Enzo simulation >> corresponds to a factor of two increase in spatial resolution. >> >> Hope that makes sense, >> >> Nathan >> >> On Thu, Nov 19, 2015 at 8:58 AM, Carla Bernhardt < >> carla.j.bernhardt@gmail.com> wrote: >> >>> Thank you for your quick response. However, when I used: >>> >>> ref = int(np.product(ds.ref_factors[0:3])), >>> >>> I got this error: >>> >>> AttributeError: 'EnzoDataset' object has no attribute 'ref_factors' >>> >>> Did I misunderstand your suggestion? Or do I need to import >>> something? >>> >>> Thanks, >>> >>> Carla >>> >>> 2015-11-19 15:39 GMT+01:00 Michael Zingale < >>> michael.zingale@stonybrook.edu>: >>> >>>> I've done this in the past: >>>> >>>> ref = int(np.product(ds.ref_factors[0:max_level])) >>>> >>>> >>>> >>>> # allocate for our uniformly-gridded result >>>> >>>> dims = ds.domain_dimensions*ref >>>> >>>> this will work for a more general case when the jump between >>>> levels can change as a function of level. >>>> >>>> On Thu, Nov 19, 2015 at 9:10 AM, Carla Bernhardt < >>>> carla.j.bernhardt@gmail.com> wrote: >>>> >>>>> Dear YT Users, >>>>> >>>>> To better understand covering_grid (or smoothed_covering_grid), >>>>> can someone explain what dimensions I should use when I have multiple >>>>> levels of refinement? If I have 1 level of refinement from AMR data, the >>>>> dimensions should be the same, I believe, but what if I have 2 or 3 levels >>>>> of refinement? Should the fixed resolution region then have dimensions of >>>>> dims*2^2 and dims*3^2 respectively? >>>>> >>>>> Here is one example from a tutorial >>>>> <http://yt-project.org/doc/examining/low_level_inspection.html#examining-grid-data-in-a-fixed-resolution-array> >>>>> if that helps isolate my question: >>>>> all_data_level_2 = ds.covering_grid(level=2, >>>>> left_edge=[0,0.0,0.0], >>>>> dims=ds.domain_dimensions >>>>> * 2**2) >>>>> >>>>> Thanks in advance, >>>>> Carla >>>>> >>>>> _______________________________________________ >>>>> yt-users mailing list >>>>> yt-users@lists.spacepope.org >>>>> http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org >>>>> >>>>> >>>> >>>> >>>> -- >>>> Michael Zingale >>>> Associate Professor >>>> >>>> Dept. of Physics & Astronomy • Stony Brook University • Stony >>>> Brook, NY 11794-3800 >>>> *phone*: 631-632-8225 >>>> *e-mail*: Michael.Zingale@stonybrook.edu >>>> *web*: http://www.astro.sunysb.edu/mzingale >>>> github: http://github.com/zingale >>>> >>>> >>>> _______________________________________________ >>>> yt-users mailing list >>>> yt-users@lists.spacepope.org >>>> http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org >>>> >>>> >>> >>> _______________________________________________ >>> yt-users mailing list >>> yt-users@lists.spacepope.org >>> http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org >>> >>> >> >> _______________________________________________ >> yt-users mailing list >> yt-users@lists.spacepope.org >> http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org >> >> >
yt-users mailing list yt-users@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
-- Michael Zingale Associate Professor
Dept. of Physics & Astronomy • Stony Brook University • Stony Brook, NY 11794-3800 *phone*: 631-632-8225 *e-mail*: Michael.Zingale@stonybrook.edu *web*: http://www.astro.sunysb.edu/mzingale github: http://github.com/zingale
_______________________________________________ yt-users mailing list yt-users@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
_______________________________________________ yt-users mailing list yt-users@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
_______________________________________________ yt-users mailing list yt-users@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
_______________________________________________ yt-users mailing list yt-users@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
participants (3)
-
Carla Bernhardt
-
Michael Zingale
-
Nathan Goldbaum