Nathan,

Thank you for your explanation. I had looked at arbitrary_grid, but I want to confirm that the data is the same as my AMR data and I thought this would present a problem. You mention that I "might also expect to be able to specify a right_edge", but where is this right_edge implemented? I tried just putting in right_edge=left_edge + dims*dx in the smoothed_covering_grid command, but it throws an error that there is no right_edge (which I knew from reading the documentation, I was just hoping I would get lucky).

Thanks,
Carla

2015-11-26 15:13 GMT+01:00 Nathan Goldbaum <nathan12343@gmail.com>:


On Thu, Nov 26, 2015 at 7:19 AM, Carla Bernhardt <carla.j.bernhardt@gmail.com> wrote:
Dear yt users,

Thanks again for the help on this topic. To make this work with the available memory, I want to take a cut region from the data set and put it into the smoothed_covering_grid(). I know smoothed_covering_grid() uses a left_edge, but why not a right_edge? Can I cut it after it is made? I have tried cut_region(), box() and region() but none of these support smoothed_covering_grid() after application. I have a center and a width in code_length units.

Both covering_grid and smoothed_covering_grid work the same way, so below I'll only refer to covering_grid, even though this applies equally to both.

The job of covering_grid is to create a uniform resolution representation of an AMR simulation, at the same resolution of one of the AMR levels. This explains why you need to specify the AMR level you want to sample at - an AMR simulation has *multiple* spatial resolutions, one for each AMR level, so covering_grid needs a user to tell it which resolution to sample at.

The AMR level determines the linear cell spacing of the resulting uniform resolution grid. In general, for a simulation with grid dimensions on level 0 given by "domain_dimensions", and a box size in code units given by box_size, the linear resolution on AMR level "level" is (in python pseudo-code, where both box_size and domain_dimensions act like numpy arrays):

dx = box_size / (domain_dimensions * 2**level)

In addition, covering_grid needs to know where in the simulation to begin sampling. This is left_edge, the bottom-left hand corner of the region to sample over. In general left_edge needs to be carefully chosen so it falls exactly at a zone boundary at the AMR level you're sampling at.  I believe that if this isn't the case, it will either floor or ceiling left_edge to the nearest zone boundary.

You might also expect to be able to to specify a right_edge, however, this would be very misleading and possibly surprising, so it is not done this way. This is because an arbitrary right_edge will most likely not happen to fall an exact integer number of zones away from the left_edge. Instead, covering_grid lets users specify the integer *dimensions* of the resulting grid. To calculate right_edge from the dims, you can do (again interpreting the arrays below as numpy arrays):

right_edge = left_edge + dims*dx

where dx is from the first equation I gave above.

This is definitely not intuitive (I was confused by it at first) but it is a sensible way of doing it once you realize the limitations imposed by the problem. Using covering_grid is "nice" because it's a "natural" way to represent AMR data: you are doing the minimum possible amount of resampling or interpolating, and are using the "real" simulation data on the AMR level you are sampling at.

I should also mention that yt also provides arbitrary_grid, which *does* take a left_edge, right_edge, and dims. On the stable branch all extant stable releases of yt, arbitrary_grid can only be used for deposited particle fields. However, in the upcoming yt 3.3. release and in the current development arbitrary_grid can be used for AMR mesh fields like ('gas', 'density'). While arbitrary_grid has an easier to understand API, it is also not as natural a representation of AMR data as covering_grid, since unless left_edge, right_edge, and the dimensions of your arbitrary_grid are carefully chosen, every zone in the result will be affected by resampling or interpolating.

Hope that explanation helped!

-Nathan
 

Cheers,
Carla

2015-11-19 18:48 GMT+01:00 Michael Zingale <michael.zingale@stonybrook.edu>:
If I understand this, you are saying it will be a 2048**3 uniform grid.  At 8 bytes per #, this is 64 GB of memory to hold the data.

On Thu, Nov 19, 2015 at 12:45 PM, Carla Bernhardt <carla.j.bernhardt@gmail.com> wrote:
Nathan,

Dims*2**3 is 256*2^3 or 2048. With the data I am working with, I think this should be on the order of 1 GB. I don't think that should overload the memory... Is it possible there is something incorrectly defined?

Thanks,
Carla


Am Donnerstag, 19. November 2015 schrieb Nathan Goldbaum :
What is dims*2**3? Depending on your data this might be a very large array that won't fit in memory on your machine.

You will need to find a machine that has more RAM to do this or create a covering grid that only covers a subset of the domain.

On Thursday, November 19, 2015, Carla Bernhardt <carla.j.bernhardt@gmail.com> wrote:
Thanks again for the help. Once I implemented that, I have a follow-up problem: Once I have this much larger grid covering, it seems possibly too large for yt/python to handle. When I run:

cg=ds.smoothed_covering_grid(3,[0,0,0], dims*2**3)
data=cg['density']

(where before I had dims*2**1), the variable data does not load and gives a few errors which lead me to believe that python or yt will not let me use that much space. Can I force yt to let me use more space?

Thanks,
Carla

2015-11-19 16:11 GMT+01:00 Nathan Goldbaum <nathan12343@gmail.com>:
Hi Carla,

So it looks like ref_factors is only defined in the boxlib frontend, which is why it doesn't work for your Enzo data. Boxlib needs a concept of ref_factors because it allows AMR refinement jumps larger than a factor of two on a single level (i.e. level 2 might be four times higher resolution as level 1). Enzo and many other AMR codes do not allow this, instead only allowing a factor of two increase in linear resolution per AMR level.

The dimensions you want to use depend on what fraction of the full domain you want to interpolate to uniform resolution. It looks like you want to create a covering grid covering your full domain, so to create a covering grid at the same resolution as AMR level 3 you would do:

all_data_level_3 = ds.covering_grid(level=2, left_edge=[0,0.0,0.0],
                                    dims=ds.domain_dimensions * 2**3)

This is because each refinement level jump in an Enzo simulation corresponds to a factor of two increase in spatial resolution.

Hope that makes sense,

Nathan

On Thu, Nov 19, 2015 at 8:58 AM, Carla Bernhardt <carla.j.bernhardt@gmail.com> wrote:
Thank you for your quick response. However, when I used:

ref = int(np.product(ds.ref_factors[0:3])),

I got this error:

AttributeError: 'EnzoDataset' object has no attribute 'ref_factors'

Did I misunderstand your suggestion? Or do I need to import something?

Thanks,

Carla


2015-11-19 15:39 GMT+01:00 Michael Zingale <michael.zingale@stonybrook.edu>:
I've done this in the past:

ref = int(np.product(ds.ref_factors[0:max_level]))                          
                                                                                
# allocate for our uniformly-gridded result                                 
dims = ds.domain_dimensions*ref       

this will work for a more general case when the jump between levels can change as a function of level.

On Thu, Nov 19, 2015 at 9:10 AM, Carla Bernhardt <carla.j.bernhardt@gmail.com> wrote:
Dear YT Users,

To better understand covering_grid (or smoothed_covering_grid), can someone explain what dimensions I should use when I have multiple levels of refinement? If I have 1 level of refinement from AMR data, the dimensions should be the same, I believe, but what if I have 2 or 3 levels of refinement? Should the fixed resolution region then have dimensions of dims*2^2 and dims*3^2 respectively?

Here is one example from a tutorial if that helps isolate my question:
all_data_level_2 = ds.covering_grid(level=2, left_edge=[0,0.0,0.0],
                                      dims=ds.domain_dimensions * 2**2)

Thanks in advance,
Carla

_______________________________________________
yt-users mailing list
yt-users@lists.spacepope.org
http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org




--
Michael Zingale
Associate Professor

Dept. of Physics & Astronomy • Stony Brook University • Stony Brook, NY 11794-3800
phone:  631-632-8225


_______________________________________________
yt-users mailing list
yt-users@lists.spacepope.org
http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org



_______________________________________________
yt-users mailing list
yt-users@lists.spacepope.org
http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org



_______________________________________________
yt-users mailing list
yt-users@lists.spacepope.org
http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org



_______________________________________________
yt-users mailing list
yt-users@lists.spacepope.org
http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org




--
Michael Zingale
Associate Professor

Dept. of Physics & Astronomy • Stony Brook University • Stony Brook, NY 11794-3800
phone:  631-632-8225


_______________________________________________
yt-users mailing list
yt-users@lists.spacepope.org
http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org



_______________________________________________
yt-users mailing list
yt-users@lists.spacepope.org
http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org



_______________________________________________
yt-users mailing list
yt-users@lists.spacepope.org
http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org