Hi, I'm working with Nick Gnedin on looking at the density properties of my galaxy on different AMR levels. For this, we'd like to grab all the cells on one AMR level, divide them so that they have a fixed cell size and output. Is there a quick way of doing this in yt? I *think* the answer will be no, since (for example) covering_grid will grab the best data available to make the mesh, not just the data from a single level. Is that right? We specifically need to avoid interpolation. If so, probably it's easiest just to grab the cells on one level via: gridnum = [g for g in pf.h.grids if g.Level == l] then divide them into the right size. Is there a way of passing a grid to covering_grid to make it a particular size? i.e. something like: for g in gridnum: grid_fixedcellsize = g.covering_grid(......) Elizabeth
Hi Elizabeth,
On Wed, Jan 16, 2013 at 9:57 AM, Elizabeth Tasker
Hi,
I'm working with Nick Gnedin on looking at the density properties of my galaxy on different AMR levels.
For this, we'd like to grab all the cells on one AMR level, divide them so that they have a fixed cell size and output.
I'm not entirely sure I understand this process... :(
Is there a quick way of doing this in yt?
I *think* the answer will be no, since (for example) covering_grid will grab the best data available to make the mesh, not just the data from a single level. Is that right? We specifically need to avoid interpolation.
If so, probably it's easiest just to grab the cells on one level via:
gridnum = [g for g in pf.h.grids if g.Level == l]
You can also do this with pf.h.select_grids( level )
then divide them into the right size. Is there a way of passing a grid to covering_grid to make it a particular size? i.e. something like:
for g in gridnum: grid_fixedcellsize = g.covering_grid(......)
Covering grids go up to a given level, specified in the constructor, which might accomplish what you're looking for; i.e., you specify a left edge, a level (which fixes the dx) and a dimensionality (which fixes the right_edge). What you are describing here is similar to what's done if you do: g.retrieve_ghost_zones( N, [field_list], smoothed = False) You can also explore grid_collection, which accepts any list of grids and then returns a data object. If you describe a bit more exactly the format you want to get out, perhaps I can give a better example? I think what you're describing is probably doable, I'm just not sure precisely what it will look like so I can't quite envision it. -Matt
Elizabeth
_______________________________________________ yt-dev mailing list yt-dev@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
Hi Matt, Thanks for your reply!
For this, we'd like to grab all the cells on one AMR level, divide them so that they have a fixed cell size and output.
I'm not entirely sure I understand this process... :(
So, if I have a root grid with cells of size 64 pc and 2 levels of AMR, I'd like to gather up all the level 0, 64pc cells and chop them up to make them 16 pc. Then take all the level 1, 32 pc cells and chop them to make them 16 pc. Then dump to a file of any type :)
Is there a quick way of doing this in yt?
I *think* the answer will be no, since (for example) covering_grid will grab the best data available to make the mesh, not just the data from a single level. Is that right? We specifically need to avoid interpolation.
If so, probably it's easiest just to grab the cells on one level via:
gridnum = [g for g in pf.h.grids if g.Level == l]
You can also do this with pf.h.select_grids( level )
Great, that's neater.
then divide them into the right size. Is there a way of passing a grid to covering_grid to make it a particular size? i.e. something like:
for g in gridnum: grid_fixedcellsize = g.covering_grid(......)
Covering grids go up to a given level, specified in the constructor, which might accomplish what you're looking for; i.e., you specify a left edge, a level (which fixes the dx) and a dimensionality (which fixes the right_edge). What you are describing here is similar to what's done if you do:
g.retrieve_ghost_zones( N, [field_list], smoothed = False)
You can also explore grid_collection, which accepts any list of grids and then returns a data object.
If you describe a bit more exactly the format you want to get out, perhaps I can give a better example? I think what you're describing is probably doable, I'm just not sure precisely what it will look like so I can't quite envision it.
The format isn't really important. I was just looking for an easy way to take grids which have cell size 64 pc and produce grids with cell size 16 pc (or whatever size I want) **just** from those grids (i.e. not by grabbing the lower level grids). Does that make more sense? Elizabeth
Hi Elizabeth The following is probably not exactly what you're looking for, since it operates on fields as opposed to grids, but this would be one way forward: grids = pf.h.select_grids(0) for g in grids: rho = g['Density'] for dim in [0,1,2]: rho = np.repeat(rho,4,axis=dim) ## do stuff with expanded rho array Cheers, Mike On Wed, Jan 16, 2013 at 3:28 PM, Elizabeth Tasker < tasker@astro1.sci.hokudai.ac.jp> wrote:
Hi Matt,
Thanks for your reply!
For this, we'd like to grab all the cells on one AMR level, divide them
so that they have a fixed cell size and output.
I'm not entirely sure I understand this process... :(
So, if I have a root grid with cells of size 64 pc and 2 levels of AMR, I'd like to gather up all the level 0, 64pc cells and chop them up to make them 16 pc. Then take all the level 1, 32 pc cells and chop them to make them 16 pc.
Then dump to a file of any type :)
Is there a quick way of doing this in yt?
I *think* the answer will be no, since (for example) covering_grid will
grab the best data available to make the mesh, not just the data from a single level. Is that right? We specifically need to avoid interpolation.
If so, probably it's easiest just to grab the cells on one level via:
gridnum = [g for g in pf.h.grids if g.Level == l]
You can also do this with pf.h.select_grids( level )
Great, that's neater.
then divide them into the right size. Is there a way of passing a grid
to covering_grid to make it a particular size? i.e. something like:
for g in gridnum: grid_fixedcellsize = g.covering_grid(......)
Covering grids go up to a given level, specified in the constructor, which might accomplish what you're looking for; i.e., you specify a left edge, a level (which fixes the dx) and a dimensionality (which fixes the right_edge). What you are describing here is similar to what's done if you do:
g.retrieve_ghost_zones( N, [field_list], smoothed = False)
You can also explore grid_collection, which accepts any list of grids and then returns a data object.
If you describe a bit more exactly the format you want to get out, perhaps I can give a better example? I think what you're describing is probably doable, I'm just not sure precisely what it will look like so I can't quite envision it.
The format isn't really important. I was just looking for an easy way to take grids which have cell size 64 pc and produce grids with cell size 16 pc (or whatever size I want) **just** from those grids (i.e. not by grabbing the lower level grids).
Does that make more sense?
Elizabeth _______________________________________________ yt-dev mailing list yt-dev@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
-- ********************************************************************* * * * Dr. Michael Kuhlen Theoretical Astrophysics Center * * email: mqk@astro.berkeley.edu UC Berkeley * * cell phone: (831) 588-1468 B-116 Hearst Field Annex # 3411 * * skype username: mikekuhlen Berkeley, CA 94720 * * * *********************************************************************
Hi Elizabeth,
(Mike's solution could also work!)
On Wed, Jan 16, 2013 at 6:28 PM, Elizabeth Tasker
Hi Matt,
Thanks for your reply!
For this, we'd like to grab all the cells on one AMR level, divide them so that they have a fixed cell size and output.
I'm not entirely sure I understand this process... :(
So, if I have a root grid with cells of size 64 pc and 2 levels of AMR, I'd like to gather up all the level 0, 64pc cells and chop them up to make them 16 pc. Then take all the level 1, 32 pc cells and chop them to make them 16 pc.
Then dump to a file of any type :)
Okay, sorry for being dense. But basically what I'm getting is that you want the functionality of covering grids, but you want to be able to restrict the grids from going above some level L, right? And for the Level 1 case you mention above, do you want contributions from Level 0? (That's not so hard to do.) To do this with a covering grid you could pre-allocate: Li = 0 Lo = 3 cg = pf.h.covering_grid(Li, [0, 0, 0], pf.domain_dimensions * pf.refine_by**Li) output = np.empty(pf.domain_dimensions * pf.refine_by**Lo) nd = pf.refine_by**(Lo-li) for i in range(nd): for j in range(nd): for k in range(nd): output[i::nd,j::nd,k::nd] = cg[field] (I think, but you might want to check that. There may also be fancier/easier ways.) You could also do this if we allowed specification of a source for the covering grid, but I think that might require some subtle changes to how it masks child zones.
Is there a quick way of doing this in yt?
I *think* the answer will be no, since (for example) covering_grid will grab the best data available to make the mesh, not just the data from a single level. Is that right? We specifically need to avoid interpolation.
If so, probably it's easiest just to grab the cells on one level via:
gridnum = [g for g in pf.h.grids if g.Level == l]
You can also do this with pf.h.select_grids( level )
Great, that's neater.
then divide them into the right size. Is there a way of passing a grid to covering_grid to make it a particular size? i.e. something like:
for g in gridnum: grid_fixedcellsize = g.covering_grid(......)
Covering grids go up to a given level, specified in the constructor, which might accomplish what you're looking for; i.e., you specify a left edge, a level (which fixes the dx) and a dimensionality (which fixes the right_edge). What you are describing here is similar to what's done if you do:
g.retrieve_ghost_zones( N, [field_list], smoothed = False)
You can also explore grid_collection, which accepts any list of grids and then returns a data object.
If you describe a bit more exactly the format you want to get out, perhaps I can give a better example? I think what you're describing is probably doable, I'm just not sure precisely what it will look like so I can't quite envision it.
The format isn't really important. I was just looking for an easy way to take grids which have cell size 64 pc and produce grids with cell size 16 pc (or whatever size I want) **just** from those grids (i.e. not by grabbing the lower level grids).
Does that make more sense?
Yes, it does. And actually, I think Cameron did this for a recent paper, so he may have additional suggestions. Let me know what you think of the solution I posted above? -Matt
Elizabeth _______________________________________________ yt-dev mailing list yt-dev@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
Hi Mike and Matt, Thank you! Both those solutions look great. I like Mike's a lot for its transparency, but possibly Matt's might end up making slightly neater code. One question, Matt:
Okay, sorry for being dense. But basically what I'm getting is that you want the functionality of covering grids, but you want to be able to restrict the grids from going above some level L, right?
Or below. I just want cells on one level only.
And for the Level 1 case you mention above, do you want contributions from Level 0? (That's not so hard to do.)
No. So I want an array that contains only cells from level 1, with no contributions from 0 or 2.
To do this with a covering grid you could pre-allocate:
Li = 0 Lo = 3
cg = pf.h.covering_grid(Li, [0, 0, 0], pf.domain_dimensions * pf.refine_by**Li) output = np.empty(pf.domain_dimensions * pf.refine_by**Lo) nd = pf.refine_by**(Lo-li) for i in range(nd): for j in range(nd): for k in range(nd): output[i::nd,j::nd,k::nd] = cg[field]
Ah ok! So this is the same as Mike's method in that a covering_grid is created at the resolution of the cell size on the specified level, and then the array values are multiplied to give the correct number for a higher resolution? (Which works fine for the basic quantities; density, energy and velocity but won't for cell mass -- which is totally fine). My only question is: If I do this for level (e.g.) 2 and level 2 doesn't cover the whole domain -- what does the covering grid do when there are no level 2 cells over a region of space? Thanks again! Elizabeth
On Wed, Jan 16, 2013 at 9:52 PM, Elizabeth Tasker
Hi Mike and Matt,
Thank you! Both those solutions look great.
I like Mike's a lot for its transparency, but possibly Matt's might end up making slightly neater code.
One question, Matt:
Okay, sorry for being dense. But basically what I'm getting is that you want the functionality of covering grids, but you want to be able to restrict the grids from going above some level L, right?
Or below. I just want cells on one level only.
Okay, the grid collection might work, but I still think the covering grid is the best, with the masking I describe below. gc = pf.h.grid_collection([0.5, 0.5, 0.5], pf.h.select_grids(L)) gc["Density"] gc["dx"] gc["x"] etc. You'll have to manually manipulate the dx stuff, cell mass, etc.
And for the Level 1 case you mention above, do you want contributions from Level 0? (That's not so hard to do.)
No. So I want an array that contains only cells from level 1, with no contributions from 0 or 2.
Okay, then mask or use the grid collection. :)
To do this with a covering grid you could pre-allocate:
Li = 0 Lo = 3
cg = pf.h.covering_grid(Li, [0, 0, 0], pf.domain_dimensions * pf.refine_by**Li) output = np.empty(pf.domain_dimensions * pf.refine_by**Lo) nd = pf.refine_by**(Lo-li) for i in range(nd): for j in range(nd): for k in range(nd): output[i::nd,j::nd,k::nd] = cg[field]
Ah ok! So this is the same as Mike's method in that a covering_grid is created at the resolution of the cell size on the specified level, and then the array values are multiplied to give the correct number for a higher resolution? (Which works fine for the basic quantities; density, energy and velocity but won't for cell mass -- which is totally fine).
Cell mass is converted correctly, I believe. This is accomplished
through a validator, NeedsGridType, which is specified if a field
*needs* to be calculated in the original grid (i.e., Level) but
otherwise fields are generated in the CG. I've verified that
specifically for CellMassMsun this is correctly done:
In [12]: ms = []
In [13]: for L in range(4):
cg = pf.h.covering_grid(L, [0,0,0], pf.domain_dimensions * pf.refine_by**L)
ms.append(cg["CellMassMsun"].sum())
....:
yt : [DEBUG ] 2013-01-16 22:00:26,392 Appending object to
galaxy0030 (type:
My only question is:
If I do this for level (e.g.) 2 and level 2 doesn't cover the whole domain -- what does the covering grid do when there are no level 2 cells over a region of space?
Fills 'em with level 1 or level 0, depending. You can just mask these out, though, which might be the best solution if the covering grid (3D, handles the mass issue, etc) is the better solution: bag_ind = cg["GridLevels"] != 2 cg[field][bad_ind] = -1.0 # or something That should do it, and that would go somewhere inside the loop above. -Matt
Thanks again!
Elizabeth _______________________________________________ yt-dev mailing list yt-dev@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
Hi Matt and Mike,
Just a finishing note to say that all the below techniques (covering_grid, grid_collection and Mike's loop) work well, but unfortunately my 190GB machine runs out of memory when I try and create a covering grid / duplicate the array elements to the size of the finest level!
So for the moment, I've abandoned that final step and just created a grid collection:
gc = pf.h.grid_collection([25e3, 25e3, 25e3], pf.h.select_grids(level)]
and written it to file.
Delightfully short; thank you!
Out of curiosity, why does grid_collection take the box centre as an argument? AMRGridCollectionBase doesn't seem to use it in its source?
Elizabeth
On Jan 17, 2013, at 12:01 PM, Matthew Turk
On Wed, Jan 16, 2013 at 9:52 PM, Elizabeth Tasker
wrote: Hi Mike and Matt,
Thank you! Both those solutions look great.
I like Mike's a lot for its transparency, but possibly Matt's might end up making slightly neater code.
One question, Matt:
Okay, sorry for being dense. But basically what I'm getting is that you want the functionality of covering grids, but you want to be able to restrict the grids from going above some level L, right?
Or below. I just want cells on one level only.
Okay, the grid collection might work, but I still think the covering grid is the best, with the masking I describe below.
gc = pf.h.grid_collection([0.5, 0.5, 0.5], pf.h.select_grids(L))
gc["Density"] gc["dx"] gc["x"]
etc. You'll have to manually manipulate the dx stuff, cell mass, etc.
And for the Level 1 case you mention above, do you want contributions from Level 0? (That's not so hard to do.)
No. So I want an array that contains only cells from level 1, with no contributions from 0 or 2.
Okay, then mask or use the grid collection. :)
To do this with a covering grid you could pre-allocate:
Li = 0 Lo = 3
cg = pf.h.covering_grid(Li, [0, 0, 0], pf.domain_dimensions * pf.refine_by**Li) output = np.empty(pf.domain_dimensions * pf.refine_by**Lo) nd = pf.refine_by**(Lo-li) for i in range(nd): for j in range(nd): for k in range(nd): output[i::nd,j::nd,k::nd] = cg[field]
Ah ok! So this is the same as Mike's method in that a covering_grid is created at the resolution of the cell size on the specified level, and then the array values are multiplied to give the correct number for a higher resolution? (Which works fine for the basic quantities; density, energy and velocity but won't for cell mass -- which is totally fine).
Cell mass is converted correctly, I believe. This is accomplished through a validator, NeedsGridType, which is specified if a field *needs* to be calculated in the original grid (i.e., Level) but otherwise fields are generated in the CG. I've verified that specifically for CellMassMsun this is correctly done:
In [12]: ms = []
In [13]: for L in range(4): cg = pf.h.covering_grid(L, [0,0,0], pf.domain_dimensions * pf.refine_by**L) ms.append(cg["CellMassMsun"].sum()) ....: yt : [DEBUG ] 2013-01-16 22:00:26,392 Appending object to galaxy0030 (type:
) yt : [DEBUG ] 2013-01-16 22:00:26,436 Getting fields ['Density'] from 1 possible grids yt : [DEBUG ] 2013-01-16 22:00:26,439 Appending object to galaxy0030 (type: ) yt : [DEBUG ] 2013-01-16 22:00:26,442 Getting fields ['Density'] from 9 possible grids yt : [DEBUG ] 2013-01-16 22:00:26,465 Appending object to galaxy0030 (type: ) yt : [DEBUG ] 2013-01-16 22:00:26,518 Getting fields ['Density'] from 17 possible grids yt : [DEBUG ] 2013-01-16 22:00:26,703 Appending object to galaxy0030 (type: ) yt : [DEBUG ] 2013-01-16 22:00:27,320 Getting fields ['Density'] from 25 possible grids In [14]: ms Out[14]: [18524964522.745506, 18524964453.698158, 18524964476.255985, 18524964484.909931]
My only question is:
If I do this for level (e.g.) 2 and level 2 doesn't cover the whole domain -- what does the covering grid do when there are no level 2 cells over a region of space?
Fills 'em with level 1 or level 0, depending. You can just mask these out, though, which might be the best solution if the covering grid (3D, handles the mass issue, etc) is the better solution:
bag_ind = cg["GridLevels"] != 2 cg[field][bad_ind] = -1.0 # or something
That should do it, and that would go somewhere inside the loop above.
-Matt
Thanks again!
Elizabeth _______________________________________________ yt-dev mailing list yt-dev@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
_______________________________________________ yt-dev mailing list yt-dev@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
On Thu, Jan 17, 2013 at 7:14 AM, Elizabeth Tasker
Hi Matt and Mike,
Just a finishing note to say that all the below techniques (covering_grid, grid_collection and Mike's loop) work well, but unfortunately my 190GB machine runs out of memory when I try and create a covering grid / duplicate the array elements to the size of the finest level!
:)
So for the moment, I've abandoned that final step and just created a grid collection:
gc = pf.h.grid_collection([25e3, 25e3, 25e3], pf.h.select_grids(level)]
and written it to file.
One last note, which is that grid collections do *no* child zone masking. So the total mass in a grid_collection != total mass in all_data().
Delightfully short; thank you!
Out of curiosity, why does grid_collection take the box centre as an argument? AMRGridCollectionBase doesn't seem to use it in its source?
It probably shouldn't. And it's not the source of the GC exactly, it's the source fed in to radius fields. But this is something of a leftover.
Elizabeth
On Jan 17, 2013, at 12:01 PM, Matthew Turk
wrote: On Wed, Jan 16, 2013 at 9:52 PM, Elizabeth Tasker
wrote: Hi Mike and Matt,
Thank you! Both those solutions look great.
I like Mike's a lot for its transparency, but possibly Matt's might end up making slightly neater code.
One question, Matt:
Okay, sorry for being dense. But basically what I'm getting is that you want the functionality of covering grids, but you want to be able to restrict the grids from going above some level L, right?
Or below. I just want cells on one level only.
Okay, the grid collection might work, but I still think the covering grid is the best, with the masking I describe below.
gc = pf.h.grid_collection([0.5, 0.5, 0.5], pf.h.select_grids(L))
gc["Density"] gc["dx"] gc["x"]
etc. You'll have to manually manipulate the dx stuff, cell mass, etc.
And for the Level 1 case you mention above, do you want contributions from Level 0? (That's not so hard to do.)
No. So I want an array that contains only cells from level 1, with no contributions from 0 or 2.
Okay, then mask or use the grid collection. :)
To do this with a covering grid you could pre-allocate:
Li = 0 Lo = 3
cg = pf.h.covering_grid(Li, [0, 0, 0], pf.domain_dimensions * pf.refine_by**Li) output = np.empty(pf.domain_dimensions * pf.refine_by**Lo) nd = pf.refine_by**(Lo-li) for i in range(nd): for j in range(nd): for k in range(nd): output[i::nd,j::nd,k::nd] = cg[field]
Ah ok! So this is the same as Mike's method in that a covering_grid is created at the resolution of the cell size on the specified level, and then the array values are multiplied to give the correct number for a higher resolution? (Which works fine for the basic quantities; density, energy and velocity but won't for cell mass -- which is totally fine).
Cell mass is converted correctly, I believe. This is accomplished through a validator, NeedsGridType, which is specified if a field *needs* to be calculated in the original grid (i.e., Level) but otherwise fields are generated in the CG. I've verified that specifically for CellMassMsun this is correctly done:
In [12]: ms = []
In [13]: for L in range(4): cg = pf.h.covering_grid(L, [0,0,0], pf.domain_dimensions * pf.refine_by**L) ms.append(cg["CellMassMsun"].sum()) ....: yt : [DEBUG ] 2013-01-16 22:00:26,392 Appending object to galaxy0030 (type:
) yt : [DEBUG ] 2013-01-16 22:00:26,436 Getting fields ['Density'] from 1 possible grids yt : [DEBUG ] 2013-01-16 22:00:26,439 Appending object to galaxy0030 (type: ) yt : [DEBUG ] 2013-01-16 22:00:26,442 Getting fields ['Density'] from 9 possible grids yt : [DEBUG ] 2013-01-16 22:00:26,465 Appending object to galaxy0030 (type: ) yt : [DEBUG ] 2013-01-16 22:00:26,518 Getting fields ['Density'] from 17 possible grids yt : [DEBUG ] 2013-01-16 22:00:26,703 Appending object to galaxy0030 (type: ) yt : [DEBUG ] 2013-01-16 22:00:27,320 Getting fields ['Density'] from 25 possible grids In [14]: ms Out[14]: [18524964522.745506, 18524964453.698158, 18524964476.255985, 18524964484.909931]
My only question is:
If I do this for level (e.g.) 2 and level 2 doesn't cover the whole domain -- what does the covering grid do when there are no level 2 cells over a region of space?
Fills 'em with level 1 or level 0, depending. You can just mask these out, though, which might be the best solution if the covering grid (3D, handles the mass issue, etc) is the better solution:
bag_ind = cg["GridLevels"] != 2 cg[field][bad_ind] = -1.0 # or something
That should do it, and that would go somewhere inside the loop above.
-Matt
Thanks again!
Elizabeth _______________________________________________ yt-dev mailing list yt-dev@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
_______________________________________________ yt-dev mailing list yt-dev@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
_______________________________________________ yt-dev mailing list yt-dev@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
participants (3)
-
Elizabeth Tasker
-
Matthew Turk
-
Michael Kuhlen