Hi all, this is about inline Enzo+yt, but since I think the solution might be on the yt end, I'm sending it here. I am attempting to profile halos located inline, and I am running into trouble when I attempt to profile over the 'TotalMassMsun' field. It is dying when it tries to access the 'Dark_Matter_Density' field, which I think it expects to be available, but isn't because Enzo only produces that when it writes to disk (AFAIK?). If this sounds correct to those in the know, I'd like to ask if anyone has a good idea how to get around this? Should I make a derived field (perhaps over-writing the built-in 'Dark_Matter_Density') using CICDeposit? I suppose I could try this solution first without emailing y'all, but I was thinking that if I/we find a working solution, it could be added to yt so the next person doesn't have to figure it out, so I'd like to minimize the false starts of poor/incorrect implementations. Thanks for your comments. -- Stephen Skory s@skory.us http://stephenskory.com/ 510.621.3687 (google voice)
Hey Stephen, I think you have two options and in principle both routes should give you the same answer. The first option is to do as you suggest and use yt to do the cloud in cell interpolation of the dark matter particles onto the grid. The second option would be to to make sure GravitatingMassFieldParticles in enzo is valid by calling the appropriate enzo grid member functions. Enzo has a ton of machinery to make sure the CIC operation happens as fast as possible using the non-blocking communication machinery, however, that means you will have a lot of programming overhead to get enzo to produce the field in the middle of a timestep. The 'best' answer probably depends on your needs, the size of the simulation, etc. Cheers, Nathan On Aug 6, 2012, at 2:56 PM, Stephen Skory wrote:
Hi all,
this is about inline Enzo+yt, but since I think the solution might be on the yt end, I'm sending it here.
I am attempting to profile halos located inline, and I am running into trouble when I attempt to profile over the 'TotalMassMsun' field. It is dying when it tries to access the 'Dark_Matter_Density' field, which I think it expects to be available, but isn't because Enzo only produces that when it writes to disk (AFAIK?). If this sounds correct to those in the know, I'd like to ask if anyone has a good idea how to get around this? Should I make a derived field (perhaps over-writing the built-in 'Dark_Matter_Density') using CICDeposit? I suppose I could try this solution first without emailing y'all, but I was thinking that if I/we find a working solution, it could be added to yt so the next person doesn't have to figure it out, so I'd like to minimize the false starts of poor/incorrect implementations.
Thanks for your comments.
-- Stephen Skory s@skory.us http://stephenskory.com/ 510.621.3687 (google voice) _______________________________________________ yt-dev mailing list yt-dev@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
Hi Stephen,
We have the capacity to allow for fallback fields; this is how
Total_Energy and Gas_Energy have been defined. If you specify a field
that duplicated one in a file, it will use the one in the file or
generate it as needed. The complication with Dark_Matter_Density is
that it's not *in* every grid, so it may not be correctly detected if
it is there, and yt would re-generate the field on the off chance it
didn't randomyl sample grids that had it.
I think on the whole, though, we should use fields that exist
everywhere. A complicating factor is that the particle deposition
routines generally don't subselect based on particle types, and when
they do, that is not robust against differences between codes. So
have a go at overriding Dark_Matter_Density, and if that fails, swap
it out for the dark matter density created by yt in the halo profiler.
The best solution, though, might be just to do what we do for
Temperature; generate it before calling yt.
-MAtt
On Mon, Aug 6, 2012 at 5:56 PM, Stephen Skory wrote:
Hi all,
this is about inline Enzo+yt, but since I think the solution might be on the yt end, I'm sending it here.
I am attempting to profile halos located inline, and I am running into trouble when I attempt to profile over the 'TotalMassMsun' field. It is dying when it tries to access the 'Dark_Matter_Density' field, which I think it expects to be available, but isn't because Enzo only produces that when it writes to disk (AFAIK?). If this sounds correct to those in the know, I'd like to ask if anyone has a good idea how to get around this? Should I make a derived field (perhaps over-writing the built-in 'Dark_Matter_Density') using CICDeposit? I suppose I could try this solution first without emailing y'all, but I was thinking that if I/we find a working solution, it could be added to yt so the next person doesn't have to figure it out, so I'd like to minimize the false starts of poor/incorrect implementations.
Thanks for your comments.
-- Stephen Skory s@skory.us http://stephenskory.com/ 510.621.3687 (google voice) _______________________________________________ yt-dev mailing list yt-dev@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
Also, it occurs to me that we should change the inner/outer iteration
to do grids on the outside, halos on the inside. With the current
profile code this is tricky or impossible, but I've been working on a
profiling refactor that could do it. You can see this here:
http://bitbucket.org/MatthewTurk/yt-profiles
https://bitbucket.org/MatthewTurk/yt-profiles/src/a6ce9142cd63/yt/data_objec...
What remains is to port all the additional options like end_collect.
As you can see, this code is much terser and simpler to modify; I
think a simple way to change with this new method would be to
construct a bunch of these new-style profiles and *not* call
add_fields, but instead unroll the add_fields loop in a routine, with
individual accumulators for each. Does that make sense?
-Matt
On Mon, Aug 6, 2012 at 9:40 PM, Matthew Turk
Hi Stephen,
We have the capacity to allow for fallback fields; this is how Total_Energy and Gas_Energy have been defined. If you specify a field that duplicated one in a file, it will use the one in the file or generate it as needed. The complication with Dark_Matter_Density is that it's not *in* every grid, so it may not be correctly detected if it is there, and yt would re-generate the field on the off chance it didn't randomyl sample grids that had it.
I think on the whole, though, we should use fields that exist everywhere. A complicating factor is that the particle deposition routines generally don't subselect based on particle types, and when they do, that is not robust against differences between codes. So have a go at overriding Dark_Matter_Density, and if that fails, swap it out for the dark matter density created by yt in the halo profiler.
The best solution, though, might be just to do what we do for Temperature; generate it before calling yt.
-MAtt
On Mon, Aug 6, 2012 at 5:56 PM, Stephen Skory
wrote:Hi all,
this is about inline Enzo+yt, but since I think the solution might be on the yt end, I'm sending it here.
I am attempting to profile halos located inline, and I am running into trouble when I attempt to profile over the 'TotalMassMsun' field. It is dying when it tries to access the 'Dark_Matter_Density' field, which I think it expects to be available, but isn't because Enzo only produces that when it writes to disk (AFAIK?). If this sounds correct to those in the know, I'd like to ask if anyone has a good idea how to get around this? Should I make a derived field (perhaps over-writing the built-in 'Dark_Matter_Density') using CICDeposit? I suppose I could try this solution first without emailing y'all, but I was thinking that if I/we find a working solution, it could be added to yt so the next person doesn't have to figure it out, so I'd like to minimize the false starts of poor/incorrect implementations.
Thanks for your comments.
-- Stephen Skory s@skory.us http://stephenskory.com/ 510.621.3687 (google voice) _______________________________________________ yt-dev mailing list yt-dev@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
Hi Matt,
Does that make sense?
How do you see this working from the end-user's prospective? Would they put all the spheres they'd like to profile over into a list, and then also put the fields & weightings they want in another? -- Stephen Skory s@skory.us http://stephenskory.com/ 510.621.3687 (google voice)
Hi Stephen,
On Tue, Aug 7, 2012 at 4:16 PM, Stephen Skory wrote:
Hi Matt,
Does that make sense?
How do you see this working from the end-user's prospective? Would they put all the spheres they'd like to profile over into a list, and then also put the fields & weightings they want in another?
I think that's one possibility. I imagined a wrapper function that accepted either data_sources or instantiated profiles along with field specs, and then iterated over them. -Matt
-- Stephen Skory s@skory.us http://stephenskory.com/ 510.621.3687 (google voice) _______________________________________________ yt-dev mailing list yt-dev@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
participants (3)
-
Matthew Turk
-
Nathan Goldbaum
-
Stephen Skory