Finding the max density from a checkpoint dump?

Hey everyone, My Enzo simulation has refined to enough levels that I need to make checkpoint dumps mid-timestep and do YT analysis from there. I can read in the checkpoint dump but when I go to find the max density I get this error: http://pastebin.com/2mNVxNyW Anybody have any idea what is going on and how to fix this? Also, since this dump is in the middle of the W-cycle, I believe the time on the finest grid is different then the root grid. How do I find the time on the finest grid for this checkpoint dump? Thanks. -- ------------------------------------------------------------------------ Joseph Smidt <josephsmidt@gmail.com> Theoretical Division P.O. Box 1663, Mail Stop B283 Los Alamos, NM 87545 Office: 505-665-9752 Fax: 505-667-1931

Hi Joseph, Can you update to the latest stable version of yt? You should be able to upgrade by doing 'yt update' from the command line. Once you've updated, you'll be able to additionally update yt's dependencies by running 'yt update --all'. Hope that helps, let us know if you're still having trouble after updating. Nathan On May 28, 2013, at 6:45 PM, Joseph Smidt <josephsmidt@gmail.com> wrote: Hey everyone, My Enzo simulation has refined to enough levels that I need to make checkpoint dumps mid-timestep and do YT analysis from there. I can read in the checkpoint dump but when I go to find the max density I get this error: http://pastebin.com/2mNVxNyW Anybody have any idea what is going on and how to fix this? Also, since this dump is in the middle of the W-cycle, I believe the time on the finest grid is different then the root grid. How do I find the time on the finest grid for this checkpoint dump? Thanks. -- ------------------------------------------------------------------------ Joseph Smidt <josephsmidt@gmail.com> Theoretical Division P.O. Box 1663, Mail Stop B283 Los Alamos, NM 87545 Office: 505-665-9752 Fax: 505-667-1931 _______________________________________________ yt-users mailing list yt-users@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org

Nathan, Thanks for your response. I believe I have the lastest version of yt. I get this if I do an hg tip: [jsmidt@mu-fe1 yt]$ hg tip changeset: 8217:fb6cce6a2936 branch: yt tag: tip user: Nathan Goldbaum <goldbaum@ucolick.org> date: Mon May 27 09:24:17 2013 -0700 files: yt/config.py description: Bumping the gold standard revision to gold0008. Now the checkpoint output is slightly different than a normal data dump. for instance, the DD0047/output_0047 file has this comment at the top which cannot be read into yt without being removed: # WARNING! This is a checkpoint dump! Lots of data! InitialCycleNumber = 228 InitialTime = 20.046508365013 InitialCPUTime = 0 And there are additional fields like Acceleration fields with these checkpoint dumps. So I was wondering if this other error when finding the max values of fields is also checkpoint dump specific as the structure seems slightly different compared to normal data dumps. Thanks again. On Tue, May 28, 2013 at 7:45 PM, Joseph Smidt <josephsmidt@gmail.com> wrote:
Hey everyone,
My Enzo simulation has refined to enough levels that I need to make checkpoint dumps mid-timestep and do YT analysis from there. I can read in the checkpoint dump but when I go to find the max density I get this error: http://pastebin.com/2mNVxNyW
Anybody have any idea what is going on and how to fix this? Also, since this dump is in the middle of the W-cycle, I believe the time on the finest grid is different then the root grid. How do I find the time on the finest grid for this checkpoint dump? Thanks.
-- ------------------------------------------------------------------------ Joseph Smidt <josephsmidt@gmail.com>
Theoretical Division P.O. Box 1663, Mail Stop B283 Los Alamos, NM 87545 Office: 505-665-9752 Fax: 505-667-1931
-- ------------------------------------------------------------------------ Joseph Smidt <josephsmidt@gmail.com> Theoretical Division P.O. Box 1663, Mail Stop B283 Los Alamos, NM 87545 Office: 505-665-9752 Fax: 505-667-1931

Hi Joseph, On Tue, May 28, 2013 at 11:07 PM, Joseph Smidt <josephsmidt@gmail.com> wrote:
Nathan,
Thanks for your response. I believe I have the lastest version of yt. I get this if I do an hg tip:
[jsmidt@mu-fe1 yt]$ hg tip changeset: 8217:fb6cce6a2936 branch: yt tag: tip user: Nathan Goldbaum <goldbaum@ucolick.org> date: Mon May 27 09:24:17 2013 -0700 files: yt/config.py description: Bumping the gold standard revision to gold0008.
Now the checkpoint output is slightly different than a normal data dump. for instance, the DD0047/output_0047 file has this comment at the top which cannot be read into yt without being removed:
# WARNING! This is a checkpoint dump! Lots of data! InitialCycleNumber = 228 InitialTime = 20.046508365013 InitialCPUTime = 0
And there are additional fields like Acceleration fields with these checkpoint dumps. So I was wondering if this other error when finding the max values of fields is also checkpoint dump specific as the structure seems slightly different compared to normal data dumps. Thanks again.
I wrote the checkpoint dump mechanism in Enzo, and it wasn't meant to be used for analysis. It stores the fluxes (where they exist), the acceleration, ghost zones, "old" baryon fields, and probably some other things I've forgotten now. The idea was to dump all of the internal data structures that accumulate over the course of a timestep, and then load them in at a later time, hopefully preserving the exact state. (For what it's worth, it never was as successful as reproducing the exact state of the computation as I had hoped it would be.) So my guess is that you're seeing issues with the ghost zones changing the dimensions of the grids in funny ways. In many ways, the checkpoint dumps are more similar to the analysis conducted inline in Enzo than they are to analysis conducted of standard Enzo data dumps -- the ghost zones will need to be sliced off and the old/new fields will need to be interpolated to the centered time. So to make a long story short, when I use checkpoint dumps in Enzo to restart the computation, I don't use them for analysis -- I also have Enzo output standard data dumps and use those. If we were to try to make this work, it would need a new IO handler similar to the Inline IO handler which interpolates between times and slices off ghost zones. -Matt
On Tue, May 28, 2013 at 7:45 PM, Joseph Smidt <josephsmidt@gmail.com> wrote:
Hey everyone,
My Enzo simulation has refined to enough levels that I need to make checkpoint dumps mid-timestep and do YT analysis from there. I can read in the checkpoint dump but when I go to find the max density I get this error: http://pastebin.com/2mNVxNyW
Anybody have any idea what is going on and how to fix this? Also, since this dump is in the middle of the W-cycle, I believe the time on the finest grid is different then the root grid. How do I find the time on the finest grid for this checkpoint dump? Thanks.
-- ------------------------------------------------------------------------ Joseph Smidt <josephsmidt@gmail.com>
Theoretical Division P.O. Box 1663, Mail Stop B283 Los Alamos, NM 87545 Office: 505-665-9752 Fax: 505-667-1931
-- ------------------------------------------------------------------------ Joseph Smidt <josephsmidt@gmail.com>
Theoretical Division P.O. Box 1663, Mail Stop B283 Los Alamos, NM 87545 Office: 505-665-9752 Fax: 505-667-1931
_______________________________________________ yt-users mailing list yt-users@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org

Thanks everyone, the outputnow and subcycleCount are reading into yt just fine. Thanks for the tips. On Tue, May 28, 2013 at 9:07 PM, Joseph Smidt <josephsmidt@gmail.com> wrote:
Nathan,
Thanks for your response. I believe I have the lastest version of yt. I get this if I do an hg tip:
[jsmidt@mu-fe1 yt]$ hg tip changeset: 8217:fb6cce6a2936 branch: yt tag: tip user: Nathan Goldbaum <goldbaum@ucolick.org> date: Mon May 27 09:24:17 2013 -0700 files: yt/config.py description: Bumping the gold standard revision to gold0008.
Now the checkpoint output is slightly different than a normal data dump. for instance, the DD0047/output_0047 file has this comment at the top which cannot be read into yt without being removed:
# WARNING! This is a checkpoint dump! Lots of data! InitialCycleNumber = 228 InitialTime = 20.046508365013 InitialCPUTime = 0
And there are additional fields like Acceleration fields with these checkpoint dumps. So I was wondering if this other error when finding the max values of fields is also checkpoint dump specific as the structure seems slightly different compared to normal data dumps. Thanks again.
On Tue, May 28, 2013 at 7:45 PM, Joseph Smidt <josephsmidt@gmail.com>wrote:
Hey everyone,
My Enzo simulation has refined to enough levels that I need to make checkpoint dumps mid-timestep and do YT analysis from there. I can read in the checkpoint dump but when I go to find the max density I get this error: http://pastebin.com/2mNVxNyW
Anybody have any idea what is going on and how to fix this? Also, since this dump is in the middle of the W-cycle, I believe the time on the finest grid is different then the root grid. How do I find the time on the finest grid for this checkpoint dump? Thanks.
-- ------------------------------------------------------------------------ Joseph Smidt <josephsmidt@gmail.com>
Theoretical Division P.O. Box 1663, Mail Stop B283 Los Alamos, NM 87545 Office: 505-665-9752 Fax: 505-667-1931
-- ------------------------------------------------------------------------ Joseph Smidt <josephsmidt@gmail.com>
Theoretical Division P.O. Box 1663, Mail Stop B283 Los Alamos, NM 87545 Office: 505-665-9752 Fax: 505-667-1931
-- ------------------------------------------------------------------------ Joseph Smidt <josephsmidt@gmail.com> Theoretical Division P.O. Box 1663, Mail Stop B283 Los Alamos, NM 87545 Office: 505-665-9752 Fax: 505-667-1931

Hi Joseph, Excuse me - I saw in your paste that your yt python sources live in a yt-2.4 egg, but apparently that has nothing to do with the current version. Looking at the code, it seems that somehow the 'x', 'y' and 'z' fields are different shapes for some of the grids in your data. You can see for yourself what's going on by turning on the %debug magic in IPython and rerunning the command that crashes. You'll be dumped into an ipdb prompt in the context where IndexError is raised. You can see how 'x', 'y', and 'z' are defined by inspecting _coordX, _coordY, and _coordZ in $YT_HG/yt/data_objects/universal_fields.py. I've never tried opening a checkpoint data dump in yt, although I suspect it's not done very often. In the past I've used Enzo's subcycle-based data outputs to get intermediate interpolated data dumps in situations where I can't simply lower dtDatDump. There are many other enzo users on this list so perhaps one of them have had luck with this in the past. Sorry I can't be of more help here. Cheers, Nathan On May 28, 2013, at 6:45 PM, Joseph Smidt <josephsmidt@gmail.com> wrote:
Hey everyone,
My Enzo simulation has refined to enough levels that I need to make checkpoint dumps mid-timestep and do YT analysis from there. I can read in the checkpoint dump but when I go to find the max density I get this error: http://pastebin.com/2mNVxNyW
Anybody have any idea what is going on and how to fix this? Also, since this dump is in the middle of the W-cycle, I believe the time on the finest grid is different then the root grid. How do I find the time on the finest grid for this checkpoint dump? Thanks.
-- ------------------------------------------------------------------------ Joseph Smidt <josephsmidt@gmail.com>
Theoretical Division P.O. Box 1663, Mail Stop B283 Los Alamos, NM 87545 Office: 505-665-9752 Fax: 505-667-1931 _______________________________________________ yt-users mailing list yt-users@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
participants (3)
-
Joseph Smidt
-
Matthew Turk
-
Nathan Goldbaum