
Hi yt-users, I am trying to make phase_objects of Athena data. I am able to make phase_objects if I use the "expected" set of variables (will show below), but kill my kernel or am kicked out of python if I try and use an "unknown field" read in from a set of vtk files. from yt.mods import * pf = load("id0/rps.0080.vtk") yt : [INFO ] 2012-12-13 23:27:08,678 Temporarily setting domain_right_edge = -domain_left_edge. This will be corrected automatically if it is not the case. yt : [INFO ] 2012-12-13 23:27:08,974 Parameters: current_time = 40.00096 yt : [INFO ] 2012-12-13 23:27:08,974 Parameters: domain_dimensions = [1048 1048 655] yt : [INFO ] 2012-12-13 23:27:08,975 Parameters: domain_left_edge = [-3.2 -3.2 -2. ] yt : [INFO ] 2012-12-13 23:27:08,976 Parameters: domain_right_edge = [ 3.2 3.2 2. ] yt : [INFO ] 2012-12-13 23:27:08,977 Parameters: cosmological_simulation = 0.0 yt : [INFO ] 2012-12-13 23:27:13,214 Adding unknown field specific_scalar[0] to list of fields pf.h.field_list ['density', 'pressure', 'velocity_z', 'velocity_x', 'velocity_y', 'specific_scalar[0]'] alld = pf.h.all_data() tracerp25 = alld.cut_region(["grid['specific_scalar[0]'] > 0.25", "grid['z'] > 0.01"]) ##These commands works: radmean = tracerp25.quantities["WeightedAverageQuantity"]("Radius","Ones") pc = PlotCollectionInteractive(pf) pc.add_phase_object(alld,["z","velocity_z","CellMass"],weight=None) #The calls below fail. There is no error message. If I am in iPython, #the kernel dies. If I am running on a computer, it freezes and #eventually I just get the message #Killed pc.add_phase_object(alld,["z","specific_scalar[0]","CellMass"],weight=None) pc.add_phase_object(tracerp25,["z","velocity_z","CellMass"],weight=None) #If I read in a single vtk file ("id122/rps-id122.0080.vtk") the above #commands work Any help would be much appreciated! Thanks, Stephanie

Hi Stephanie, Just want to make sure that you're using the latest version of the yt development tree--it contains some important Athena fixes that were put in a few weeks back. If so, then we'll work from there to figure out what's gone wrong. Best, John On Dec 17, 2012, at 5:44 PM, Stephanie Tonnesen <stonnes@gmail.com> wrote:
Hi yt-users,
I am trying to make phase_objects of Athena data. I am able to make phase_objects if I use the "expected" set of variables (will show below), but kill my kernel or am kicked out of python if I try and use an "unknown field" read in from a set of vtk files.
from yt.mods import *
pf = load("id0/rps.0080.vtk")
yt : [INFO ] 2012-12-13 23:27:08,678 Temporarily setting domain_right_edge = -domain_left_edge. This will be corrected automatically if it is not the case. yt : [INFO ] 2012-12-13 23:27:08,974 Parameters: current_time = 40.00096 yt : [INFO ] 2012-12-13 23:27:08,974 Parameters: domain_dimensions = [1048 1048 655] yt : [INFO ] 2012-12-13 23:27:08,975 Parameters: domain_left_edge = [-3.2 -3.2 -2. ] yt : [INFO ] 2012-12-13 23:27:08,976 Parameters: domain_right_edge = [ 3.2 3.2 2. ] yt : [INFO ] 2012-12-13 23:27:08,977 Parameters: cosmological_simulation = 0.0 yt : [INFO ] 2012-12-13 23:27:13,214 Adding unknown field specific_scalar[0] to list of fields
pf.h.field_list
['density', 'pressure', 'velocity_z', 'velocity_x', 'velocity_y', 'specific_scalar[0]'] alld = pf.h.all_data() tracerp25 = alld.cut_region(["grid['specific_scalar[0]'] > 0.25", "grid['z'] > 0.01"])
##These commands works: radmean = tracerp25.quantities["WeightedAverageQuantity"]("Radius","Ones")
pc = PlotCollectionInteractive(pf)
pc.add_phase_object(alld,["z","velocity_z","CellMass"],weight=None)
#The calls below fail. There is no error message. If I am in iPython, #the kernel dies. If I am running on a computer, it freezes and #eventually I just get the message #Killed
pc.add_phase_object(alld,["z","specific_scalar[0]","CellMass"],weight=None) pc.add_phase_object(tracerp25,["z","velocity_z","CellMass"],weight=None)
#If I read in a single vtk file ("id122/rps-id122.0080.vtk") the above #commands work Any help would be much appreciated! Thanks,
Stephanie _______________________________________________ yt-users mailing list yt-users@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org

Hi, Yup. Just to be sure I updated again and reattempted to make the phase_object and python was Killed again. Thanks! Stephanie On Tue, Dec 18, 2012 at 12:35 AM, John ZuHone <jzuhone@gmail.com> wrote:
Hi Stephanie,
Just want to make sure that you're using the latest version of the yt development tree--it contains some important Athena fixes that were put in a few weeks back.
If so, then we'll work from there to figure out what's gone wrong.
Best,
John
On Dec 17, 2012, at 5:44 PM, Stephanie Tonnesen <stonnes@gmail.com> wrote:
Hi yt-users,
I am trying to make phase_objects of Athena data. I am able to make phase_objects if I use the "expected" set of variables (will show below), but kill my kernel or am kicked out of python if I try and use an "unknown field" read in from a set of vtk files.
from yt.mods import *
pf = load("id0/rps.0080.vtk")
yt : [INFO ] 2012-12-13 23:27:08,678 Temporarily setting domain_right_edge = -domain_left_edge. This will be corrected automatically if it is not the case. yt : [INFO ] 2012-12-13 23:27:08,974 Parameters: current_time = 40.00096 yt : [INFO ] 2012-12-13 23:27:08,974 Parameters: domain_dimensions = [1048 1048 655] yt : [INFO ] 2012-12-13 23:27:08,975 Parameters: domain_left_edge = [-3.2 -3.2 -2. ] yt : [INFO ] 2012-12-13 23:27:08,976 Parameters: domain_right_edge = [ 3.2 3.2 2. ] yt : [INFO ] 2012-12-13 23:27:08,977 Parameters: cosmological_simulation = 0.0 yt : [INFO ] 2012-12-13 23:27:13,214 Adding unknown field specific_scalar[0] to list of fields
pf.h.field_list
['density', 'pressure', 'velocity_z', 'velocity_x', 'velocity_y', 'specific_scalar[0]']
alld = pf.h.all_data() tracerp25 = alld.cut_region(["grid['specific_scalar[0]'] > 0.25", "grid['z'] > 0.01"])
##These commands works:
radmean = tracerp25.quantities["WeightedAverageQuantity"]("Radius","Ones")
pc = PlotCollectionInteractive(pf)
pc.add_phase_object(alld,["z","velocity_z","CellMass"],weight=None)
#The calls below fail. There is no error message. If I am in iPython, #the kernel dies. If I am running on a computer, it freezes and #eventually I just get the message #Killed
pc.add_phase_object(alld,["z","specific_scalar[0]","CellMass"],weight=None) pc.add_phase_object(tracerp25,["z","velocity_z","CellMass"],weight=None)
#If I read in a single vtk file ("id122/rps-id122.0080.vtk") the above #commands work
Any help would be much appreciated! Thanks,
Stephanie _______________________________________________ yt-users mailing list yt-users@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
_______________________________________________ yt-users mailing list yt-users@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org

Hi Stephanie, Ok, thanks for checking that. I guess I'm not sure what you mean by the "expected" set of variables. Do you mean the ones that yt defines, or the ones that are in the field list? It looks like "velocity_z" works for one call and doesn't for another. Are you able to do anything else with "specific_scalar[0]"? Like make a slice, or a projection of some sort? Or maybe take the weighted average of it in a sphere. I'd be interested in seeing if the problem was restricted to phase plotting. How big is the dataset, by the way? Is there something that fails in the same way that the both of us can test on? Best, John On Tue, Dec 18, 2012 at 10:53 AM, Stephanie Tonnesen <stonnes@gmail.com>wrote:
Hi,
Yup. Just to be sure I updated again and reattempted to make the phase_object and python was Killed again.
Thanks!
Stephanie
On Tue, Dec 18, 2012 at 12:35 AM, John ZuHone <jzuhone@gmail.com> wrote:
Hi Stephanie,
Just want to make sure that you're using the latest version of the yt development tree--it contains some important Athena fixes that were put in a few weeks back.
If so, then we'll work from there to figure out what's gone wrong.
Best,
John
On Dec 17, 2012, at 5:44 PM, Stephanie Tonnesen <stonnes@gmail.com> wrote:
Hi yt-users,
I am trying to make phase_objects of Athena data. I am able to make phase_objects if I use the "expected" set of variables (will show below), but kill my kernel or am kicked out of python if I try and use an "unknown field" read in from a set of vtk files.
from yt.mods import *
pf = load("id0/rps.0080.vtk")
yt : [INFO ] 2012-12-13 23:27:08,678 Temporarily setting domain_right_edge = -domain_left_edge. This will be corrected automatically if it is not the case. yt : [INFO ] 2012-12-13 23:27:08,974 Parameters: current_time = 40.00096 yt : [INFO ] 2012-12-13 23:27:08,974 Parameters: domain_dimensions = [1048 1048 655] yt : [INFO ] 2012-12-13 23:27:08,975 Parameters: domain_left_edge = [-3.2 -3.2 -2. ] yt : [INFO ] 2012-12-13 23:27:08,976 Parameters: domain_right_edge = [ 3.2 3.2 2. ] yt : [INFO ] 2012-12-13 23:27:08,977 Parameters: cosmological_simulation = 0.0 yt : [INFO ] 2012-12-13 23:27:13,214 Adding unknown field specific_scalar[0] to list of fields
pf.h.field_list
['density', 'pressure', 'velocity_z', 'velocity_x', 'velocity_y', 'specific_scalar[0]']
alld = pf.h.all_data() tracerp25 = alld.cut_region(["grid['specific_scalar[0]'] > 0.25", "grid['z'] > 0.01"])
##These commands works:
radmean = tracerp25.quantities["WeightedAverageQuantity"]("Radius","Ones")
pc = PlotCollectionInteractive(pf)
pc.add_phase_object(alld,["z","velocity_z","CellMass"],weight=None)
#The calls below fail. There is no error message. If I am in iPython, #the kernel dies. If I am running on a computer, it freezes and #eventually I just get the message #Killed
pc.add_phase_object(alld,["z","specific_scalar[0]","CellMass"],weight=None) pc.add_phase_object(tracerp25,["z","velocity_z","CellMass"],weight=None)
#If I read in a single vtk file ("id122/rps-id122.0080.vtk") the above #commands work
Any help would be much appreciated! Thanks,
Stephanie _______________________________________________ yt-users mailing list yt-users@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
_______________________________________________ yt-users mailing list yt-users@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
_______________________________________________ yt-users mailing list yt-users@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
-- John ZuHone Postdoctoral Researcher NASA/Goddard Space Flight Center jzuhone@gmail.com john.zuhone@nasa.gov

Hi John, So, what I mean by "expected" variables are basically all variables BUT specific_scalar[0]. When I read in the vtk file, you can see that the bottom line says yt : [INFO ] 2012-12-13 23:27:13,214 Adding unknown field specific_scalar[0] to list of fields Yes, I can do any of the "quantities" commands, for example: alld = pf.h.all_data() tracerp25 = alld.cut_region(["grid['specific_scalar[0]'] > 0.25", "grid['z'] > 0.01"]) radmean = tracerp25.quantities["WeightedAverageQuantity"]("Radius","Ones") Slicing also works: p = SlicePlot(pf,"x","specific_scalar[0]") Now I am starting to worry that it is because output is so big--each vtk file is 72 MB, so a full output is almost 25 GB. When I just use two outputs (moving them so yt only sees id0 and id1 folders) the phase plot works. But, after running the add_phase_object command on the whole data set for more than 1.5 hours I still have no result and my computer keeps freezing--although top shows that only 42% of memory is being used. Is it possible that my output is just too big? Stephanie On Tue, Dec 18, 2012 at 11:00 AM, John Zuhone <jzuhone@gmail.com> wrote:
Hi Stephanie,
Ok, thanks for checking that.
I guess I'm not sure what you mean by the "expected" set of variables. Do you mean the ones that yt defines, or the ones that are in the field list? It looks like "velocity_z" works for one call and doesn't for another.
Are you able to do anything else with "specific_scalar[0]"? Like make a slice, or a projection of some sort? Or maybe take the weighted average of it in a sphere. I'd be interested in seeing if the problem was restricted to phase plotting.
How big is the dataset, by the way? Is there something that fails in the same way that the both of us can test on?
Best,
John
On Tue, Dec 18, 2012 at 10:53 AM, Stephanie Tonnesen <stonnes@gmail.com>wrote:
Hi,
Yup. Just to be sure I updated again and reattempted to make the phase_object and python was Killed again.
Thanks!
Stephanie
On Tue, Dec 18, 2012 at 12:35 AM, John ZuHone <jzuhone@gmail.com> wrote:
Hi Stephanie,
Just want to make sure that you're using the latest version of the yt development tree--it contains some important Athena fixes that were put in a few weeks back.
If so, then we'll work from there to figure out what's gone wrong.
Best,
John
On Dec 17, 2012, at 5:44 PM, Stephanie Tonnesen <stonnes@gmail.com> wrote:
Hi yt-users,
I am trying to make phase_objects of Athena data. I am able to make phase_objects if I use the "expected" set of variables (will show below), but kill my kernel or am kicked out of python if I try and use an "unknown field" read in from a set of vtk files.
from yt.mods import *
pf = load("id0/rps.0080.vtk")
yt : [INFO ] 2012-12-13 23:27:08,678 Temporarily setting domain_right_edge = -domain_left_edge. This will be corrected automatically if it is not the case. yt : [INFO ] 2012-12-13 23:27:08,974 Parameters: current_time = 40.00096 yt : [INFO ] 2012-12-13 23:27:08,974 Parameters: domain_dimensions = [1048 1048 655] yt : [INFO ] 2012-12-13 23:27:08,975 Parameters: domain_left_edge = [-3.2 -3.2 -2. ] yt : [INFO ] 2012-12-13 23:27:08,976 Parameters: domain_right_edge = [ 3.2 3.2 2. ] yt : [INFO ] 2012-12-13 23:27:08,977 Parameters: cosmological_simulation = 0.0 yt : [INFO ] 2012-12-13 23:27:13,214 Adding unknown field specific_scalar[0] to list of fields
pf.h.field_list
['density', 'pressure', 'velocity_z', 'velocity_x', 'velocity_y', 'specific_scalar[0]']
alld = pf.h.all_data() tracerp25 = alld.cut_region(["grid['specific_scalar[0]'] > 0.25", "grid['z'] > 0.01"])
##These commands works:
radmean = tracerp25.quantities["WeightedAverageQuantity"]("Radius","Ones")
pc = PlotCollectionInteractive(pf)
pc.add_phase_object(alld,["z","velocity_z","CellMass"],weight=None)
#The calls below fail. There is no error message. If I am in iPython, #the kernel dies. If I am running on a computer, it freezes and #eventually I just get the message #Killed
pc.add_phase_object(alld,["z","specific_scalar[0]","CellMass"],weight=None) pc.add_phase_object(tracerp25,["z","velocity_z","CellMass"],weight=None)
#If I read in a single vtk file ("id122/rps-id122.0080.vtk") the above #commands work
Any help would be much appreciated! Thanks,
Stephanie _______________________________________________ yt-users mailing list yt-users@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
_______________________________________________ yt-users mailing list yt-users@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
_______________________________________________ yt-users mailing list yt-users@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
-- John ZuHone
Postdoctoral Researcher NASA/Goddard Space Flight Center
jzuhone@gmail.com john.zuhone@nasa.gov
_______________________________________________ yt-users mailing list yt-users@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org

Hi Stephanie, I doubt that 25 GB is too big. Can you also try a weighted average quantity of specific_scalar[0] itself? When you use only id0 and id1, you are potentially also changing domain_right_edge, since for Athena yt has to go through all of the id*/* to find the largest right edge for any grid to determine domain_right_edge. Can you figure out what pf.domain_right_edge and pf.domain_dimensions are for the two different cases (where you have all the data and also where you have just id0 and id1)? I'm not sure at the moment what that would tell us, but I'm curious about it. As another possible way to get some clues, I am aware that Athena has a tool that joins the vtk files into one. Is it possible to run that and then try the same script on the merged file? I'm also wondering if Matt and Sam have any thoughts on this. Best, John On Dec 18, 2012, at 1:21 PM, Stephanie Tonnesen <stonnes@gmail.com> wrote:
Hi John,
So, what I mean by "expected" variables are basically all variables BUT specific_scalar[0]. When I read in the vtk file, you can see that the bottom line says
yt : [INFO ] 2012-12-13 23:27:13,214 Adding unknown field specific_scalar[0] to list of fields
Yes, I can do any of the "quantities" commands, for example:
alld = pf.h.all_data() tracerp25 = alld.cut_region(["grid['specific_scalar[0]'] > 0.25", "grid['z'] > 0.01"]) radmean = tracerp25.quantities["WeightedAverageQuantity"]("Radius","Ones")
Slicing also works:
p = SlicePlot(pf,"x","specific_scalar[0]")
Now I am starting to worry that it is because output is so big--each vtk file is 72 MB, so a full output is almost 25 GB. When I just use two outputs (moving them so yt only sees id0 and id1 folders) the phase plot works. But, after running the add_phase_object command on the whole data set for more than 1.5 hours I still have no result and my computer keeps freezing--although top shows that only 42% of memory is being used.
Is it possible that my output is just too big?
Stephanie
On Tue, Dec 18, 2012 at 11:00 AM, John Zuhone <jzuhone@gmail.com> wrote: Hi Stephanie,
Ok, thanks for checking that.
I guess I'm not sure what you mean by the "expected" set of variables. Do you mean the ones that yt defines, or the ones that are in the field list? It looks like "velocity_z" works for one call and doesn't for another.
Are you able to do anything else with "specific_scalar[0]"? Like make a slice, or a projection of some sort? Or maybe take the weighted average of it in a sphere. I'd be interested in seeing if the problem was restricted to phase plotting.
How big is the dataset, by the way? Is there something that fails in the same way that the both of us can test on?
Best,
John
On Tue, Dec 18, 2012 at 10:53 AM, Stephanie Tonnesen <stonnes@gmail.com> wrote:
Hi,
Yup. Just to be sure I updated again and reattempted to make the phase_object and python was Killed again.
Thanks!
Stephanie
On Tue, Dec 18, 2012 at 12:35 AM, John ZuHone <jzuhone@gmail.com> wrote: Hi Stephanie,
Just want to make sure that you're using the latest version of the yt development tree--it contains some important Athena fixes that were put in a few weeks back.
If so, then we'll work from there to figure out what's gone wrong.
Best,
John
On Dec 17, 2012, at 5:44 PM, Stephanie Tonnesen <stonnes@gmail.com> wrote:
Hi yt-users,
I am trying to make phase_objects of Athena data. I am able to make phase_objects if I use the "expected" set of variables (will show below), but kill my kernel or am kicked out of python if I try and use an "unknown field" read in from a set of vtk files.
from yt.mods import *
pf = load("id0/rps.0080.vtk")
yt : [INFO ] 2012-12-13 23:27:08,678 Temporarily setting domain_right_edge = -domain_left_edge. This will be corrected automatically if it is not the case. yt : [INFO ] 2012-12-13 23:27:08,974 Parameters: current_time = 40.00096 yt : [INFO ] 2012-12-13 23:27:08,974 Parameters: domain_dimensions = [1048 1048 655] yt : [INFO ] 2012-12-13 23:27:08,975 Parameters: domain_left_edge = [-3.2 -3.2 -2. ] yt : [INFO ] 2012-12-13 23:27:08,976 Parameters: domain_right_edge = [ 3.2 3.2 2. ] yt : [INFO ] 2012-12-13 23:27:08,977 Parameters: cosmological_simulation = 0.0 yt : [INFO ] 2012-12-13 23:27:13,214 Adding unknown field specific_scalar[0] to list of fields
pf.h.field_list
['density', 'pressure', 'velocity_z', 'velocity_x', 'velocity_y', 'specific_scalar[0]'] alld = pf.h.all_data() tracerp25 = alld.cut_region(["grid['specific_scalar[0]'] > 0.25", "grid['z'] > 0.01"])
##These commands works: radmean = tracerp25.quantities["WeightedAverageQuantity"]("Radius","Ones")
pc = PlotCollectionInteractive(pf)
pc.add_phase_object(alld,["z","velocity_z","CellMass"],weight=None)
#The calls below fail. There is no error message. If I am in iPython, #the kernel dies. If I am running on a computer, it freezes and #eventually I just get the message #Killed
pc.add_phase_object(alld,["z","specific_scalar[0]","CellMass"],weight=None) pc.add_phase_object(tracerp25,["z","velocity_z","CellMass"],weight=None)
#If I read in a single vtk file ("id122/rps-id122.0080.vtk") the above #commands work Any help would be much appreciated! Thanks,
Stephanie _______________________________________________ yt-users mailing list yt-users@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
_______________________________________________ yt-users mailing list yt-users@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
_______________________________________________ yt-users mailing list yt-users@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
-- John ZuHone
Postdoctoral Researcher NASA/Goddard Space Flight Center
jzuhone@gmail.com john.zuhone@nasa.gov
_______________________________________________ yt-users mailing list yt-users@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
_______________________________________________ yt-users mailing list yt-users@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org

Hi John, The weighted average quantity call of specific_scalar[0] worked. The domain info: pf.domain_right_edge array([ 3.2, 3.2, 3.8])
pf.domain_dimensions array([1048, 1048, 950])
And then using id0 and id1: pf2.domain_right_edge array([-1.349618, -2.259542, -1.145263])
pf2.domain_dimensions array([303, 154, 140])
When I combined the vtk files into a single 25 GB file, I get an error when creating the PlotCollection: pc = PlotCollection(pf) Traceback (most recent call last): File "<stdin>", line 1, in <module> File "/u/stonnes/Installs/yt-x86_64/src/yt-hg/yt/visualization/plot_collection.py", line 121, in __init__ v,self.c = pf.h.find_max("Density") # @todo: ensure no caching File "/u/stonnes/Installs/yt-x86_64/src/yt-hg/yt/data_objects/object_finding_mixin.py", line 61, in find_max mg, mc, mv, pos = self.find_max_cell_location(field, finest_levels) File "/u/stonnes/Installs/yt-x86_64/src/yt-hg/yt/data_objects/object_finding_mixin.py", line 74, in find_max_cell_location source.quantities["MaxLocation"]( field, lazy_reader=True) File "/u/stonnes/Installs/yt-x86_64/src/yt-hg/yt/data_objects/derived_quantities.py", line 92, in __call__ return self._call_func_lazy(args, kwargs) File "/u/stonnes/Installs/yt-x86_64/src/yt-hg/yt/data_objects/derived_quantities.py", line 99, in _call_func_lazy rv = self.func(GridChildMaskWrapper(g, self._data_source), *args, **kwargs) File "/u/stonnes/Installs/yt-x86_64/src/yt-hg/yt/data_objects/derived_quantities.py", line 649, in _MaxLocation mx, my, mz = [data[ax][maxi] for ax in 'xyz'] File "/u/stonnes/Installs/yt-x86_64/src/yt-hg/yt/data_objects/derived_quantities.py", line 60, in __getitem__ data = self.data_source._get_data_from_grid(self.grid, item) File "/u/stonnes/Installs/yt-x86_64/src/yt-hg/yt/data_objects/data_containers.py", line 93, in save_state tr = func(self, grid, field, *args, **kwargs) File "/u/stonnes/Installs/yt-x86_64/src/yt-hg/yt/data_objects/data_containers.py", line 2630, in _get_data_from_grid tr = grid[field] File "/u/stonnes/Installs/yt-x86_64/src/yt-hg/yt/data_objects/grid_patch.py", line 157, in __getitem__ self.get_data(key) File "/u/stonnes/Installs/yt-x86_64/src/yt-hg/yt/data_objects/grid_patch.py", line 200, in get_data self._generate_field(field) File "/u/stonnes/Installs/yt-x86_64/src/yt-hg/yt/data_objects/grid_patch.py", line 145, in _generate_field self[field] = self.pf.field_info[field](self) File "/u/stonnes/Installs/yt-x86_64/src/yt-hg/yt/data_objects/field_info_container.py", line 378, in __call__ dd = self._function(self, data) File "/u/stonnes/Installs/yt-x86_64/src/yt-hg/yt/data_objects/universal_fields.py", line 94, in _coordX return (np.ones(data.ActiveDimensions, dtype='float64') File "/u/stonnes/Installs/yt-x86_64/lib/python2.7/site-packages/numpy/core/numeric.py", line 1819, in ones a = empty(shape, dtype, order) MemoryError On Tue, Dec 18, 2012 at 1:49 PM, John ZuHone <jzuhone@gmail.com> wrote:
Hi Stephanie,
I doubt that 25 GB is too big.
Can you also try a weighted average quantity of specific_scalar[0] itself?
When you use only id0 and id1, you are potentially also changing domain_right_edge, since for Athena yt has to go through all of the id*/* to find the largest right edge for any grid to determine domain_right_edge.
Can you figure out what pf.domain_right_edge and pf.domain_dimensions are for the two different cases (where you have all the data and also where you have just id0 and id1)?
I'm not sure at the moment what that would tell us, but I'm curious about it.
As another possible way to get some clues, I am aware that Athena has a tool that joins the vtk files into one. Is it possible to run that and then try the same script on the merged file?
I'm also wondering if Matt and Sam have any thoughts on this.
Best,
John
On Dec 18, 2012, at 1:21 PM, Stephanie Tonnesen <stonnes@gmail.com> wrote:
Hi John,
So, what I mean by "expected" variables are basically all variables BUT specific_scalar[0]. When I read in the vtk file, you can see that the bottom line says
yt : [INFO ] 2012-12-13 23:27:13,214 Adding unknown field specific_scalar[0] to list of fields
Yes, I can do any of the "quantities" commands, for example:
alld = pf.h.all_data() tracerp25 = alld.cut_region(["grid['specific_scalar[0]'] > 0.25", "grid['z'] > 0.01"])
radmean = tracerp25.quantities["WeightedAverageQuantity"]("Radius","Ones")
Slicing also works:
p = SlicePlot(pf,"x","specific_scalar[0]")
Now I am starting to worry that it is because output is so big--each vtk file is 72 MB, so a full output is almost 25 GB. When I just use two outputs (moving them so yt only sees id0 and id1 folders) the phase plot works. But, after running the add_phase_object command on the whole data set for more than 1.5 hours I still have no result and my computer keeps freezing--although top shows that only 42% of memory is being used.
Is it possible that my output is just too big?
Stephanie
On Tue, Dec 18, 2012 at 11:00 AM, John Zuhone <jzuhone@gmail.com> wrote:
Hi Stephanie,
Ok, thanks for checking that.
I guess I'm not sure what you mean by the "expected" set of variables. Do you mean the ones that yt defines, or the ones that are in the field list? It looks like "velocity_z" works for one call and doesn't for another.
Are you able to do anything else with "specific_scalar[0]"? Like make a slice, or a projection of some sort? Or maybe take the weighted average of it in a sphere. I'd be interested in seeing if the problem was restricted to phase plotting.
How big is the dataset, by the way? Is there something that fails in the same way that the both of us can test on?
Best,
John
On Tue, Dec 18, 2012 at 10:53 AM, Stephanie Tonnesen <stonnes@gmail.com>wrote:
Hi,
Yup. Just to be sure I updated again and reattempted to make the phase_object and python was Killed again.
Thanks!
Stephanie
On Tue, Dec 18, 2012 at 12:35 AM, John ZuHone <jzuhone@gmail.com> wrote:
Hi Stephanie,
Just want to make sure that you're using the latest version of the yt development tree--it contains some important Athena fixes that were put in a few weeks back.
If so, then we'll work from there to figure out what's gone wrong.
Best,
John
On Dec 17, 2012, at 5:44 PM, Stephanie Tonnesen <stonnes@gmail.com> wrote:
Hi yt-users,
I am trying to make phase_objects of Athena data. I am able to make phase_objects if I use the "expected" set of variables (will show below), but kill my kernel or am kicked out of python if I try and use an "unknown field" read in from a set of vtk files.
from yt.mods import *
pf = load("id0/rps.0080.vtk")
yt : [INFO ] 2012-12-13 23:27:08,678 Temporarily setting domain_right_edge = -domain_left_edge. This will be corrected automatically if it is not the case. yt : [INFO ] 2012-12-13 23:27:08,974 Parameters: current_time = 40.00096 yt : [INFO ] 2012-12-13 23:27:08,974 Parameters: domain_dimensions = [1048 1048 655] yt : [INFO ] 2012-12-13 23:27:08,975 Parameters: domain_left_edge = [-3.2 -3.2 -2. ] yt : [INFO ] 2012-12-13 23:27:08,976 Parameters: domain_right_edge = [ 3.2 3.2 2. ] yt : [INFO ] 2012-12-13 23:27:08,977 Parameters: cosmological_simulation = 0.0 yt : [INFO ] 2012-12-13 23:27:13,214 Adding unknown field specific_scalar[0] to list of fields
pf.h.field_list
['density', 'pressure', 'velocity_z', 'velocity_x', 'velocity_y', 'specific_scalar[0]']
alld = pf.h.all_data() tracerp25 = alld.cut_region(["grid['specific_scalar[0]'] > 0.25", "grid['z'] > 0.01"])
##These commands works:
radmean = tracerp25.quantities["WeightedAverageQuantity"]("Radius","Ones")
pc = PlotCollectionInteractive(pf)
pc.add_phase_object(alld,["z","velocity_z","CellMass"],weight=None)
#The calls below fail. There is no error message. If I am in iPython, #the kernel dies. If I am running on a computer, it freezes and #eventually I just get the message #Killed
pc.add_phase_object(alld,["z","specific_scalar[0]","CellMass"],weight=None) pc.add_phase_object(tracerp25,["z","velocity_z","CellMass"],weight=None)
#If I read in a single vtk file ("id122/rps-id122.0080.vtk") the above #commands work
Any help would be much appreciated! Thanks,
Stephanie _______________________________________________ yt-users mailing list yt-users@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
_______________________________________________ yt-users mailing list yt-users@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
_______________________________________________ yt-users mailing list yt-users@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
-- John ZuHone
Postdoctoral Researcher NASA/Goddard Space Flight Center
jzuhone@gmail.com john.zuhone@nasa.gov
_______________________________________________ yt-users mailing list yt-users@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
_______________________________________________ yt-users mailing list yt-users@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
_______________________________________________ yt-users mailing list yt-users@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org

Hi Stephanie, Well, this particular thing is definitely a memory issue. It looks like it failed when trying to open the x-coordinate dataset for all_data(). By itself, this is about 7 GB, given the domain dimensions that you gave. By the way, do those look right (1048,1048,950)? By the way, is this uniform grid or does this have SMR? I don't think that we've verified that SMR works. I'm sorry I'm not able to be more definitive yet--let me think on it a bit more. Best, John On Dec 18, 2012, at 4:27 PM, Stephanie Tonnesen <stonnes@gmail.com> wrote:
Hi John,
The weighted average quantity call of specific_scalar[0] worked.
The domain info:
pf.domain_right_edge array([ 3.2, 3.2, 3.8])
pf.domain_dimensions array([1048, 1048, 950])
And then using id0 and id1:
pf2.domain_right_edge array([-1.349618, -2.259542, -1.145263])
pf2.domain_dimensions array([303, 154, 140])
When I combined the vtk files into a single 25 GB file, I get an error when creating the PlotCollection:
pc = PlotCollection(pf) Traceback (most recent call last): File "<stdin>", line 1, in <module> File "/u/stonnes/Installs/yt-x86_64/src/yt-hg/yt/visualization/plot_collection.py", line 121, in __init__ v,self.c = pf.h.find_max("Density") # @todo: ensure no caching File "/u/stonnes/Installs/yt-x86_64/src/yt-hg/yt/data_objects/object_finding_mixin.py", line 61, in find_max mg, mc, mv, pos = self.find_max_cell_location(field, finest_levels) File "/u/stonnes/Installs/yt-x86_64/src/yt-hg/yt/data_objects/object_finding_mixin.py", line 74, in find_max_cell_location source.quantities["MaxLocation"]( field, lazy_reader=True) File "/u/stonnes/Installs/yt-x86_64/src/yt-hg/yt/data_objects/derived_quantities.py", line 92, in __call__ return self._call_func_lazy(args, kwargs) File "/u/stonnes/Installs/yt-x86_64/src/yt-hg/yt/data_objects/derived_quantities.py", line 99, in _call_func_lazy rv = self.func(GridChildMaskWrapper(g, self._data_source), *args, **kwargs) File "/u/stonnes/Installs/yt-x86_64/src/yt-hg/yt/data_objects/derived_quantities.py", line 649, in _MaxLocation mx, my, mz = [data[ax][maxi] for ax in 'xyz'] File "/u/stonnes/Installs/yt-x86_64/src/yt-hg/yt/data_objects/derived_quantities.py", line 60, in __getitem__ data = self.data_source._get_data_from_grid(self.grid, item) File "/u/stonnes/Installs/yt-x86_64/src/yt-hg/yt/data_objects/data_containers.py", line 93, in save_state tr = func(self, grid, field, *args, **kwargs) File "/u/stonnes/Installs/yt-x86_64/src/yt-hg/yt/data_objects/data_containers.py", line 2630, in _get_data_from_grid tr = grid[field] File "/u/stonnes/Installs/yt-x86_64/src/yt-hg/yt/data_objects/grid_patch.py", line 157, in __getitem__ self.get_data(key) File "/u/stonnes/Installs/yt-x86_64/src/yt-hg/yt/data_objects/grid_patch.py", line 200, in get_data self._generate_field(field) File "/u/stonnes/Installs/yt-x86_64/src/yt-hg/yt/data_objects/grid_patch.py", line 145, in _generate_field self[field] = self.pf.field_info[field](self) File "/u/stonnes/Installs/yt-x86_64/src/yt-hg/yt/data_objects/field_info_container.py", line 378, in __call__ dd = self._function(self, data) File "/u/stonnes/Installs/yt-x86_64/src/yt-hg/yt/data_objects/universal_fields.py", line 94, in _coordX return (np.ones(data.ActiveDimensions, dtype='float64') File "/u/stonnes/Installs/yt-x86_64/lib/python2.7/site-packages/numpy/core/numeric.py", line 1819, in ones a = empty(shape, dtype, order) MemoryError
On Tue, Dec 18, 2012 at 1:49 PM, John ZuHone <jzuhone@gmail.com> wrote: Hi Stephanie,
I doubt that 25 GB is too big.
Can you also try a weighted average quantity of specific_scalar[0] itself?
When you use only id0 and id1, you are potentially also changing domain_right_edge, since for Athena yt has to go through all of the id*/* to find the largest right edge for any grid to determine domain_right_edge.
Can you figure out what pf.domain_right_edge and pf.domain_dimensions are for the two different cases (where you have all the data and also where you have just id0 and id1)?
I'm not sure at the moment what that would tell us, but I'm curious about it.
As another possible way to get some clues, I am aware that Athena has a tool that joins the vtk files into one. Is it possible to run that and then try the same script on the merged file?
I'm also wondering if Matt and Sam have any thoughts on this.
Best,
John
On Dec 18, 2012, at 1:21 PM, Stephanie Tonnesen <stonnes@gmail.com> wrote:
Hi John,
So, what I mean by "expected" variables are basically all variables BUT specific_scalar[0]. When I read in the vtk file, you can see that the bottom line says
yt : [INFO ] 2012-12-13 23:27:13,214 Adding unknown field specific_scalar[0] to list of fields
Yes, I can do any of the "quantities" commands, for example:
alld = pf.h.all_data() tracerp25 = alld.cut_region(["grid['specific_scalar[0]'] > 0.25", "grid['z'] > 0.01"]) radmean = tracerp25.quantities["WeightedAverageQuantity"]("Radius","Ones")
Slicing also works:
p = SlicePlot(pf,"x","specific_scalar[0]")
Now I am starting to worry that it is because output is so big--each vtk file is 72 MB, so a full output is almost 25 GB. When I just use two outputs (moving them so yt only sees id0 and id1 folders) the phase plot works. But, after running the add_phase_object command on the whole data set for more than 1.5 hours I still have no result and my computer keeps freezing--although top shows that only 42% of memory is being used.
Is it possible that my output is just too big?
Stephanie
On Tue, Dec 18, 2012 at 11:00 AM, John Zuhone <jzuhone@gmail.com> wrote: Hi Stephanie,
Ok, thanks for checking that.
I guess I'm not sure what you mean by the "expected" set of variables. Do you mean the ones that yt defines, or the ones that are in the field list? It looks like "velocity_z" works for one call and doesn't for another.
Are you able to do anything else with "specific_scalar[0]"? Like make a slice, or a projection of some sort? Or maybe take the weighted average of it in a sphere. I'd be interested in seeing if the problem was restricted to phase plotting.
How big is the dataset, by the way? Is there something that fails in the same way that the both of us can test on?
Best,
John
On Tue, Dec 18, 2012 at 10:53 AM, Stephanie Tonnesen <stonnes@gmail.com> wrote:
Hi,
Yup. Just to be sure I updated again and reattempted to make the phase_object and python was Killed again.
Thanks!
Stephanie
On Tue, Dec 18, 2012 at 12:35 AM, John ZuHone <jzuhone@gmail.com> wrote: Hi Stephanie,
Just want to make sure that you're using the latest version of the yt development tree--it contains some important Athena fixes that were put in a few weeks back.
If so, then we'll work from there to figure out what's gone wrong.
Best,
John
On Dec 17, 2012, at 5:44 PM, Stephanie Tonnesen <stonnes@gmail.com> wrote:
Hi yt-users,
I am trying to make phase_objects of Athena data. I am able to make phase_objects if I use the "expected" set of variables (will show below), but kill my kernel or am kicked out of python if I try and use an "unknown field" read in from a set of vtk files.
from yt.mods import *
pf = load("id0/rps.0080.vtk")
yt : [INFO ] 2012-12-13 23:27:08,678 Temporarily setting domain_right_edge = -domain_left_edge. This will be corrected automatically if it is not the case. yt : [INFO ] 2012-12-13 23:27:08,974 Parameters: current_time = 40.00096 yt : [INFO ] 2012-12-13 23:27:08,974 Parameters: domain_dimensions = [1048 1048 655] yt : [INFO ] 2012-12-13 23:27:08,975 Parameters: domain_left_edge = [-3.2 -3.2 -2. ] yt : [INFO ] 2012-12-13 23:27:08,976 Parameters: domain_right_edge = [ 3.2 3.2 2. ] yt : [INFO ] 2012-12-13 23:27:08,977 Parameters: cosmological_simulation = 0.0 yt : [INFO ] 2012-12-13 23:27:13,214 Adding unknown field specific_scalar[0] to list of fields
pf.h.field_list
['density', 'pressure', 'velocity_z', 'velocity_x', 'velocity_y', 'specific_scalar[0]'] alld = pf.h.all_data() tracerp25 = alld.cut_region(["grid['specific_scalar[0]'] > 0.25", "grid['z'] > 0.01"])
##These commands works: radmean = tracerp25.quantities["WeightedAverageQuantity"]("Radius","Ones")
pc = PlotCollectionInteractive(pf)
pc.add_phase_object(alld,["z","velocity_z","CellMass"],weight=None)
#The calls below fail. There is no error message. If I am in iPython, #the kernel dies. If I am running on a computer, it freezes and #eventually I just get the message #Killed
pc.add_phase_object(alld,["z","specific_scalar[0]","CellMass"],weight=None) pc.add_phase_object(tracerp25,["z","velocity_z","CellMass"],weight=None)
#If I read in a single vtk file ("id122/rps-id122.0080.vtk") the above #commands work Any help would be much appreciated! Thanks,
Stephanie _______________________________________________ yt-users mailing list yt-users@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
_______________________________________________ yt-users mailing list yt-users@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
_______________________________________________ yt-users mailing list yt-users@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
-- John ZuHone
Postdoctoral Researcher NASA/Goddard Space Flight Center
jzuhone@gmail.com john.zuhone@nasa.gov
_______________________________________________ yt-users mailing list yt-users@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
_______________________________________________ yt-users mailing list yt-users@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
_______________________________________________ yt-users mailing list yt-users@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
_______________________________________________ yt-users mailing list yt-users@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org

Hi John, Yup, definitely a memory issue, and yes those dimensions are right. It is a uniform grid--I think you are right that yt and SMR don't mix yet. Thanks for working on this problem with me! Stephanie On Tue, Dec 18, 2012 at 4:41 PM, John ZuHone <jzuhone@gmail.com> wrote:
Hi Stephanie,
Well, this particular thing is definitely a memory issue. It looks like it failed when trying to open the x-coordinate dataset for all_data(). By itself, this is about 7 GB, given the domain dimensions that you gave. By the way, do those look right (1048,1048,950)?
By the way, is this uniform grid or does this have SMR? I don't think that we've verified that SMR works.
I'm sorry I'm not able to be more definitive yet--let me think on it a bit more.
Best,
John
On Dec 18, 2012, at 4:27 PM, Stephanie Tonnesen <stonnes@gmail.com> wrote:
Hi John,
The weighted average quantity call of specific_scalar[0] worked.
The domain info:
pf.domain_right_edge array([ 3.2, 3.2, 3.8])
pf.domain_dimensions array([1048, 1048, 950])
And then using id0 and id1:
pf2.domain_right_edge array([-1.349618, -2.259542, -1.145263])
pf2.domain_dimensions array([303, 154, 140])
When I combined the vtk files into a single 25 GB file, I get an error when creating the PlotCollection:
pc = PlotCollection(pf) Traceback (most recent call last): File "<stdin>", line 1, in <module> File "/u/stonnes/Installs/yt-x86_64/src/yt-hg/yt/visualization/plot_collection.py", line 121, in __init__ v,self.c = pf.h.find_max("Density") # @todo: ensure no caching File "/u/stonnes/Installs/yt-x86_64/src/yt-hg/yt/data_objects/object_finding_mixin.py", line 61, in find_max mg, mc, mv, pos = self.find_max_cell_location(field, finest_levels) File "/u/stonnes/Installs/yt-x86_64/src/yt-hg/yt/data_objects/object_finding_mixin.py", line 74, in find_max_cell_location source.quantities["MaxLocation"]( field, lazy_reader=True) File "/u/stonnes/Installs/yt-x86_64/src/yt-hg/yt/data_objects/derived_quantities.py", line 92, in __call__ return self._call_func_lazy(args, kwargs) File "/u/stonnes/Installs/yt-x86_64/src/yt-hg/yt/data_objects/derived_quantities.py", line 99, in _call_func_lazy rv = self.func(GridChildMaskWrapper(g, self._data_source), *args, **kwargs) File "/u/stonnes/Installs/yt-x86_64/src/yt-hg/yt/data_objects/derived_quantities.py", line 649, in _MaxLocation mx, my, mz = [data[ax][maxi] for ax in 'xyz'] File "/u/stonnes/Installs/yt-x86_64/src/yt-hg/yt/data_objects/derived_quantities.py", line 60, in __getitem__ data = self.data_source._get_data_from_grid(self.grid, item) File "/u/stonnes/Installs/yt-x86_64/src/yt-hg/yt/data_objects/data_containers.py", line 93, in save_state tr = func(self, grid, field, *args, **kwargs) File "/u/stonnes/Installs/yt-x86_64/src/yt-hg/yt/data_objects/data_containers.py", line 2630, in _get_data_from_grid tr = grid[field] File "/u/stonnes/Installs/yt-x86_64/src/yt-hg/yt/data_objects/grid_patch.py", line 157, in __getitem__ self.get_data(key) File "/u/stonnes/Installs/yt-x86_64/src/yt-hg/yt/data_objects/grid_patch.py", line 200, in get_data self._generate_field(field) File "/u/stonnes/Installs/yt-x86_64/src/yt-hg/yt/data_objects/grid_patch.py", line 145, in _generate_field self[field] = self.pf.field_info[field](self) File "/u/stonnes/Installs/yt-x86_64/src/yt-hg/yt/data_objects/field_info_container.py", line 378, in __call__ dd = self._function(self, data) File "/u/stonnes/Installs/yt-x86_64/src/yt-hg/yt/data_objects/universal_fields.py", line 94, in _coordX return (np.ones(data.ActiveDimensions, dtype='float64') File "/u/stonnes/Installs/yt-x86_64/lib/python2.7/site-packages/numpy/core/numeric.py", line 1819, in ones a = empty(shape, dtype, order) MemoryError
On Tue, Dec 18, 2012 at 1:49 PM, John ZuHone <jzuhone@gmail.com> wrote:
Hi Stephanie,
I doubt that 25 GB is too big.
Can you also try a weighted average quantity of specific_scalar[0] itself?
When you use only id0 and id1, you are potentially also changing domain_right_edge, since for Athena yt has to go through all of the id*/* to find the largest right edge for any grid to determine domain_right_edge.
Can you figure out what pf.domain_right_edge and pf.domain_dimensions are for the two different cases (where you have all the data and also where you have just id0 and id1)?
I'm not sure at the moment what that would tell us, but I'm curious about it.
As another possible way to get some clues, I am aware that Athena has a tool that joins the vtk files into one. Is it possible to run that and then try the same script on the merged file?
I'm also wondering if Matt and Sam have any thoughts on this.
Best,
John
On Dec 18, 2012, at 1:21 PM, Stephanie Tonnesen <stonnes@gmail.com> wrote:
Hi John,
So, what I mean by "expected" variables are basically all variables BUT specific_scalar[0]. When I read in the vtk file, you can see that the bottom line says
yt : [INFO ] 2012-12-13 23:27:13,214 Adding unknown field specific_scalar[0] to list of fields
Yes, I can do any of the "quantities" commands, for example:
alld = pf.h.all_data() tracerp25 = alld.cut_region(["grid['specific_scalar[0]'] > 0.25", "grid['z'] > 0.01"])
radmean = tracerp25.quantities["WeightedAverageQuantity"]("Radius","Ones")
Slicing also works:
p = SlicePlot(pf,"x","specific_scalar[0]")
Now I am starting to worry that it is because output is so big--each vtk file is 72 MB, so a full output is almost 25 GB. When I just use two outputs (moving them so yt only sees id0 and id1 folders) the phase plot works. But, after running the add_phase_object command on the whole data set for more than 1.5 hours I still have no result and my computer keeps freezing--although top shows that only 42% of memory is being used.
Is it possible that my output is just too big?
Stephanie
On Tue, Dec 18, 2012 at 11:00 AM, John Zuhone <jzuhone@gmail.com> wrote:
Hi Stephanie,
Ok, thanks for checking that.
I guess I'm not sure what you mean by the "expected" set of variables. Do you mean the ones that yt defines, or the ones that are in the field list? It looks like "velocity_z" works for one call and doesn't for another.
Are you able to do anything else with "specific_scalar[0]"? Like make a slice, or a projection of some sort? Or maybe take the weighted average of it in a sphere. I'd be interested in seeing if the problem was restricted to phase plotting.
How big is the dataset, by the way? Is there something that fails in the same way that the both of us can test on?
Best,
John
On Tue, Dec 18, 2012 at 10:53 AM, Stephanie Tonnesen <stonnes@gmail.com>wrote:
Hi,
Yup. Just to be sure I updated again and reattempted to make the phase_object and python was Killed again.
Thanks!
Stephanie
On Tue, Dec 18, 2012 at 12:35 AM, John ZuHone <jzuhone@gmail.com>wrote:
Hi Stephanie,
Just want to make sure that you're using the latest version of the yt development tree--it contains some important Athena fixes that were put in a few weeks back.
If so, then we'll work from there to figure out what's gone wrong.
Best,
John
On Dec 17, 2012, at 5:44 PM, Stephanie Tonnesen <stonnes@gmail.com> wrote:
Hi yt-users,
I am trying to make phase_objects of Athena data. I am able to make phase_objects if I use the "expected" set of variables (will show below), but kill my kernel or am kicked out of python if I try and use an "unknown field" read in from a set of vtk files.
from yt.mods import *
pf = load("id0/rps.0080.vtk")
yt : [INFO ] 2012-12-13 23:27:08,678 Temporarily setting domain_right_edge = -domain_left_edge. This will be corrected automatically if it is not the case. yt : [INFO ] 2012-12-13 23:27:08,974 Parameters: current_time = 40.00096 yt : [INFO ] 2012-12-13 23:27:08,974 Parameters: domain_dimensions = [1048 1048 655] yt : [INFO ] 2012-12-13 23:27:08,975 Parameters: domain_left_edge = [-3.2 -3.2 -2. ] yt : [INFO ] 2012-12-13 23:27:08,976 Parameters: domain_right_edge = [ 3.2 3.2 2. ] yt : [INFO ] 2012-12-13 23:27:08,977 Parameters: cosmological_simulation = 0.0 yt : [INFO ] 2012-12-13 23:27:13,214 Adding unknown field specific_scalar[0] to list of fields
pf.h.field_list
['density', 'pressure', 'velocity_z', 'velocity_x', 'velocity_y', 'specific_scalar[0]']
alld = pf.h.all_data() tracerp25 = alld.cut_region(["grid['specific_scalar[0]'] > 0.25", "grid['z'] > 0.01"])
##These commands works:
radmean = tracerp25.quantities["WeightedAverageQuantity"]("Radius","Ones")
pc = PlotCollectionInteractive(pf)
pc.add_phase_object(alld,["z","velocity_z","CellMass"],weight=None)
#The calls below fail. There is no error message. If I am in iPython, #the kernel dies. If I am running on a computer, it freezes and #eventually I just get the message #Killed
pc.add_phase_object(alld,["z","specific_scalar[0]","CellMass"],weight=None) pc.add_phase_object(tracerp25,["z","velocity_z","CellMass"],weight=None)
#If I read in a single vtk file ("id122/rps-id122.0080.vtk") the above #commands work
Any help would be much appreciated! Thanks,
Stephanie _______________________________________________ yt-users mailing list yt-users@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
_______________________________________________ yt-users mailing list yt-users@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
_______________________________________________ yt-users mailing list yt-users@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
-- John ZuHone
Postdoctoral Researcher NASA/Goddard Space Flight Center
jzuhone@gmail.com john.zuhone@nasa.gov
_______________________________________________ yt-users mailing list yt-users@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
_______________________________________________ yt-users mailing list yt-users@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
_______________________________________________ yt-users mailing list yt-users@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
_______________________________________________ yt-users mailing list yt-users@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
_______________________________________________ yt-users mailing list yt-users@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org

Hi again, Okay, I still don't know what is going on, but have run a couple tests to try and narrow it down. So, I was trying to focus on the fact that using all 343 grids, alld = pf.h.all_data() tracerp25 = alld.cut_region(["grid['specific_scalar[0]'] > 0.25"]) pc.add_phase_object(tracerp25,["z","velocity_z","CellMass"],weight=None) fails. But using a single grid or 2 grids I *can* add that phase_object. So I checked to see if it worked if I used the upper 245 grids (removed the two 7x7 x-y planes at the lowest z values). Did NOT work. But if I ALSO removed the top 7x7 x-y plane, it works (196 grids). If I only remove the top 7x7 x-y plane, it does NOT work. If I remove the top and bottom x-y planes, it also does NOT work. However, an alld.cut_region(["grid['density'] > 10."]) is always able to make a phase_object. Is this the clue that makes is all make sense to anyone? Stephanie On Tue, Dec 18, 2012 at 9:36 PM, Stephanie Tonnesen <stonnes@gmail.com>wrote:
Hi John,
Yup, definitely a memory issue, and yes those dimensions are right. It is a uniform grid--I think you are right that yt and SMR don't mix yet.
Thanks for working on this problem with me!
Stephanie
On Tue, Dec 18, 2012 at 4:41 PM, John ZuHone <jzuhone@gmail.com> wrote:
Hi Stephanie,
Well, this particular thing is definitely a memory issue. It looks like it failed when trying to open the x-coordinate dataset for all_data(). By itself, this is about 7 GB, given the domain dimensions that you gave. By the way, do those look right (1048,1048,950)?
By the way, is this uniform grid or does this have SMR? I don't think that we've verified that SMR works.
I'm sorry I'm not able to be more definitive yet--let me think on it a bit more.
Best,
John
On Dec 18, 2012, at 4:27 PM, Stephanie Tonnesen <stonnes@gmail.com> wrote:
Hi John,
The weighted average quantity call of specific_scalar[0] worked.
The domain info:
pf.domain_right_edge array([ 3.2, 3.2, 3.8])
pf.domain_dimensions array([1048, 1048, 950])
And then using id0 and id1:
pf2.domain_right_edge array([-1.349618, -2.259542, -1.145263])
pf2.domain_dimensions array([303, 154, 140])
When I combined the vtk files into a single 25 GB file, I get an error when creating the PlotCollection:
pc = PlotCollection(pf) Traceback (most recent call last): File "<stdin>", line 1, in <module> File "/u/stonnes/Installs/yt-x86_64/src/yt-hg/yt/visualization/plot_collection.py", line 121, in __init__ v,self.c = pf.h.find_max("Density") # @todo: ensure no caching File "/u/stonnes/Installs/yt-x86_64/src/yt-hg/yt/data_objects/object_finding_mixin.py", line 61, in find_max mg, mc, mv, pos = self.find_max_cell_location(field, finest_levels) File "/u/stonnes/Installs/yt-x86_64/src/yt-hg/yt/data_objects/object_finding_mixin.py", line 74, in find_max_cell_location source.quantities["MaxLocation"]( field, lazy_reader=True) File "/u/stonnes/Installs/yt-x86_64/src/yt-hg/yt/data_objects/derived_quantities.py", line 92, in __call__ return self._call_func_lazy(args, kwargs) File "/u/stonnes/Installs/yt-x86_64/src/yt-hg/yt/data_objects/derived_quantities.py", line 99, in _call_func_lazy rv = self.func(GridChildMaskWrapper(g, self._data_source), *args, **kwargs) File "/u/stonnes/Installs/yt-x86_64/src/yt-hg/yt/data_objects/derived_quantities.py", line 649, in _MaxLocation mx, my, mz = [data[ax][maxi] for ax in 'xyz'] File "/u/stonnes/Installs/yt-x86_64/src/yt-hg/yt/data_objects/derived_quantities.py", line 60, in __getitem__ data = self.data_source._get_data_from_grid(self.grid, item) File "/u/stonnes/Installs/yt-x86_64/src/yt-hg/yt/data_objects/data_containers.py", line 93, in save_state tr = func(self, grid, field, *args, **kwargs) File "/u/stonnes/Installs/yt-x86_64/src/yt-hg/yt/data_objects/data_containers.py", line 2630, in _get_data_from_grid tr = grid[field] File "/u/stonnes/Installs/yt-x86_64/src/yt-hg/yt/data_objects/grid_patch.py", line 157, in __getitem__ self.get_data(key) File "/u/stonnes/Installs/yt-x86_64/src/yt-hg/yt/data_objects/grid_patch.py", line 200, in get_data self._generate_field(field) File "/u/stonnes/Installs/yt-x86_64/src/yt-hg/yt/data_objects/grid_patch.py", line 145, in _generate_field self[field] = self.pf.field_info[field](self) File "/u/stonnes/Installs/yt-x86_64/src/yt-hg/yt/data_objects/field_info_container.py", line 378, in __call__ dd = self._function(self, data) File "/u/stonnes/Installs/yt-x86_64/src/yt-hg/yt/data_objects/universal_fields.py", line 94, in _coordX return (np.ones(data.ActiveDimensions, dtype='float64') File "/u/stonnes/Installs/yt-x86_64/lib/python2.7/site-packages/numpy/core/numeric.py", line 1819, in ones a = empty(shape, dtype, order) MemoryError
On Tue, Dec 18, 2012 at 1:49 PM, John ZuHone <jzuhone@gmail.com> wrote:
Hi Stephanie,
I doubt that 25 GB is too big.
Can you also try a weighted average quantity of specific_scalar[0] itself?
When you use only id0 and id1, you are potentially also changing domain_right_edge, since for Athena yt has to go through all of the id*/* to find the largest right edge for any grid to determine domain_right_edge.
Can you figure out what pf.domain_right_edge and pf.domain_dimensions are for the two different cases (where you have all the data and also where you have just id0 and id1)?
I'm not sure at the moment what that would tell us, but I'm curious about it.
As another possible way to get some clues, I am aware that Athena has a tool that joins the vtk files into one. Is it possible to run that and then try the same script on the merged file?
I'm also wondering if Matt and Sam have any thoughts on this.
Best,
John
On Dec 18, 2012, at 1:21 PM, Stephanie Tonnesen <stonnes@gmail.com> wrote:
Hi John,
So, what I mean by "expected" variables are basically all variables BUT specific_scalar[0]. When I read in the vtk file, you can see that the bottom line says
yt : [INFO ] 2012-12-13 23:27:13,214 Adding unknown field specific_scalar[0] to list of fields
Yes, I can do any of the "quantities" commands, for example:
alld = pf.h.all_data() tracerp25 = alld.cut_region(["grid['specific_scalar[0]'] > 0.25", "grid['z'] > 0.01"])
radmean = tracerp25.quantities["WeightedAverageQuantity"]("Radius","Ones")
Slicing also works:
p = SlicePlot(pf,"x","specific_scalar[0]")
Now I am starting to worry that it is because output is so big--each vtk file is 72 MB, so a full output is almost 25 GB. When I just use two outputs (moving them so yt only sees id0 and id1 folders) the phase plot works. But, after running the add_phase_object command on the whole data set for more than 1.5 hours I still have no result and my computer keeps freezing--although top shows that only 42% of memory is being used.
Is it possible that my output is just too big?
Stephanie
On Tue, Dec 18, 2012 at 11:00 AM, John Zuhone <jzuhone@gmail.com> wrote:
Hi Stephanie,
Ok, thanks for checking that.
I guess I'm not sure what you mean by the "expected" set of variables. Do you mean the ones that yt defines, or the ones that are in the field list? It looks like "velocity_z" works for one call and doesn't for another.
Are you able to do anything else with "specific_scalar[0]"? Like make a slice, or a projection of some sort? Or maybe take the weighted average of it in a sphere. I'd be interested in seeing if the problem was restricted to phase plotting.
How big is the dataset, by the way? Is there something that fails in the same way that the both of us can test on?
Best,
John
On Tue, Dec 18, 2012 at 10:53 AM, Stephanie Tonnesen <stonnes@gmail.com
wrote:
Hi,
Yup. Just to be sure I updated again and reattempted to make the phase_object and python was Killed again.
Thanks!
Stephanie
On Tue, Dec 18, 2012 at 12:35 AM, John ZuHone <jzuhone@gmail.com>wrote:
Hi Stephanie,
Just want to make sure that you're using the latest version of the yt development tree--it contains some important Athena fixes that were put in a few weeks back.
If so, then we'll work from there to figure out what's gone wrong.
Best,
John
On Dec 17, 2012, at 5:44 PM, Stephanie Tonnesen <stonnes@gmail.com> wrote:
Hi yt-users,
I am trying to make phase_objects of Athena data. I am able to make phase_objects if I use the "expected" set of variables (will show below), but kill my kernel or am kicked out of python if I try and use an "unknown field" read in from a set of vtk files.
from yt.mods import *
pf = load("id0/rps.0080.vtk")
yt : [INFO ] 2012-12-13 23:27:08,678 Temporarily setting domain_right_edge = -domain_left_edge. This will be corrected automatically if it is not the case. yt : [INFO ] 2012-12-13 23:27:08,974 Parameters: current_time = 40.00096 yt : [INFO ] 2012-12-13 23:27:08,974 Parameters: domain_dimensions = [1048 1048 655] yt : [INFO ] 2012-12-13 23:27:08,975 Parameters: domain_left_edge = [-3.2 -3.2 -2. ] yt : [INFO ] 2012-12-13 23:27:08,976 Parameters: domain_right_edge = [ 3.2 3.2 2. ] yt : [INFO ] 2012-12-13 23:27:08,977 Parameters: cosmological_simulation = 0.0 yt : [INFO ] 2012-12-13 23:27:13,214 Adding unknown field specific_scalar[0] to list of fields
pf.h.field_list
['density', 'pressure', 'velocity_z', 'velocity_x', 'velocity_y', 'specific_scalar[0]']
alld = pf.h.all_data() tracerp25 = alld.cut_region(["grid['specific_scalar[0]'] > 0.25", "grid['z'] > 0.01"])
##These commands works:
radmean = tracerp25.quantities["WeightedAverageQuantity"]("Radius","Ones")
pc = PlotCollectionInteractive(pf)
pc.add_phase_object(alld,["z","velocity_z","CellMass"],weight=None)
#The calls below fail. There is no error message. If I am in iPython, #the kernel dies. If I am running on a computer, it freezes and #eventually I just get the message #Killed
pc.add_phase_object(alld,["z","specific_scalar[0]","CellMass"],weight=None) pc.add_phase_object(tracerp25,["z","velocity_z","CellMass"],weight=None)
#If I read in a single vtk file ("id122/rps-id122.0080.vtk") the above #commands work
Any help would be much appreciated! Thanks,
Stephanie _______________________________________________ yt-users mailing list yt-users@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
_______________________________________________ yt-users mailing list yt-users@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
_______________________________________________ yt-users mailing list yt-users@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
-- John ZuHone
Postdoctoral Researcher NASA/Goddard Space Flight Center
jzuhone@gmail.com john.zuhone@nasa.gov
_______________________________________________ yt-users mailing list yt-users@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
_______________________________________________ yt-users mailing list yt-users@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
_______________________________________________ yt-users mailing list yt-users@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
_______________________________________________ yt-users mailing list yt-users@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
_______________________________________________ yt-users mailing list yt-users@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
participants (3)
-
John Zuhone
-
John ZuHone
-
Stephanie Tonnesen