Everybody good afternoon, about Fortran kD Tree Setup, just to keep thread with my fix. 1) I downloaded the YT-DEV version and it didn't fix my problem. However, when I downloaded Forthon-0.8.7.tgz it did the trick. 2) Now I want to talk about another issue that I am getting. This time is with clumps. I am using an enzo data, with the code example in the YT documentation. this is my code: ----- import sys from yt.mods import * # set up our namespace from yt.analysis_modules.level_sets.api import * import yt.frontends.enzo.api yt.frontends.enzo.api.EnzoHierarchy._bn = "%s.grid.cpu%%04i" fn = "/astro/data/redshift0058/redshift0058" # parameter file to load field = "Density" step = 10.0 pf = load(fn) # load data c = [0.403290331 , 0.471765608 , 0.461319029] r = [0.411690331 , 0.480165608 , 0.469719029] l = [0.394890331 , 0.463365608 , 0.452919029] data_source = pf.h.region(c,l,r) c_min = 10**na.floor(na.log10(data_source[field]).min() ) c_max = 10**na.floor(na.log10(data_source[field]).max()+1) master_clump = Clump(data_source, None, field) find_clumps(master_clump, c_min, c_max, step) f = open('%s_clump_hierarchy.txt' % pf,'w') write_clump_hierarchy(master_clump, 0, f) f.close() f = open('%s_clumps.txt' % pf,'w') write_clumps(master_clump,0,f) f.close() --------- and I got the follow error. Traceback (most recent call last): File "/astro/py/clump.py", line 34, in <module> find_clumps(master_clump, c_min, c_max, step) File "/astro/yt-dev/yt-x86_64/src/yt-hg/yt/analysis_modules/level_sets/clump_handling.py", line 181, in find_clumps find_clumps(clump, min_val*d_clump, max_val, d_clump) File "/astro/yt-dev/yt-x86_64/src/yt-hg/yt/analysis_modules/level_sets/clump_handling.py", line 181, in find_clumps find_clumps(clump, min_val*d_clump, max_val, d_clump) File "/astro/yt-dev/yt-x86_64/src/yt-hg/yt/analysis_modules/level_sets/clump_handling.py", line 190, in find_clumps elif (child._isValid()): File "/astro/yt-dev/yt-x86_64/src/yt-hg/yt/analysis_modules/level_sets/clump_handling.py", line 144, in _isValid self.function_value = eval(self.function) File "<string>", line 1, in <module> File "/astro/yt-dev/yt-x86_64/src/yt-hg/yt/data_objects/derived_quantities.py", line 94, in __call__ return self._call_func_unlazy(args, kwargs) File "/astro/yt-dev/yt-x86_64/src/yt-hg/yt/data_objects/derived_quantities.py", line 119, in _call_func_unlazy retval = self.func(self._data_source, *args, **kwargs) File "/astro/yt-dev/yt-x86_64/src/yt-hg/yt/data_objects/derived_quantities.py", line 360, in _IsBound thermal = (data["ThermalEnergy"] * mass_to_use).sum() File "/astro/yt-dev/yt-x86_64/src/yt-hg/yt/data_objects/data_containers.py", line 327, in __getitem__ self.get_data(key) File "/astro/yt-dev/yt-x86_64/src/yt-hg/yt/data_objects/data_containers.py", line 2536, in get_data if self._generate_field(field): File "/astro/yt-dev/yt-x86_64/src/yt-hg/yt/data_objects/data_containers.py", line 359, in _generate_field self[field] = self.pf.field_info[field](self) File "/astro/yt-dev/yt-x86_64/src/yt-hg/yt/data_objects/field_info_container.py", line 316, in __call__ dd = self._function(self, data) File "/astro/yt-dev/yt-x86_64/src/yt-hg/yt/frontends/enzo/fields.py", line 134, in _ThermalEnergy return data["TotalEnergy"] File "/astro/yt-dev/yt-x86_64/src/yt-hg/yt/data_objects/data_containers.py", line 327, in __getitem__ self.get_data(key) File "/astro/yt-dev/yt-x86_64/src/yt-hg/yt/data_objects/data_containers.py", line 2536, in get_data if self._generate_field(field): File "/astro/yt-dev/yt-x86_64/src/yt-hg/yt/data_objects/data_containers.py", line 363, in _generate_field raise KeyError(field) KeyError: 'TotalEnergy' I did it with the whole data (it took forever), however, I got the same exactly error. Anybody has an advice?? -- Francia F.R.
Hi Francia, I'm sorry to hear that you're still having trouble with this. On Thu, Aug 30, 2012 at 12:38 PM, Francia F. Riesco <ff2214@columbia.edu> wrote:
Stephen, Geoffrey -- what's the summary of why this is still a problem? If we know that Forthon-0.8.7 fixes the issue, why can't we just revert to that in the install script?
[snip]
I did it with the whole data (it took forever), however, I got the same exactly error. Anybody has an advice??
I think I've figured out what's going on. This data was generated with a version of Enzo that diverged from main development before field names had been standardized, and while it contains "Total_Energy" it does not contain "TotalEnergy" which is what fields are now referred to as. (It also must have been done using the Zeus hydro solver.) So it's a perfect storm for yt not being sure how to get the thermal energy back, which is where this is all coming from -- the clump finder (as opposed to the level set identifier) only returns gravitationally bound clumps. The fix for this is to alias Total_Energy to TotalEnergy, so that when the latter is requested, the former is returned. I have now done this in the development repository. You can update your installation by running: yt update You can test that this will work (without running the full clump finder) by using a command like this: dd = pf.h.sphere("c", 100.0/pf['kpc']) print dd.quantities["Extrema"]("ThermalEnergy") Please do let us know if this fixes the issue. Best, Matt
Hi Francia & Matt,
The issue is that right now a user still needs to manually cd into the yt/utilities/kdtree directory and run "make", and then do "python setup.py install" at the top level, in order for the fortran shared object to work. Starting from a completely clean install of yt-dev (from the install_script.sh), I just now verified that this is what needs to be done to make things work. Is there a way to have "make" called from the setup process? -- Stephen Skory s@skory.us http://stephenskory.com/ 510.621.3687 (google voice)
On Thu, Aug 30, 2012 at 1:24 PM, Stephen Skory <s@skory.us> wrote:
Not without mucking about with subprocess, I think. What about contacting the author and asking if he could modify the way that forthon calls distutils so that we could add it as a subpackage, like we do for Cython code. Or, alternately, why not just do this in the install script? -Matt
Hi Matt,
Or, alternately, why not just do this in the install script?
Ok, I'll put on my 'bash scripting hat' and see if I can make this work. That would be a good solution. -- Stephen Skory s@skory.us http://stephenskory.com/ 510.621.3687 (google voice)
I would much prefer Forthon be installed as part of the install script (downloaded Forthon and installed it so many times by hand), so +1 on that suggestion... on that note, is it hard to add mpi4py to the list as well? From G.S. On Thu, Aug 30, 2012 at 10:26 AM, Matthew Turk <matthewturk@gmail.com>wrote:
On Thu, Aug 30, 2012 at 1:46 PM, Geoffrey So <gsiisg@gmail.com> wrote:
Forthon is already part of the install script. mpi4py is a lot harder to get installed correctly. We could make it optional, but I am somewhat disinclined to take responsibility for that. It's also been recently brought to my attention how long the dependency list has gotten (especially since we started encouraging the IPython notebook by default) and I'd like to try to winnow it down before we talk about adding new ones. -Matt
Hi all, I've issued a pull request that should automatically build and install the Fortran kD-tree when yt is installed. In my tests it works fine: https://bitbucket.org/yt_analysis/yt/pull-request/254/adding-a-line-to-autom... -- Stephen Skory s@skory.us http://stephenskory.com/ 510.621.3687 (google voice)
Hi Francia, I'm sorry to hear that you're still having trouble with this. On Thu, Aug 30, 2012 at 12:38 PM, Francia F. Riesco <ff2214@columbia.edu> wrote:
Stephen, Geoffrey -- what's the summary of why this is still a problem? If we know that Forthon-0.8.7 fixes the issue, why can't we just revert to that in the install script?
[snip]
I did it with the whole data (it took forever), however, I got the same exactly error. Anybody has an advice??
I think I've figured out what's going on. This data was generated with a version of Enzo that diverged from main development before field names had been standardized, and while it contains "Total_Energy" it does not contain "TotalEnergy" which is what fields are now referred to as. (It also must have been done using the Zeus hydro solver.) So it's a perfect storm for yt not being sure how to get the thermal energy back, which is where this is all coming from -- the clump finder (as opposed to the level set identifier) only returns gravitationally bound clumps. The fix for this is to alias Total_Energy to TotalEnergy, so that when the latter is requested, the former is returned. I have now done this in the development repository. You can update your installation by running: yt update You can test that this will work (without running the full clump finder) by using a command like this: dd = pf.h.sphere("c", 100.0/pf['kpc']) print dd.quantities["Extrema"]("ThermalEnergy") Please do let us know if this fixes the issue. Best, Matt
Hi Francia & Matt,
The issue is that right now a user still needs to manually cd into the yt/utilities/kdtree directory and run "make", and then do "python setup.py install" at the top level, in order for the fortran shared object to work. Starting from a completely clean install of yt-dev (from the install_script.sh), I just now verified that this is what needs to be done to make things work. Is there a way to have "make" called from the setup process? -- Stephen Skory s@skory.us http://stephenskory.com/ 510.621.3687 (google voice)
On Thu, Aug 30, 2012 at 1:24 PM, Stephen Skory <s@skory.us> wrote:
Not without mucking about with subprocess, I think. What about contacting the author and asking if he could modify the way that forthon calls distutils so that we could add it as a subpackage, like we do for Cython code. Or, alternately, why not just do this in the install script? -Matt
Hi Matt,
Or, alternately, why not just do this in the install script?
Ok, I'll put on my 'bash scripting hat' and see if I can make this work. That would be a good solution. -- Stephen Skory s@skory.us http://stephenskory.com/ 510.621.3687 (google voice)
I would much prefer Forthon be installed as part of the install script (downloaded Forthon and installed it so many times by hand), so +1 on that suggestion... on that note, is it hard to add mpi4py to the list as well? From G.S. On Thu, Aug 30, 2012 at 10:26 AM, Matthew Turk <matthewturk@gmail.com>wrote:
On Thu, Aug 30, 2012 at 1:46 PM, Geoffrey So <gsiisg@gmail.com> wrote:
Forthon is already part of the install script. mpi4py is a lot harder to get installed correctly. We could make it optional, but I am somewhat disinclined to take responsibility for that. It's also been recently brought to my attention how long the dependency list has gotten (especially since we started encouraging the IPython notebook by default) and I'd like to try to winnow it down before we talk about adding new ones. -Matt
Hi all, I've issued a pull request that should automatically build and install the Fortran kD-tree when yt is installed. In my tests it works fine: https://bitbucket.org/yt_analysis/yt/pull-request/254/adding-a-line-to-autom... -- Stephen Skory s@skory.us http://stephenskory.com/ 510.621.3687 (google voice)
participants (4)
-
Francia F. Riesco -
Geoffrey So -
Matthew Turk -
Stephen Skory