Re: [yt-users] problem with covering_grid for Nyx data sets (Wolfram Schmidt), yt-users Digest, Vol 66, Issue 17
Hi Nathan, thanks a lot for your suggestions! I updated yt to the current development version, which didn't fix the problem. So I tried what you suggested: On 08/06/2013 07:52 PM, yt-users-request@lists.spacepope.org wrote:
and then try your script again. If the issue isn't fixed by updating, I would then try applying the following diff to the nyx frontend:
diff -r f936432ed45d yt/frontends/nyx/data_structures.py --- a/yt/frontends/nyx/data_structures.py Tue Jul 30 10:08:24 2013 -0700 +++ b/yt/frontends/nyx/data_structures.py Tue Aug 06 08:10:25 2013 -0700 @@ -611,7 +611,7 @@ # aliases we need self.parameters["TopGridRank"] = len(self.parameters["TopGridDimensions"] self.dimensionality = self.parameters["TopGridRank"] - self.domain_dimensions = self.parameters["TopGridDimensions"] + self.domain_dimensions = np.array(self.parameters["TopGridDimensions"]) self.refine_by = self.parameters.get("RefineBy", 2) # 2 is silent defaul
# Nyx is always cosmological.
I replaced the definition of self.domain_dimensions with the np.array thing. Now, the script actually reads in the data, but then another problem is encountered when an FFT is applied to the data: Searching grids for values 100% ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||| Time: 00:03:04 yt : [ERROR ] 2013-08-07 11:56:48,958 Covering problem: 16777216 cells are uncovered Traceback (most recent call last): File "/usr/users/wschmid2/scripts/spect_sb.py", line 72, in <module> velx_hat = na.fft.fftn( cg['xmom']/cg['density'] )/cg['xmom'].size File "/home/uni05/wschmid2/yt-i686/src/yt-hg/yt/data_objects/data_containers.py", line 332, in __getitem__ self.get_data(key) File "/home/uni05/wschmid2/yt-i686/src/yt-hg/yt/data_objects/data_containers.py", line 3776, in get_data raise KeyError(n_bad) KeyError: 16777216 The line with the fft actually used to work and it still works just fine if I use an old yt version (which, fortunately, I was able to dig out). Do you understand what is the problem here? Cheers, Wolfram
Hi Wolfram, The error you're seeing happens when the covering grid isn't properly filled with data sampled from the original AMR dataset. Unfortunately, It looks like the covering grid isn't being generated correctly in the current version of yt. I'm no expert in this part of the codebase so I don't think I can help out much in debugging the issue. Can you go ahead and open an issue about this? The link to open a new issue is here: https://bitbucket.org/yt_analysis/yt/issues/new There's no need to register a bitbucket account to report an issue, but if you do you will get e-mails when there are updates and when the issue is fixed. It would ease debugging and increase the probability that your issue will be fixed if you share a script that runs successfully in the old version of yt and unsuccessfully in the current develoment version, as well as the mercurial changeset hash (obtained by running `yt instinfo` from the command line) for the old yt version. I believe Matt has some Nyx test datasets, but if your script acts on a smallish (< 1 GB) dataset that you can share (via e.g. dropbox, google drive, or a personal website), that would also ease debugging. Again, sorry to hear about the regression, hopefully we'll be able to figure out the root cause of the issue here. Cheers, Nathan On Wed, Aug 7, 2013 at 3:17 AM, Wolfram Schmidt < schmidt@astro.physik.uni-goettingen.de> wrote:
Hi Nathan,
thanks a lot for your suggestions! I updated yt to the current development version, which didn't fix the problem. So I tried what you suggested:
On 08/06/2013 07:52 PM, yt-users-request@lists.**spacepope.org<yt-users-request@lists.spacepope.org>wrote:
and then try your script again. If the issue isn't fixed by updating, I would then try applying the following diff to the nyx frontend:
diff -r f936432ed45d yt/frontends/nyx/data_**structures.py --- a/yt/frontends/nyx/data_**structures.py Tue Jul 30 10:08:24 2013 -0700 +++ b/yt/frontends/nyx/data_**structures.py Tue Aug 06 08:10:25 2013 -0700 @@ -611,7 +611,7 @@ # aliases we need self.parameters["TopGridRank"] = len(self.parameters["**TopGridDimensions"] self.dimensionality = self.parameters["TopGridRank"] - self.domain_dimensions = self.parameters["**TopGridDimensions"] + self.domain_dimensions = np.array(self.parameters["**TopGridDimensions"]) self.refine_by = self.parameters.get("RefineBy"**, 2) # 2 is silent defaul
# Nyx is always cosmological.
I replaced the definition of self.domain_dimensions with the np.array thing. Now, the script actually reads in the data, but then another problem is encountered when an FFT is applied to the data:
Searching grids for values 100% ||||||||||||||||||||||||||||||** ||||||||||||||||||||||||||||||**||||||||||||||||||||||||||||||**||||||||||||||||||||||||| Time: 00:03:04 yt : [ERROR ] 2013-08-07 11:56:48,958 Covering problem: 16777216 cells are uncovered Traceback (most recent call last): File "/usr/users/wschmid2/scripts/**spect_sb.py", line 72, in <module> velx_hat = na.fft.fftn( cg['xmom']/cg['density'] )/cg['xmom'].size File "/home/uni05/wschmid2/yt-i686/**src/yt-hg/yt/data_objects/**data_containers.py", line 332, in __getitem__ self.get_data(key) File "/home/uni05/wschmid2/yt-i686/**src/yt-hg/yt/data_objects/**data_containers.py", line 3776, in get_data raise KeyError(n_bad) KeyError: 16777216
The line with the fft actually used to work and it still works just fine if I use an old yt version (which, fortunately, I was able to dig out). Do you understand what is the problem here?
Cheers, Wolfram
______________________________**_________________ yt-users mailing list yt-users@lists.spacepope.org http://lists.spacepope.org/**listinfo.cgi/yt-users-**spacepope.org<http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org>
Hi Wolfram, I'd like to address this issue; I've been away but am returning now to online status. I'd be happy to provide space to upload Nyx data, if you send me an SSH key off-list. -Matt On Wed, Aug 7, 2013 at 12:48 PM, Nathan Goldbaum <nathan12343@gmail.com> wrote:
Hi Wolfram,
The error you're seeing happens when the covering grid isn't properly filled with data sampled from the original AMR dataset. Unfortunately, It looks like the covering grid isn't being generated correctly in the current version of yt. I'm no expert in this part of the codebase so I don't think I can help out much in debugging the issue.
Can you go ahead and open an issue about this? The link to open a new issue is here: https://bitbucket.org/yt_analysis/yt/issues/new
There's no need to register a bitbucket account to report an issue, but if you do you will get e-mails when there are updates and when the issue is fixed.
It would ease debugging and increase the probability that your issue will be fixed if you share a script that runs successfully in the old version of yt and unsuccessfully in the current develoment version, as well as the mercurial changeset hash (obtained by running `yt instinfo` from the command line) for the old yt version. I believe Matt has some Nyx test datasets, but if your script acts on a smallish (< 1 GB) dataset that you can share (via e.g. dropbox, google drive, or a personal website), that would also ease debugging.
Again, sorry to hear about the regression, hopefully we'll be able to figure out the root cause of the issue here.
Cheers,
Nathan
On Wed, Aug 7, 2013 at 3:17 AM, Wolfram Schmidt <schmidt@astro.physik.uni-goettingen.de> wrote:
Hi Nathan,
thanks a lot for your suggestions! I updated yt to the current development version, which didn't fix the problem. So I tried what you suggested:
On 08/06/2013 07:52 PM, yt-users-request@lists.spacepope.org wrote:
and then try your script again. If the issue isn't fixed by updating, I would then try applying the following diff to the nyx frontend:
diff -r f936432ed45d yt/frontends/nyx/data_structures.py --- a/yt/frontends/nyx/data_structures.py Tue Jul 30 10:08:24 2013 -0700 +++ b/yt/frontends/nyx/data_structures.py Tue Aug 06 08:10:25 2013 -0700 @@ -611,7 +611,7 @@ # aliases we need self.parameters["TopGridRank"] = len(self.parameters["TopGridDimensions"] self.dimensionality = self.parameters["TopGridRank"] - self.domain_dimensions = self.parameters["TopGridDimensions"] + self.domain_dimensions = np.array(self.parameters["TopGridDimensions"]) self.refine_by = self.parameters.get("RefineBy", 2) # 2 is silent defaul
# Nyx is always cosmological.
I replaced the definition of self.domain_dimensions with the np.array thing. Now, the script actually reads in the data, but then another problem is encountered when an FFT is applied to the data:
Searching grids for values 100% ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||| Time: 00:03:04 yt : [ERROR ] 2013-08-07 11:56:48,958 Covering problem: 16777216 cells are uncovered Traceback (most recent call last): File "/usr/users/wschmid2/scripts/spect_sb.py", line 72, in <module> velx_hat = na.fft.fftn( cg['xmom']/cg['density'] )/cg['xmom'].size File "/home/uni05/wschmid2/yt-i686/src/yt-hg/yt/data_objects/data_containers.py", line 332, in __getitem__ self.get_data(key) File "/home/uni05/wschmid2/yt-i686/src/yt-hg/yt/data_objects/data_containers.py", line 3776, in get_data raise KeyError(n_bad) KeyError: 16777216
The line with the fft actually used to work and it still works just fine if I use an old yt version (which, fortunately, I was able to dig out). Do you understand what is the problem here?
Cheers, Wolfram
_______________________________________________ yt-users mailing list yt-users@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
_______________________________________________ yt-users mailing list yt-users@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
participants (3)
-
Matthew Turk -
Nathan Goldbaum -
Wolfram Schmidt