Hello fellow yt'ers: I'm having some issues with some fields that I don't think are behaving the way they should be in yt. I'm encountering a number of problems with a simple 2D enzo dataset (x and y), so I put together a notebook demonstrating my various issues. https://hub.yt-project.org/nb/727j2j To summarize the weirdness from the notebook: 1) The 'dx' field doesn't seem to be able to be displayed (i.e. when I try to slice it), whereas the 'dy' field is fine. 'dz' even exists, but it's just uniformly 0, as is expected from a 2D dataset. 2) When I try to produce a new field which is 2^GridLevel, I get weird results (ignored levels and more). 3-5) I get strange behavior when trying to make a new field combining GridLevel and 'dy', since GridLevel has twice as many indices as any of the spatial fields (in the z direction no less). This results in some odd behavior when I try to pair various subsets of GridLevel with 'dy'. Any clarification of these ideas is welcome. The docs and source haven't yet revealed what is going on here. Thanks! Cameron -- Cameron Hummels Postdoctoral Researcher Steward Observatory University of Arizona http://chummels.org
Hi Cameron, Just for completeness, Nathan and I were discussing in irc but I'll respond here my thoughts up until now. At least the first piece about dx is a bug: I think this comes from dx being an "excluded" field in a fixed resolution buffer, and when the SlicePlot goes to loop over all the data source fields, it gets explicitly ignored. Obviously this should not be the case if the user explicitly adds a slice of dx. This is a bug. In yt/visualization/fixed_resolution.py you can see why dy works: _exclude_fields = ['pz','dpz','dx','x','y','z'] For the rest of the issues, I think if you get rid of ValidateSpatial for everything that you have in the notebook, it runs fine. ValidateSpatial of more than 0 will mean you need to account for that and take the correct slices. That said, in 2d I can't make sense of the shapes that come out of ValidateSpatial(1) for GridLevel. I don't know what is up with that. I'll also just say that either: sl.set_log(field, boolean) or pf.h; pf.field_info[field].take_log=False is easier than a derived field just to set the log. One last thing -- I'm not sure why the following code has 2 returns in the universal_fields.py. def _dy(field, data): return data.dds[1] <--- note the return here. The rest should probably be removed. Not sure why it is even there. return np.ones(data.ActiveDimensions, dtype='float64') * data.dds[1]add_field('dy_lin', function=_dy, display_field=False, validators=[ValidateSpatial(0)], take_log=False) Best, Sam On Mon, Sep 23, 2013 at 8:00 PM, Cameron Hummels <chummels@gmail.com> wrote:
Hello fellow yt'ers:
I'm having some issues with some fields that I don't think are behaving the way they should be in yt. I'm encountering a number of problems with a simple 2D enzo dataset (x and y), so I put together a notebook demonstrating my various issues.
https://hub.yt-project.org/nb/727j2j
To summarize the weirdness from the notebook:
1) The 'dx' field doesn't seem to be able to be displayed (i.e. when I try to slice it), whereas the 'dy' field is fine. 'dz' even exists, but it's just uniformly 0, as is expected from a 2D dataset.
2) When I try to produce a new field which is 2^GridLevel, I get weird results (ignored levels and more).
3-5) I get strange behavior when trying to make a new field combining GridLevel and 'dy', since GridLevel has twice as many indices as any of the spatial fields (in the z direction no less). This results in some odd behavior when I try to pair various subsets of GridLevel with 'dy'.
Any clarification of these ideas is welcome. The docs and source haven't yet revealed what is going on here. Thanks!
Cameron
-- Cameron Hummels Postdoctoral Researcher Steward Observatory University of Arizona http://chummels.org
_______________________________________________ yt-users mailing list yt-users@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
On Tue, Sep 24, 2013 at 12:21 AM, Sam Skillman <samskillman@gmail.com> wrote:
Hi Cameron,
Just for completeness, Nathan and I were discussing in irc but I'll respond here my thoughts up until now. At least the first piece about dx is a bug:
I think this comes from dx being an "excluded" field in a fixed resolution buffer, and when the SlicePlot goes to loop over all the data source fields, it gets explicitly ignored. Obviously this should not be the case if the user explicitly adds a slice of dx. This is a bug. In yt/visualization/fixed_resolution.py you can see why dy works: _exclude_fields = ['pz','dpz','dx','x','y','z']
For the rest of the issues, I think if you get rid of ValidateSpatial for everything that you have in the notebook, it runs fine. ValidateSpatial of more than 0 will mean you need to account for that and take the correct slices. That said, in 2d I can't make sense of the shapes that come out of ValidateSpatial(1) for GridLevel. I don't know what is up with that.
I'll also just say that either: sl.set_log(field, boolean) or pf.h; pf.field_info[field].take_log=False is easier than a derived field just to set the log.
One last thing -- I'm not sure why the following code has 2 returns in the universal_fields.py.
def _dy(field, data): return data.dds[1] <--- note the return here. The rest should probably be removed. Not sure why it is even there. return np.ones(data.ActiveDimensions, dtype='float64') * data.dds[1] add_field('dy_lin', function=_dy, display_field=False, validators=[ValidateSpatial(0)], take_log=False)
I left in the old code after I added the short-circuit, in case we ever needed to go back. Note that this returns a scalar (since it's validate spatial and so it should be identical everywhere) rather than an array.
Best, Sam
On Mon, Sep 23, 2013 at 8:00 PM, Cameron Hummels <chummels@gmail.com> wrote:
Hello fellow yt'ers:
I'm having some issues with some fields that I don't think are behaving the way they should be in yt. I'm encountering a number of problems with a simple 2D enzo dataset (x and y), so I put together a notebook demonstrating my various issues.
https://hub.yt-project.org/nb/727j2j
To summarize the weirdness from the notebook:
1) The 'dx' field doesn't seem to be able to be displayed (i.e. when I try to slice it), whereas the 'dy' field is fine. 'dz' even exists, but it's just uniformly 0, as is expected from a 2D dataset.
2) When I try to produce a new field which is 2^GridLevel, I get weird results (ignored levels and more).
3-5) I get strange behavior when trying to make a new field combining GridLevel and 'dy', since GridLevel has twice as many indices as any of the spatial fields (in the z direction no less). This results in some odd behavior when I try to pair various subsets of GridLevel with 'dy'.
Any clarification of these ideas is welcome. The docs and source haven't yet revealed what is going on here. Thanks!
Cameron
-- Cameron Hummels Postdoctoral Researcher Steward Observatory University of Arizona http://chummels.org
_______________________________________________ yt-users mailing list yt-users@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
_______________________________________________ yt-users mailing list yt-users@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
The bug that prevented you from plotting the 'dx' field has been fixed: https://bitbucket.org/yt_analysis/yt/pull-request/607/allow-plotting-of-excl... Nathan On Monday, September 23, 2013, Sam Skillman wrote:
Hi Cameron,
Just for completeness, Nathan and I were discussing in irc but I'll respond here my thoughts up until now. At least the first piece about dx is a bug:
I think this comes from dx being an "excluded" field in a fixed resolution buffer, and when the SlicePlot goes to loop over all the data source fields, it gets explicitly ignored. Obviously this should not be the case if the user explicitly adds a slice of dx. This is a bug. In yt/visualization/fixed_resolution.py you can see why dy works: _exclude_fields = ['pz','dpz','dx','x','y','z']
For the rest of the issues, I think if you get rid of ValidateSpatial for everything that you have in the notebook, it runs fine. ValidateSpatial of more than 0 will mean you need to account for that and take the correct slices. That said, in 2d I can't make sense of the shapes that come out of ValidateSpatial(1) for GridLevel. I don't know what is up with that.
I'll also just say that either: sl.set_log(field, boolean) or pf.h; pf.field_info[field].take_log=False is easier than a derived field just to set the log.
One last thing -- I'm not sure why the following code has 2 returns in the universal_fields.py.
def _dy(field, data): return data.dds[1] <--- note the return here. The rest should probably be removed. Not sure why it is even there. return np.ones(data.ActiveDimensions, dtype='float64') * data.dds[1]add_field('dy_lin', function=_dy, display_field=False, validators=[ValidateSpatial(0)], take_log=False)
Best, Sam
On Mon, Sep 23, 2013 at 8:00 PM, Cameron Hummels <chummels@gmail.com<javascript:_e({}, 'cvml', 'chummels@gmail.com');>
wrote:
Hello fellow yt'ers:
I'm having some issues with some fields that I don't think are behaving the way they should be in yt. I'm encountering a number of problems with a simple 2D enzo dataset (x and y), so I put together a notebook demonstrating my various issues.
https://hub.yt-project.org/nb/727j2j
To summarize the weirdness from the notebook:
1) The 'dx' field doesn't seem to be able to be displayed (i.e. when I try to slice it), whereas the 'dy' field is fine. 'dz' even exists, but it's just uniformly 0, as is expected from a 2D dataset.
2) When I try to produce a new field which is 2^GridLevel, I get weird results (ignored levels and more).
3-5) I get strange behavior when trying to make a new field combining GridLevel and 'dy', since GridLevel has twice as many indices as any of the spatial fields (in the z direction no less). This results in some odd behavior when I try to pair various subsets of GridLevel with 'dy'.
Any clarification of these ideas is welcome. The docs and source haven't yet revealed what is going on here. Thanks!
Cameron
-- Cameron Hummels Postdoctoral Researcher Steward Observatory University of Arizona http://chummels.org
_______________________________________________ yt-users mailing list yt-users@lists.spacepope.org <javascript:_e({}, 'cvml', 'yt-users@lists.spacepope.org');> http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
I wanted to follow up on this thread. First of all, thank you Sam, Nathan, and Matt for your input. You were correct on many levels: that the main problem stemmed from using ValidateSpatial(1) on a 2D dataset, that the 'dx' field could not be plotted because of some assumptions made in the PlotWindow interface (which have now been fixed), etc. Thanks! But I have some related questions regarding this. My main reason for asking the questions in the notebook is that I'm trying to create a more complex derived field, shear, which I would like to work on both 2D and 3D datasets. It requires having 1 additional ghostzone on all sides in order to work (at least for velocity fields). I'm simply trying to scale the field by dx*(2^GridLevel), and this is what led to the weirdness that I distilled into the notebook I sent last week. When I try to use the dx and GridLevel fields as ValidateSpatial(1), I end up with the strange behavior I demonstrated in the notebook (and which is fixed by removing it or going to ValidateSpatial(0) ) , but I need to include that in order to have them work compatibly with the Velocity fields which are ValidateSpatial(1). Or is there a way to designate ValidateSpatial(1) for some fields and ValidateSpatial(0) for other fields? I tried doing it as a list of [ValidateSpatial(1), ValidateSpatial(0)] in the add_field call, but it still gave me mismatch errors in indices when trying to combine the fields. Does anyone have any ideas as to how to match up these seemingly incompatible field initializations under one roof? I've put together another notebook which identifies some of the problems I'm encountering on this front: https://hub.yt-project.org/nb/4i8fdg Thanks for all of the help, everyone! Cameron On Tue, Sep 24, 2013 at 8:41 AM, Nathan Goldbaum <nathan12343@gmail.com>wrote:
The bug that prevented you from plotting the 'dx' field has been fixed:
https://bitbucket.org/yt_analysis/yt/pull-request/607/allow-plotting-of-excl...
Nathan
On Monday, September 23, 2013, Sam Skillman wrote:
Hi Cameron,
Just for completeness, Nathan and I were discussing in irc but I'll respond here my thoughts up until now. At least the first piece about dx is a bug:
I think this comes from dx being an "excluded" field in a fixed resolution buffer, and when the SlicePlot goes to loop over all the data source fields, it gets explicitly ignored. Obviously this should not be the case if the user explicitly adds a slice of dx. This is a bug. In yt/visualization/fixed_resolution.py you can see why dy works: _exclude_fields = ['pz','dpz','dx','x','y','z']
For the rest of the issues, I think if you get rid of ValidateSpatial for everything that you have in the notebook, it runs fine. ValidateSpatial of more than 0 will mean you need to account for that and take the correct slices. That said, in 2d I can't make sense of the shapes that come out of ValidateSpatial(1) for GridLevel. I don't know what is up with that.
I'll also just say that either: sl.set_log(field, boolean) or pf.h; pf.field_info[field].take_log=False is easier than a derived field just to set the log.
One last thing -- I'm not sure why the following code has 2 returns in the universal_fields.py.
def _dy(field, data): return data.dds[1] <--- note the return here. The rest should probably be removed. Not sure why it is even there. return np.ones(data.ActiveDimensions, dtype='float64') * data.dds[1]add_field('dy_lin', function=_dy, display_field=False, validators=[ValidateSpatial(0)], take_log=False)
Best, Sam
On Mon, Sep 23, 2013 at 8:00 PM, Cameron Hummels <chummels@gmail.com>wrote:
Hello fellow yt'ers:
I'm having some issues with some fields that I don't think are behaving the way they should be in yt. I'm encountering a number of problems with a simple 2D enzo dataset (x and y), so I put together a notebook demonstrating my various issues.
https://hub.yt-project.org/nb/727j2j
To summarize the weirdness from the notebook:
1) The 'dx' field doesn't seem to be able to be displayed (i.e. when I try to slice it), whereas the 'dy' field is fine. 'dz' even exists, but it's just uniformly 0, as is expected from a 2D dataset.
2) When I try to produce a new field which is 2^GridLevel, I get weird results (ignored levels and more).
3-5) I get strange behavior when trying to make a new field combining GridLevel and 'dy', since GridLevel has twice as many indices as any of the spatial fields (in the z direction no less). This results in some odd behavior when I try to pair various subsets of GridLevel with 'dy'.
Any clarification of these ideas is welcome. The docs and source haven't yet revealed what is going on here. Thanks!
Cameron
-- Cameron Hummels Postdoctoral Researcher Steward Observatory University of Arizona http://chummels.org
_______________________________________________ yt-users mailing list yt-users@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
_______________________________________________ yt-users mailing list yt-users@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
-- Cameron Hummels Postdoctoral Researcher Steward Observatory University of Arizona http://chummels.org
Hi Cameron, On Wed, Oct 2, 2013 at 11:46 AM, Cameron Hummels <chummels@gmail.com> wrote:
I wanted to follow up on this thread. First of all, thank you Sam, Nathan, and Matt for your input. You were correct on many levels: that the main problem stemmed from using ValidateSpatial(1) on a 2D dataset, that the 'dx' field could not be plotted because of some assumptions made in the PlotWindow interface (which have now been fixed), etc. Thanks!
But I have some related questions regarding this. My main reason for asking the questions in the notebook is that I'm trying to create a more complex derived field, shear, which I would like to work on both 2D and 3D datasets. It requires having 1 additional ghostzone on all sides in order to work (at least for velocity fields). I'm simply trying to scale the field by dx*(2^GridLevel), and this is what led to the weirdness that I distilled into the notebook I sent last week. When I try to use the dx and GridLevel fields as ValidateSpatial(1), I end up with the strange behavior I demonstrated in the notebook (and which is fixed by removing it or going to ValidateSpatial(0) ) , but I need to include that in order to have them work compatibly with the Velocity fields which are ValidateSpatial(1). Or is there a way to designate ValidateSpatial(1) for some fields and ValidateSpatial(0) for other fields? I tried doing it as a list of [ValidateSpatial(1), ValidateSpatial(0)] in the add_field call, but it still gave me mismatch errors in indices when trying to combine the fields. Does anyone have any ideas as to how to match up these seemingly incompatible field initializations under one roof?
Yes -- you need to strip off the extra zones before returning a field from a ValidateSpatial(N) where N > 0. For instance, this means this would be a bad definition: @derived_field(name = "MyField1", validators=[ValidateSpatial(1)]) def _some_field(field, data): return data["Density"] but this would be a good one: @derived_field(name = "MyField1", validators=[ValidateSpatial(1)]) def _some_field(field, data): return data["Density"][1:-1,1:-1,1:-1] That's why "ShearMach" doesn't work, because the field is incorrectly sized when it is returned, and why the error shows up the same for the next item. I should note that what happens when your field definition is wrapped in a ValidateSpatial is that *all* of the fields accessed inside the field definition have the additional ghost zones. What I would recommend you do is: * Do not nest ValidateSpatial fields. As in, do not call a ValidateSpatial field from within another. Instead, consolidate all definitions. You should be able to access GridLevel inside your mach, and it should have the correct size. * Use the ShearMach3 definition as your starting place. The "data" variable like has a .Level attribute as well, but I do not think you should rely on it. That being said, the usage of 2D data for ghost zones is mostly unexplored. Do you have your data somewhere I can test on it? -Matt
I've put together another notebook which identifies some of the problems I'm encountering on this front:
https://hub.yt-project.org/nb/4i8fdg
Thanks for all of the help, everyone!
Cameron
On Tue, Sep 24, 2013 at 8:41 AM, Nathan Goldbaum <nathan12343@gmail.com> wrote:
The bug that prevented you from plotting the 'dx' field has been fixed:
https://bitbucket.org/yt_analysis/yt/pull-request/607/allow-plotting-of-excl...
Nathan
On Monday, September 23, 2013, Sam Skillman wrote:
Hi Cameron,
Just for completeness, Nathan and I were discussing in irc but I'll respond here my thoughts up until now. At least the first piece about dx is a bug:
I think this comes from dx being an "excluded" field in a fixed resolution buffer, and when the SlicePlot goes to loop over all the data source fields, it gets explicitly ignored. Obviously this should not be the case if the user explicitly adds a slice of dx. This is a bug. In yt/visualization/fixed_resolution.py you can see why dy works: _exclude_fields = ['pz','dpz','dx','x','y','z']
For the rest of the issues, I think if you get rid of ValidateSpatial for everything that you have in the notebook, it runs fine. ValidateSpatial of more than 0 will mean you need to account for that and take the correct slices. That said, in 2d I can't make sense of the shapes that come out of ValidateSpatial(1) for GridLevel. I don't know what is up with that.
I'll also just say that either: sl.set_log(field, boolean) or pf.h; pf.field_info[field].take_log=False is easier than a derived field just to set the log.
One last thing -- I'm not sure why the following code has 2 returns in the universal_fields.py.
def _dy(field, data): return data.dds[1] <--- note the return here. The rest should probably be removed. Not sure why it is even there. return np.ones(data.ActiveDimensions, dtype='float64') * data.dds[1] add_field('dy_lin', function=_dy, display_field=False, validators=[ValidateSpatial(0)], take_log=False)
Best, Sam
On Mon, Sep 23, 2013 at 8:00 PM, Cameron Hummels <chummels@gmail.com> wrote:
Hello fellow yt'ers:
I'm having some issues with some fields that I don't think are behaving the way they should be in yt. I'm encountering a number of problems with a simple 2D enzo dataset (x and y), so I put together a notebook demonstrating my various issues.
https://hub.yt-project.org/nb/727j2j
To summarize the weirdness from the notebook:
1) The 'dx' field doesn't seem to be able to be displayed (i.e. when I try to slice it), whereas the 'dy' field is fine. 'dz' even exists, but it's just uniformly 0, as is expected from a 2D dataset.
2) When I try to produce a new field which is 2^GridLevel, I get weird results (ignored levels and more).
3-5) I get strange behavior when trying to make a new field combining GridLevel and 'dy', since GridLevel has twice as many indices as any of the spatial fields (in the z direction no less). This results in some odd behavior when I try to pair various subsets of GridLevel with 'dy'.
Any clarification of these ideas is welcome. The docs and source haven't yet revealed what is going on here. Thanks!
Cameron
-- Cameron Hummels Postdoctoral Researcher Steward Observatory University of Arizona http://chummels.org
_______________________________________________ yt-users mailing list yt-users@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
_______________________________________________ yt-users mailing list yt-users@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
-- Cameron Hummels Postdoctoral Researcher Steward Observatory University of Arizona http://chummels.org
_______________________________________________ yt-users mailing list yt-users@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
Hi Matt, Thanks for the message and suggestions.
Yes -- you need to strip off the extra zones before returning a field from a ValidateSpatial(N) where N > 0. For instance, this means this would be a bad definition:
@derived_field(name = "MyField1", validators=[ValidateSpatial(1)]) def _some_field(field, data): return data["Density"]
but this would be a good one:
@derived_field(name = "MyField1", validators=[ValidateSpatial(1)]) def _some_field(field, data): return data["Density"][1:-1,1:-1,1:-1]
I agree that this makes sense, but I actually mostly copied the shear field definition from the VorticitySquared field definition in the code, which did not observe this rule. In fact, neither does DivV. These fields return a field which is [x+2][y+2][z+2] in shape, which I found odd, but that is why I left it as it was in my code. Either way, when I make the correction to return a field of size [x][y][z], it doesn't seem to fix the problem, unfortunately. The error seems to occur prior to the field being returned, so I think that it may be an independent problem.
That's why "ShearMach" doesn't work, because the field is incorrectly sized when it is returned, and why the error shows up the same for the next item. I should note that what happens when your field definition is wrapped in a ValidateSpatial is that *all* of the fields accessed inside the field definition have the additional ghost zones.
I don't understand this. How is my entire field definition wrapped in a ValidateSpatial? I only include ValidateSpatial in the validators when I add the field--it doesn't seem to me that it is wrapping the whole thing in any of my 3 field definitions, but maybe I'm just missing this. In my ShearMach3, I simply have a list of separate validators for different fields, one ValidateSpatial with 1 ghost zone for the velocity fields, and one ValidateSpatial with 0 ghost zones for the dx and GridLevel fields. I guess my use of whitespace makes it look like they are nested, but they are not.
The "data" variable like has a .Level attribute as well, but I do not think you should rely on it.
I'll look at this level attribute to see if this helps.
That being said, the usage of 2D data for ghost zones is mostly unexplored. Do you have your data somewhere I can test on it?
The dataset is up at http://www.astro.columbia.edu/~chummels/DD0100.tar.gzif you want to take a look at it. It's small and a quick download. Thanks for the help with this! Cameron -- Cameron Hummels Postdoctoral Researcher Steward Observatory University of Arizona http://chummels.org
On Wed, Oct 2, 2013 at 6:20 PM, Cameron Hummels <chummels@gmail.com> wrote:
Hi Matt,
Thanks for the message and suggestions.
Yes -- you need to strip off the extra zones before returning a field from a ValidateSpatial(N) where N > 0. For instance, this means this would be a bad definition:
@derived_field(name = "MyField1", validators=[ValidateSpatial(1)]) def _some_field(field, data): return data["Density"]
but this would be a good one:
@derived_field(name = "MyField1", validators=[ValidateSpatial(1)]) def _some_field(field, data): return data["Density"][1:-1,1:-1,1:-1]
I agree that this makes sense, but I actually mostly copied the shear field definition from the VorticitySquared field definition in the code, which did not observe this rule. In fact, neither does DivV. These fields return a field which is [x+2][y+2][z+2] in shape, which I found odd, but that is why I left it as it was in my code. Either way, when I make the correction to return a field of size [x][y][z], it doesn't seem to fix the problem, unfortunately. The error seems to occur prior to the field being returned, so I think that it may be an independent problem.
You're right, I was wrong -- the field needs to be Ngz*2 bigger in each dimension.
That's why "ShearMach" doesn't work, because the field is incorrectly sized when it is returned, and why the error shows up the same for the next item. I should note that what happens when your field definition is wrapped in a ValidateSpatial is that *all* of the fields accessed inside the field definition have the additional ghost zones.
I don't understand this. How is my entire field definition wrapped in a ValidateSpatial? I only include ValidateSpatial in the validators when I add the field--it doesn't seem to me that it is wrapping the whole thing in any of my 3 field definitions, but maybe I'm just missing this. In my ShearMach3, I simply have a list of separate validators for different fields, one ValidateSpatial with 1 ghost zone for the velocity fields, and one ValidateSpatial with 0 ghost zones for the dx and GridLevel fields. I guess my use of whitespace makes it look like they are nested, but they are not.
Nothing, besides "dx", "dy" and "dz", will have varying shape inside the field definition. What ValidateSpatial *does* is receive how many ghost zones you need as well as the fields you think you might want. If you ask for a field you do not enumerate in that list, it too will have N+2*NGZ in its shape, but it will be generated a second time. This is a hint to yt so that it can consolidate IO, as generating ghost zones can be expensive. Supplying multiple ValidateSpatial arguments will not vary the behavior depending on the field.
The "data" variable like has a .Level attribute as well, but I do not think you should rely on it.
I'll look at this level attribute to see if this helps.
That being said, the usage of 2D data for ghost zones is mostly unexplored. Do you have your data somewhere I can test on it?
The dataset is up at http://www.astro.columbia.edu/~chummels/DD0100.tar.gz if you want to take a look at it. It's small and a quick download.
Thank you, I will take a look. -Matt
Thanks for the help with this!
Cameron
-- Cameron Hummels Postdoctoral Researcher Steward Observatory University of Arizona http://chummels.org
_______________________________________________ yt-users mailing list yt-users@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
On Wed, Oct 2, 2013 at 6:28 PM, Matthew Turk <matthewturk@gmail.com> wrote:
On Wed, Oct 2, 2013 at 6:20 PM, Cameron Hummels <chummels@gmail.com> wrote:
Hi Matt,
Thanks for the message and suggestions.
Yes -- you need to strip off the extra zones before returning a field from a ValidateSpatial(N) where N > 0. For instance, this means this would be a bad definition:
@derived_field(name = "MyField1", validators=[ValidateSpatial(1)]) def _some_field(field, data): return data["Density"]
but this would be a good one:
@derived_field(name = "MyField1", validators=[ValidateSpatial(1)]) def _some_field(field, data): return data["Density"][1:-1,1:-1,1:-1]
I agree that this makes sense, but I actually mostly copied the shear field definition from the VorticitySquared field definition in the code, which did not observe this rule. In fact, neither does DivV. These fields return a field which is [x+2][y+2][z+2] in shape, which I found odd, but that is why I left it as it was in my code. Either way, when I make the correction to return a field of size [x][y][z], it doesn't seem to fix the problem, unfortunately. The error seems to occur prior to the field being returned, so I think that it may be an independent problem.
You're right, I was wrong -- the field needs to be Ngz*2 bigger in each dimension.
That's why "ShearMach" doesn't work, because the field is incorrectly sized when it is returned, and why the error shows up the same for the next item. I should note that what happens when your field definition is wrapped in a ValidateSpatial is that *all* of the fields accessed inside the field definition have the additional ghost zones.
I don't understand this. How is my entire field definition wrapped in a ValidateSpatial? I only include ValidateSpatial in the validators when I add the field--it doesn't seem to me that it is wrapping the whole thing in any of my 3 field definitions, but maybe I'm just missing this. In my ShearMach3, I simply have a list of separate validators for different fields, one ValidateSpatial with 1 ghost zone for the velocity fields, and one ValidateSpatial with 0 ghost zones for the dx and GridLevel fields. I guess my use of whitespace makes it look like they are nested, but they are not.
Nothing, besides "dx", "dy" and "dz", will have varying shape inside the field definition.
What ValidateSpatial *does* is receive how many ghost zones you need as well as the fields you think you might want. If you ask for a field you do not enumerate in that list, it too will have N+2*NGZ in its shape, but it will be generated a second time. This is a hint to yt so that it can consolidate IO, as generating ghost zones can be expensive. Supplying multiple ValidateSpatial arguments will not vary the behavior depending on the field.
The "data" variable like has a .Level attribute as well, but I do not think you should rely on it.
I'll look at this level attribute to see if this helps.
That being said, the usage of 2D data for ghost zones is mostly unexplored. Do you have your data somewhere I can test on it?
The dataset is up at http://www.astro.columbia.edu/~chummels/DD0100.tar.gz if you want to take a look at it. It's small and a quick download.
Thank you, I will take a look.
The problem seems to be that the nested spatial fields -- which you are not responsible for, but is how yt generates "GridLevel" and "dx" -- are causing the issue. Try using "data.level" and "data.dds". Incidentally, it seems to be a combination of the nested spatial fields and the 2D dataset. In 3D, your definitions work for me. -Matt
-Matt
Thanks for the help with this!
Cameron
-- Cameron Hummels Postdoctoral Researcher Steward Observatory University of Arizona http://chummels.org
_______________________________________________ yt-users mailing list yt-users@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
Interesting. When I replace data["GridLevel"] with data.Level, it works in cases where I'm using ValidateSpatial(0), but not in cases when ValidateSpatial(1) is called. Even in a simple test case this is true. So for instance this works: def _testfield(field, data): level = data.Level return data.dds[0]*2.0**level add_field("TestField", function=_testfield, validators=[ValidateSpatial(0)]) But this does not: def _testfield(field, data): level = data.Level return data.dds[0]*2.0**level add_field("TestField", function=_testfield, validators=[ValidateSpatial(1,["dx"])]) with the traceback pointing to line: "level = data.Level" stating that AMRSmoothedCoveringGrid object has no attribute 'Level'. This is weird because the same dataset works for the previous field call so clearly the AMRSmoothedCoveringGrid *does* have that attribute. So I guess there is no way around this for 2D datasets, or at least, this 2D dataset? I'm out of ideas here. On Wed, Oct 2, 2013 at 3:38 PM, Matthew Turk <matthewturk@gmail.com> wrote:
On Wed, Oct 2, 2013 at 6:20 PM, Cameron Hummels <chummels@gmail.com> wrote:
Hi Matt,
Thanks for the message and suggestions.
Yes -- you need to strip off the extra zones before returning a field from a ValidateSpatial(N) where N > 0. For instance, this means this would be a bad definition:
@derived_field(name = "MyField1", validators=[ValidateSpatial(1)]) def _some_field(field, data): return data["Density"]
but this would be a good one:
@derived_field(name = "MyField1", validators=[ValidateSpatial(1)]) def _some_field(field, data): return data["Density"][1:-1,1:-1,1:-1]
I agree that this makes sense, but I actually mostly copied the shear field definition from the VorticitySquared field definition in the code, which did not observe this rule. In fact, neither does DivV. These fields return a field which is [x+2][y+2][z+2] in shape, which I found odd, but that is why I left it as it was in my code. Either way, when I make the correction to return a field of size [x][y][z], it doesn't seem to fix the problem, unfortunately. The error seems to occur prior to the field being returned, so I think that it may be an independent problem.
You're right, I was wrong -- the field needs to be Ngz*2 bigger in each dimension.
That's why "ShearMach" doesn't work, because the field is incorrectly sized when it is returned, and why the error shows up the same for the next item. I should note that what happens when your field definition is wrapped in a ValidateSpatial is that *all* of the fields accessed inside the field definition have the additional ghost zones.
I don't understand this. How is my entire field definition wrapped in a ValidateSpatial? I only include ValidateSpatial in the validators when
I
add the field--it doesn't seem to me that it is wrapping the whole
On Wed, Oct 2, 2013 at 6:28 PM, Matthew Turk <matthewturk@gmail.com> wrote: thing in
any of my 3 field definitions, but maybe I'm just missing this. In my ShearMach3, I simply have a list of separate validators for different fields, one ValidateSpatial with 1 ghost zone for the velocity fields, and one ValidateSpatial with 0 ghost zones for the dx and GridLevel fields. I guess my use of whitespace makes it look like they are nested, but they are not.
Nothing, besides "dx", "dy" and "dz", will have varying shape inside the field definition.
What ValidateSpatial *does* is receive how many ghost zones you need as well as the fields you think you might want. If you ask for a field you do not enumerate in that list, it too will have N+2*NGZ in its shape, but it will be generated a second time. This is a hint to yt so that it can consolidate IO, as generating ghost zones can be expensive. Supplying multiple ValidateSpatial arguments will not vary the behavior depending on the field.
The "data" variable like has a .Level attribute as well, but I do not think you should rely on it.
I'll look at this level attribute to see if this helps.
That being said, the usage of 2D data for ghost zones is mostly unexplored. Do you have your data somewhere I can test on it?
The dataset is up at
http://www.astro.columbia.edu/~chummels/DD0100.tar.gz
if you want to take a look at it. It's small and a quick download.
Thank you, I will take a look.
The problem seems to be that the nested spatial fields -- which you are not responsible for, but is how yt generates "GridLevel" and "dx" -- are causing the issue. Try using "data.level" and "data.dds".
Incidentally, it seems to be a combination of the nested spatial fields and the 2D dataset. In 3D, your definitions work for me.
-Matt
-Matt
Thanks for the help with this!
Cameron
-- Cameron Hummels Postdoctoral Researcher Steward Observatory University of Arizona http://chummels.org
_______________________________________________ yt-users mailing list yt-users@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
_______________________________________________ yt-users mailing list yt-users@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
-- Cameron Hummels Postdoctoral Researcher Steward Observatory University of Arizona http://chummels.org
Hi Cameron, On Wed, Oct 2, 2013 at 7:06 PM, Cameron Hummels <chummels@gmail.com> wrote:
Interesting. When I replace data["GridLevel"] with data.Level, it works in cases where I'm using ValidateSpatial(0), but not in cases when ValidateSpatial(1) is called. Even in a simple test case this is true. So for instance this works:
def _testfield(field, data): level = data.Level return data.dds[0]*2.0**level add_field("TestField", function=_testfield, validators=[ValidateSpatial(0)])
But this does not:
def _testfield(field, data): level = data.Level return data.dds[0]*2.0**level add_field("TestField", function=_testfield, validators=[ValidateSpatial(1,["dx"])])
with the traceback pointing to line: "level = data.Level" stating that AMRSmoothedCoveringGrid object has no attribute 'Level'. This is weird because the same dataset works for the previous field call so clearly the AMRSmoothedCoveringGrid *does* have that attribute.
data.level works, but data.Level does not.
So I guess there is no way around this for 2D datasets, or at least, this 2D dataset? I'm out of ideas here.
This script produces three identical, nice-looking plots for me: http://paste.yt-project.org/show/3914/ It is based on your notebook. The problem is almost certainly the combination of the following things: * GridLevel (which is ValidateSpatial(0)) * dx (which as I mentioned previously in IRC usually is used with a "just_one" call inside a ValidateSpatial field) * 2D data * Asking for ghost zones What appears to be going on is that some logic in how the size of the grids is calculated is causing wraparound along the (false) third dimension that is added for 2D data, which escalates once you throw in nested ValidateSpatial fields. I don't know why you want to use either VS=0 or VS=1 interchangeably; these are not necessarily meant to do the same thing. My suggestion would be that if you want to use the fields "dx" and "GridLevel" that you instead manually strip off the z-axis edges of those particular fields if dimensionality < 2. -Matt
Hi Cameron, On Mon, Sep 23, 2013 at 11:00 PM, Cameron Hummels <chummels@gmail.com> wrote:
Hello fellow yt'ers:
I'm having some issues with some fields that I don't think are behaving the way they should be in yt. I'm encountering a number of problems with a simple 2D enzo dataset (x and y), so I put together a notebook demonstrating my various issues.
https://hub.yt-project.org/nb/727j2j
To summarize the weirdness from the notebook:
1) The 'dx' field doesn't seem to be able to be displayed (i.e. when I try to slice it), whereas the 'dy' field is fine. 'dz' even exists, but it's just uniformly 0, as is expected from a 2D dataset.
2) When I try to produce a new field which is 2^GridLevel, I get weird results (ignored levels and more).
3-5) I get strange behavior when trying to make a new field combining GridLevel and 'dy', since GridLevel has twice as many indices as any of the spatial fields (in the z direction no less). This results in some odd behavior when I try to pair various subsets of GridLevel with 'dy'.
Just to echo what Sam said, the field definitions you are using in many cases have an additional index from the ValidateSpatial call you supply. An additional comment is that oftentimes, visualization is not the only way to inspect what appear to be oddities. As an example, operations like this: dd = pf.h.all_data() print np.unique(dd["dx"] * 2**dd["GridLevels"]) and so on can give useful pieces of information as well.
Any clarification of these ideas is welcome. The docs and source haven't yet revealed what is going on here. Thanks!
Cameron
-- Cameron Hummels Postdoctoral Researcher Steward Observatory University of Arizona http://chummels.org
_______________________________________________ yt-users mailing list yt-users@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
participants (4)
-
Cameron Hummels -
Matthew Turk -
Nathan Goldbaum -
Sam Skillman