Re: [Yt-dev] not_in_all

Hi David, On Sun, Jun 26, 2011 at 9:34 PM, david collins wrote:

Hey, Matt--

Sorry it took me a minute to get back to this.  I have a work around, so this is low priority for all parties.  But it's behavior I don't quite understand, so I thought I'd ask.

I added a conversion specifier as you suggested, in my plugins file, as this below, but I still see the same error.

for aaa in list_of_fields:    add_field(aaa,lambda a, b: None, convert_function = lambda a:1.0, take_log = False, not_in_all=True)

The traceback I get is:

<trace> Traceback (most recent call last):  File "mrg_check_pstp.py", line 29, in <module>    ud(output='x')  File "/Users/dcollins/Sites/Research/CodeGames/yt2.0/uberdiff.py", line 166, in __call__    g1_full = self.uber1.h.grids[grid1-1][field1]  File "/Users/dcollins/local/lib/python2.6/site-packages/yt-2.1dev-py2.6-macosx-10.4-i386.egg/yt/data_objects/grid_patch.py", line 147, in __getitem__    self.get_data(key)  File "/Users/dcollins/local/lib/python2.6/site-packages/yt-2.1dev-py2.6-macosx-10.4-i386.egg/yt/data_objects/grid_patch.py", line 190, in get_data    self._generate_field(field)  File "/Users/dcollins/local/lib/python2.6/site-packages/yt-2.1dev-py2.6-macosx-10.4-i386.egg/yt/data_objects/grid_patch.py", line 135, in _generate_field    self[field] = self.pf.field_info[field](self)  File "/Users/dcollins/local/lib/python2.6/site-packages/yt-2.1dev-py2.6-macosx-10.4-i386.egg/yt/data_objects/field_info_container.py", line 308, in __call__    dd *= self._convert_function(data) TypeError: unsupported operand type(s) for *=: 'NoneType' and 'float' </trace>

When I jump in with pdb, self._convert_function(data) returns 1.0, as intended, but

dd = self._function(self,data)

is returning none.  (the location of that is below) I suppose this isn't surprising, since that's the function I gave it.  When I force the field to be defined on all grids, the problem goes away.  I feel like this code has worked in the past for a different not_in_all field.  But I also feel like I should be living in a state that doesn't catch on fire so readily, so apparently I'm wrong a lot.  So is the "function" argument to my add_field wrong, or is there a better choice?

I agree, this used to work, but right now the field state is a bit mixed up. In order to meet the needs of the various different incompatible Enzo versions, I made a few changes that made things work more reliably in the case of derived fields that mirror non-derived fields. This likely changed the behavior you're seeing. I would say that perhaps you could fix it by lookign for where the not_in_all is checked inside the grid patch object and then tryign to determine which portions (hg blame) of that code have recently changed. Unfortunately, as I mentioned, Casey and I are working to replace the current field system with a new one. The new one should remove a number of the ifs/elifs/etc that make the field system a bit spaghetti. So I am going to try to focus on that, since you have a workaround for now; it should improve maintainability. Please do feel free to fix the necessary code and either submit a pull request or push it to the repository. Thanks, Matt

Thanks a ton, d.

<pdb list>

...

...

...

pdb.pm() /Users/dcollins/local/lib/python2.6/site-packages/yt-2.1dev-py2.6-macosx-10.4-i386.egg/yt/data_objects/field_info_container.py(308)__call__() -> dd *= self._convert_function(data) (Pdb) list 303             Return the value of the field in a given *data* object. 304             """ 305             ii = self.check_available(data) 306             original_fields = data.keys() # Copy 307             dd = self._function(self, data) 308  ->         dd *= self._convert_function(data) 309             for field_name in data.keys():

On Mon, Jun 20, 2011 at 9:55 PM, Matthew Turk wrote:

...

Hi Dave,

This comes from not having a conversion specifier.  You can mock this up by manually adding an entry to pf.conversion_factors, or, better yet, adding the field in your plugins with a lambda a: 1.0 for the conversion function.

The long term solution is to follow Enzo's conventions and include a #DataCGSConversion line above the name of the data label.  This would auto-supply a conversion factor.

Casey and I have been working hard on re-doing the field specifiers to create classes of "known" and "derived" fields that are specific to each code.  That would help somewhat with this, but unfortunately it is very, very challenging in general to detect fields and try to manipulate them productively.  The new field system will help make this clearer and more straightforward.

-Matt

On Mon, Jun 20, 2011 at 8:50 PM, david collins wrote:

...

It does.

However I changed something since my last email, now I'm getting a different error.  Now I get:

Traceback (most recent call last):  File "<stdin>", line 1, in <module>  File "/Users/dccollins/local/src/yt-hg/yt/data_objects/grid_patch.py", line 147, in __getitem__    self.get_data(key)  File "/Users/dccollins/local/src/yt-hg/yt/data_objects/grid_patch.py", line 190, in get_data    self._generate_field(field)  File "/Users/dccollins/local/src/yt-hg/yt/data_objects/grid_patch.py", line 135, in _generate_field    self[field] = self.pf.field_info[field](self)  File "/Users/dccollins/local/src/yt-hg/yt/data_objects/field_info_container.py", line 308, in __call__    dd *= self._convert_function(data) TypeError: unsupported operand type(s) for *=: 'NoneType' and 'float'

That leads me to believe this is operator error-- I'll poke around more and get back to you.

Thanks, d.

On Mon, Jun 20, 2011 at 8:58 PM, Matthew Turk wrote:

...

Hi Dave,

Does it show up in the list of detected fields when debug mode is on?

-Matt

On Mon, Jun 20, 2011 at 7:53 PM, david collins wrote:

...

Hi--

I have (a probably stupid) problem.  I have a field that I'm writing out to some grids.  The field is called 'AvgElec0', and only exists on level>0 grids (non-root-grids).  I've defined this field

def _AEx(field,data):    return data['AvgElec0'][:,:-1,:-1] add_field('AEx',function=_AEx,validators=[ValidateSpatial(0)],take_log=False,not_in_all=True)

(the slice is for the centering of the field).  When I do something like pf.h.grids[1]['AEx'] I get a key error, "AvgElec0," even though double checking the field is in fact in that grid.  If I change the code so it's written on all levels, the same pf.h.grids[1]['AEx'] works fine, as one would expect.  Has the not_in_all behavior changed?  Might I be doing something stupid?

Thanks, d.

-- Sent from my computer. _______________________________________________ Yt-dev mailing list Yt-dev@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org

_______________________________________________ Yt-dev mailing list Yt-dev@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org

-- Sent from my computer. _______________________________________________ Yt-dev mailing list Yt-dev@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org

_______________________________________________ Yt-dev mailing list Yt-dev@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org

-- Sent from my computer. _______________________________________________ Yt-dev mailing list Yt-dev@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org