Nathan, I think it was my bug - the field detector wants to see all the fields accessed inside the call. If I do the following, then the code works: def StellarLuminosity(field,data): ts = data[('STAR','age')] zs = data[('STAR','metallicity_snii')] + data[('STAR','metallicity_snia')] ms = data[('STAR','particle_mass')] if('FieldDetector' in str(type(data))): return ms else: #ts = data[('STAR','age')].in_units("yr").v #zs = (data[('STAR','metallicity_snii')]+data[('STAR','metallicity_snia')]).v return ms.in_units("Msun") # return sps.GetLnu(ts,zs)*data[('STAR','mass')].in_units("Msun") Thank you so much for all the help, sorry for keeping pestering you. n On 5/27/2018 1:37 PM, Nathan Goldbaum wrote:
So in this case you've written the field to behave one way during field detection and another way during the real field evaluation. This is confusing the field machinery.
I think you're running into a bug in how the artio frontend handles some of its particle fields, see this pull request I just opened:
https://github.com/yt-project/yt/pull/1802 <https://github.com/yt-project/yt/pull/1802>
After applying that patch I think you'll be able to avoid the check to see if you're in field detection.
On Sun, May 27, 2018 at 1:20 PM, Nick Gnedin <ngnedin@gmail.com <mailto:ngnedin@gmail.com>> wrote:
Nathan,
I made progress with the real code, but then I hit another error that leaves me dumbfounded:
---------------------------- import yt
def StellarLuminosity(field,data): if(not 'FieldDetector' in str(type(data))): ts = data[('STAR','age')].in_units("yr").v zs = (data[('STAR','metallicity_snii')]+data[('STAR','metallicity_snia')]).v return data[('STAR','mass')].in_units("Msun") else: return data[('STAR','age')]
d = yt.load("rei10_a0.1001/rei10_a0.1001.art")
d.add_field(("STAR","luminosity"),function=StellarLuminosity,units="Msun",sampling_type="particle") d.add_deposited_particle_field(("STAR","luminosity"),"nearest",weight_field='particle_ones')
print(d.derived_field_list)
print(d.all_data()[('STAR','metallicity_snii')]) print(d.all_data()[("deposit","STAR_nn_luminosity")]) --------------------------
Below is the error I get. ('STAR','metallicity_snii') certainly exists, I can print it, but the deposited field seems to be unable to find it.
yt : [INFO ] 2018-05-27 13:15:16,205 Created 256 chunks for ARTIO [ 5.39916630e-30 2.21650149e-22 2.95487393e-06 ..., 1.23813297e-05 1.28902484e-05 1.42594863e-05] dimensionless yt : [INFO ] 2018-05-27 13:15:16,396 Created 256 chunks for ARTIO Traceback (most recent call last): File "/data/gnedin/soft/anaconda3/lib/python3.6/site-packages/yt/data_objects/data_containers.py", line 335, in _generate_fluid_field finfo.check_available(gen_obj) File "/data/gnedin/soft/anaconda3/lib/python3.6/site-packages/yt/fields/derived_field.py", line 199, in check_available validator(data) File "/data/gnedin/soft/anaconda3/lib/python3.6/site-packages/yt/fields/derived_field.py", line 418, in __call__ raise NeedsGridType(self.ghost_zones,self.fields) yt.fields.field_exceptions.NeedsGridType: (0, None)
During handling of the above exception, another exception occurred:
Traceback (most recent call last): File "debug.py", line 23, in <module> print(d.all_data()[("deposit","STAR_nn_luminosity")]) File "/data/gnedin/soft/anaconda3/lib/python3.6/site-packages/yt/data_objects/data_containers.py", line 282, in __getitem__ self.get_data(f) File "/data/gnedin/soft/anaconda3/lib/python3.6/site-packages/yt/data_objects/data_containers.py", line 1337, in get_data self._generate_fields(fields_to_generate) File "/data/gnedin/soft/anaconda3/lib/python3.6/site-packages/yt/data_objects/data_containers.py", line 1357, in _generate_fields fd = self._generate_field(field) File "/data/gnedin/soft/anaconda3/lib/python3.6/site-packages/yt/data_objects/data_containers.py", line 319, in _generate_field tr = self._generate_fluid_field(field) File "/data/gnedin/soft/anaconda3/lib/python3.6/site-packages/yt/data_objects/data_containers.py", line 337, in _generate_fluid_field rv = self._generate_spatial_fluid(field, ngt_exception.ghost_zones) File "/data/gnedin/soft/anaconda3/lib/python3.6/site-packages/yt/data_objects/data_containers.py", line 357, in _generate_spatial_fluid ind += o.select(self.selector, self[field], rv, ind) File "/data/gnedin/soft/anaconda3/lib/python3.6/site-packages/yt/data_objects/data_containers.py", line 282, in __getitem__ self.get_data(f) File "/data/gnedin/soft/anaconda3/lib/python3.6/site-packages/yt/data_objects/data_containers.py", line 1337, in get_data self._generate_fields(fields_to_generate) File "/data/gnedin/soft/anaconda3/lib/python3.6/site-packages/yt/data_objects/data_containers.py", line 1357, in _generate_fields fd = self._generate_field(field) File "/data/gnedin/soft/anaconda3/lib/python3.6/site-packages/yt/data_objects/data_containers.py", line 319, in _generate_field tr = self._generate_fluid_field(field) File "/data/gnedin/soft/anaconda3/lib/python3.6/site-packages/yt/data_objects/data_containers.py", line 339, in _generate_fluid_field rv = finfo(gen_obj) File "/data/gnedin/soft/anaconda3/lib/python3.6/site-packages/yt/fields/derived_field.py", line 237, in __call__ dd = self._function(self, data) File "/data/gnedin/soft/anaconda3/lib/python3.6/site-packages/yt/data_objects/static_output.py", line 1274, in _deposit_field fields = [data[ptype, deposit_field]] File "/data/gnedin/soft/anaconda3/lib/python3.6/site-packages/yt/data_objects/octree_subset.py", line 75, in __getitem__ tr = super(OctreeSubset, self).__getitem__(key) File "/data/gnedin/soft/anaconda3/lib/python3.6/site-packages/yt/data_objects/data_containers.py", line 282, in __getitem__ self.get_data(f) File "/data/gnedin/soft/anaconda3/lib/python3.6/site-packages/yt/data_objects/data_containers.py", line 1337, in get_data self._generate_fields(fields_to_generate) File "/data/gnedin/soft/anaconda3/lib/python3.6/site-packages/yt/data_objects/data_containers.py", line 1357, in _generate_fields fd = self._generate_field(field) File "/data/gnedin/soft/anaconda3/lib/python3.6/site-packages/yt/data_objects/data_containers.py", line 317, in _generate_field tr = self._generate_particle_field(field) File "/data/gnedin/soft/anaconda3/lib/python3.6/site-packages/yt/data_objects/data_containers.py", line 402, in _generate_particle_field rv = self.ds._get_field_info(*field)(gen_obj) File "/data/gnedin/soft/anaconda3/lib/python3.6/site-packages/yt/fields/derived_field.py", line 235, in __call__ "for %s" % (self.name <http://self.name>,)) RuntimeError: Something has gone terribly wrong, _function is NullFunc for ('STAR', 'METALLICITY_SNII')
n
On 5/27/2018 12:08 PM, Nathan Goldbaum wrote:
Hi Nick,
What you're seeing in those print statements is how yt's field detection system works. Before we actually select and return data, we pass your field function fake data to make sure it doesn't raise errors or it doesn't select fields that aren't available for your dataset.
Here's a script I made based on yours that helps explain what's going on:
https://gist.github.com/ngoldbaum/4d27987bda8ecbce5c9bd9b98bbbbf49 <https://gist.github.com/ngoldbaum/4d27987bda8ecbce5c9bd9b98bbbbf49>
And here's the output of that script:
https://gist.github.com/ngoldbaum/475fcd827a53e4d02e76cb25711ff5ea <https://gist.github.com/ngoldbaum/475fcd827a53e4d02e76cb25711ff5ea>
I made a couple modifications compared to your script. First, I added a line at the end that selects your new field from a data object, which causes real data to be fed into the field function after field detection completes. I also specified "units='auto'" in the call to add_field. The dimensions keyword is only used when "units='auto'" is set.
You can see that your field function is called twice. First with a `FieldDetector` instance and then the second time with a real yt data object. In general the field function might be called several times before real data is supplied to it. One way I commonly deal with this is the following hack to only see output when the field definition is called with real data:
def NewField(field, data): q = data[('STAR', 'age')] if 'FieldDetector' in str(type(data)): print("BBB",data) return q
Although more often I use python's debugger via "import pdb; pdb.set_trace()" instead of print statements.
Hope that helps,
Nathan
On Sun, May 27, 2018 at 10:51 AM, Nick Gnedin <ngnedin@gmail.com <mailto:ngnedin@gmail.com> <mailto:ngnedin@gmail.com <mailto:ngnedin@gmail.com>>> wrote:
Guys,
I am trying to add a new particle field, and apparently I am doing something wrong, but I could find the documentation for this:
import yt def NewField(field,data): q = data[('STAR','age')] print("BBB",data) return q d = yt.load("rei10_a0.1001/rei10_a0.1001.art") print('AAA',d.all_data()[("STAR","age")])
d.add_field(("STAR","new-field"),function=NewField,dimensions="time",sampling_type="particle")
The result is here:
AAA [ 1.35689106e+08 1.29578798e+08 1.15241829e+08 ..., 2.97175878e+07 2.04648908e+07 1.35365980e+07] yr BBB defaultdict(<function FieldDetector.__init__.<locals>.<lambda> at 0x2b9e698e3598>, {('STAR', 'BIRTH_TIME'): YTArray([ 1.]) (dimensionless), ('STAR', 'creation_time'): YTArray([ 1.]) (dimensionless), ('STAR', 'age'): YTArray([ 1.]) (dimensionless)})
It appears that the data object sent to NewField(field,data) has all its fields set to dimensionless.
I tried omitting sampling_type="particle", but it does not help, just generates an incomprehensible warning.
n _______________________________________________ yt-users mailing list -- yt-users@python.org <mailto:yt-users@python.org> <mailto:yt-users@python.org <mailto:yt-users@python.org>> To unsubscribe send an email to yt-users-leave@python.org <mailto:yt-users-leave@python.org> <mailto:yt-users-leave@python.org <mailto:yt-users-leave@python.org>>
participants (1)
-
Nick Gnedin