Problem with Derived Quantities/Fields while running in parallel
I am running yt-2.1stable with python2.7 on Kraken and run into this problem: When I call a derived quantity that calls a derived field that calls a universal field, I get an error: "AttributeError: 'fake_parameter_file' object has no attribute 'parameters'" which arises in the universal field's convert function in 'data_objects/universal_fields.py' With the following code: #derived Field def _test2(field,data): return data['Baryon_Overdensity'] add_field('test2',function=_test2,units=r"") #derived quantity def _test(data): return data['test2'] def _combTest(data,arr): return arr.sum() add_quantity('test',function=_test,combine_function=_combTest,n_ret=1) dd.quantities['test']() I get the error: AttributeError: 'fake_parameter_file' object has no attribute 'parameters' from line 418, in _Convert_Baryon_Overdensity in universal_fields.py: "data.pf.parameters.has_key('omega_baryon_now'):" So it seems as if the parameter file is not being passed from the derived quantitty all the way to the universal field. This works fine when I run it in serial, but fails when it is ran in parallel. Any suggestions? Thanks
Hi Anthony, I can replicate this problem, as can Sam. Here's what I think is going on. When you run in parallel and access a derived quantity, it really wants to try to batch the IO as best it can. To do this, it uses a FieldDector. This is a fake object that accepts requests for fields, data, parameters, etc. It does this by tracking accesses to fields and parameters, but it doesn't mock up a complete parameter file. The baryon overdensity field uses a non-standard method of accessing data from the parameter file, which the fake parameter file didn't know. Rather than access the dictionary directly (pf['omega_baryon_now']) it accessed it by peeking into the underlying data structure 'parameters.' I have changed this and pushed a fix to the main repository, in changeset 7270e9e83be7. If you're running from an installation you maintain, you should be able to do "yt instinfo -u" to get the latest version. Let me know if that doesn't fix it for you -- it does for me, but there might be an additional complicating factor I'm missing. -Matt On Mon, May 23, 2011 at 5:49 PM, Anthony Harness <anthony.harness@colorado.edu> wrote:
I am running yt-2.1stable with python2.7 on Kraken and run into this problem:
When I call a derived quantity that calls a derived field that calls a universal field, I get an error: "AttributeError: 'fake_parameter_file' object has no attribute 'parameters'" which arises in the universal field's convert function in 'data_objects/universal_fields.py'
With the following code: #derived Field def _test2(field,data): return data['Baryon_Overdensity']
add_field('test2',function=_test2,units=r"")
#derived quantity def _test(data): return data['test2']
def _combTest(data,arr): return arr.sum()
add_quantity('test',function=_test,combine_function=_combTest,n_ret=1)
dd.quantities['test']()
I get the error:
AttributeError: 'fake_parameter_file' object has no attribute 'parameters'
from line 418, in _Convert_Baryon_Overdensity in universal_fields.py: "data.pf.parameters.has_key('omega_baryon_now'):"
So it seems as if the parameter file is not being passed from the derived quantitty all the way to the universal field. This works fine when I run it in serial, but fails when it is ran in parallel. Any suggestions?
Thanks
_______________________________________________ yt-users mailing list yt-users@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
participants (2)
-
Anthony Harness -
Matthew Turk