Hi Casey, I'm back from vacation and ready to start thinking about this again. :) On Thu, Jul 14, 2011 at 12:31 PM, Casey W. Stark wrote:

Hi Matt. Thanks for the reply. And don't worry about me getting stuck - I've been developing frontend stuff on another branch. That's a good point - I hadn't thought of data and derived fields with the same name. I think I'm confused about how this is supposed to fit together now. Here's an example from the current version of the code. In the field_info_container module, you have FieldInfoContainer the class, and FieldInfo, an instantiation of FieldInfoContainer.

Yup, and FieldInfo is the container for *all* the derived fields that are meant to be universal. This is stuff like cosmological over density, for instance, which broadly speaking has a unique method of generation independent of code. (And where the gas relationships are fully determinable from the parameter file, this class of field can include things like pressure, sound speed, etc.) In theory the idea of a set of universal fields works out better than in practice, for the most part, simply because of such divergent problem domains.

Then in the enzo frontend fields module, we have: EnzoFieldInfo = FieldInfoContainer.create_with_fallback(FieldInfo) add_field = EnzoFieldInfo.add_field KnownEnzoFields = FieldInfoContainer() add_enzo_field = KnownEnzoFields.add_field I think I wrote this when I didn't exactly understand the design. I believe it should be... derivable_enzo_fields = FieldInfo add_field = derivable_enzo_fields.add_field known_enzo_fields = FieldInfoContainer.create_with_fallback(derivable_enzo_fields) add_enzo_field = known_enzo_fields.add_field

This sounds good -- an important semantic difference. Note that one reason we keep add_field around is for legacy code which uses it, particularly in users' ~/.yt/my_plugins.py file.

The part that is confusing me is in the frontend StaticOutput classes. Here it is in enzo data_structures. class EnzoStaticOutput(StaticOutput):     ...     _fieldinfo_fallback = EnzoFieldInfo     _fieldinfo_known = KnownEnzoFields     ...     def __init__(...):         ...         self.field_info = FieldInfoContainer.create_with_fallback(self._fieldinfo_fallback) So it looks like _fieldinfo_known is never used and we make a new FieldInfoContainer instead. In this case, how does field_info know which fields can be in an enzo data file and which ones exist in the current staticoutput?

Ah, but fieldinfo_known *is* used, but in the base class. Check yt/data_objects/hierarchy.py . What happens is that known_fields is queried and if the field is found, it is added to the pf.field_info. This way the definition is grabbed from the list of known items.

The way I thought about it, each StaticOutput object would have a copy of that frontend's known_field object and customize the fields during __init__. Something like... self.field_info = self._fieldinfo_known.setup_for_output(self) I'll try to stare at the code some more, but if anyone has experience with this part of yt, please let me know what you think and I can code it up. Best, Casey

I think that's sort of how I aimed to implement it, too. Talk to you soon, Matt

On Mon, Jul 11, 2011 at 8:55 AM, Matthew Turk wrote:

...

Hi Casey,

Sorry for the brief reply; I'm on vacation but have a handful of minutes to browse my email and I don't want you to get stuck too long.  I don't think I understand the issue -- I think the pf.field_info object is supposed to contain all of the data fields, with a fallback for derived.  The reason for this is so that adding a derived field after the instantiation of a parameter file will allow it to be accessed.  i.e.,

pf = load(...) add_field("Something")

"Something" won't be available if the fields are fixed at instantiation time with no fallback.  One of the reasons we have the "known in data" and "derived" separation is so that if for instance the code can optionally output a field (like "Dark_Matter_Density") we should be able to use it, but if it doesn't, then it should make a new derived field.  Does that make sense?

However, it sounds like something is more broadly wrong than just the fallbacks -- likely my fault somewhere along the line.  If you can post a slightly more detailed design description maybe I can understand better and we can then adopt your method.

-Matt

On Wed, Jul 6, 2011 at 5:46 PM, Casey W. Stark wrote:

...

Hello yt. I'm working on the deliberate_fields branch and came up with a design idea. I want to check in with yt-dev before implementing it. I was trying to get the new design to pass tests, but could not get it working. It looks like pf.field_info is fine during the first test and is empty when the second test starts. I suspect this issue is related to the new fallback design, but I could not track it down. The new field_info design gives FieldInfoContainer a fallback attribute that should be another FieldInfoContainer. The idea is that a parent FieldInfoContainer contains all the fields from data, and the fallback attribute contains all of the derivable fields. Fields in both of these objects are instances of `yt.data_objects.field_info_container.DerivedField`, created with the add_field method in a frontend fields module. I thought it would be easier to have a FieldInfoContainer that contains all fields (for a given frontend) and is aware of which fields are from data and which are derivable. I think you can get the same behavior as the fallback design without modifying the dict behavior. When defining the frontend fields, you add fields that might be in data files with code_field_info.add_data_field and you add fields that could be derived given all dependences with code_field_info.add_derivable_field. When a pf is loaded, prune the default field_info to contain only the available fields. Of course, I'm pretty new to the yt code, so I don't want to break any behavior before 3.0. I would really appreciate any feedback, especially if you know this would create problems. I will stash my changes for now and come back to it. Best, Casey _______________________________________________ Yt-dev mailing list Yt-dev@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org

_______________________________________________ Yt-dev mailing list Yt-dev@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org

_______________________________________________ Yt-dev mailing list Yt-dev@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org