Hi all, The method for adding derived fields has somewhat changed in the unitrefactor of 3.0, and we have the opportunity to make it a bit easier to add new fields -- especially after a parameter file has been created, which previously has been a pain. Below is a brief summary. The current way we do this is with: add_field(...) or @derived_field def some_function(... The field system is now plugin based, which means we load plugins for different things and fields are dynamically added. One big advantage is that logic for things like strides and whatnot doesn't need to happen inside the field functions, but it also provides some compartmentalization of things. I haven't implemented @derived_field and add_field yet in this system, but my plan is to do so by creating a "custom" plugin, to which these will add field definitions, which will always get loaded. But if you do: pf = ... pf.h.whatever... @derived_field def func(...)... currently, and in the proposed implementation, this will not let them be detected. But I want to make it so that we *can* do this: pf.add_field( ... ) and @pf.derived_field, which will both dynamically add fields to existing parameter files. What this means: * This will not change behavior of @derived_field and add_field, in that they only apply to pfs instantiated *after* the call happens. * There will be a way to dynamically add fields, with field dependencies and all, by using pf attributes. * Adding fields *specific to an output type* will no longer be possible outside of field plugins. This means there will be the ability to add pf-specific and universal fields dynamically in a script, but all fields that are specific to an output type will need to be added in the fields.py file for that frontend. So this means that anything you might have used add_enzo_field or add_orion_field on *in your scripts* (not in anything inside yt/frontends/*/fields.py) will no longer work. I'm going to be implementing the uncontroversial part of this, which is the dynamic field adding to a "custom" plugin, and unless I hear otherwise I'll continue on this track with the dynamic pf field addition. -Matt
I think I have a use case that this design might make more difficult.
Let's say I'm adding new fields to an existing code for some reason (like
radiation, chemistry, or MHD). I'd like to make it so users can read in my
fields with yt and then make derived fields of their own using my fields.
Under your proposal, would I need to modify the yt source to be able to
read the fields in properly and assign them units? If so, that seems a bit
awkward to me since there would need to be upstream changes to yt that only
make sense relative to someone's private fork of a hydro code.
On Thu, Jan 23, 2014 at 1:34 PM, Matthew Turk
Hi all,
The method for adding derived fields has somewhat changed in the unitrefactor of 3.0, and we have the opportunity to make it a bit easier to add new fields -- especially after a parameter file has been created, which previously has been a pain. Below is a brief summary.
The current way we do this is with:
add_field(...)
or
@derived_field def some_function(...
The field system is now plugin based, which means we load plugins for different things and fields are dynamically added. One big advantage is that logic for things like strides and whatnot doesn't need to happen inside the field functions, but it also provides some compartmentalization of things.
I haven't implemented @derived_field and add_field yet in this system, but my plan is to do so by creating a "custom" plugin, to which these will add field definitions, which will always get loaded. But if you do:
pf = ... pf.h.whatever...
@derived_field def func(...)...
currently, and in the proposed implementation, this will not let them be detected. But I want to make it so that we *can* do this:
pf.add_field( ... ) and @pf.derived_field, which will both dynamically add fields to existing parameter files. What this means:
* This will not change behavior of @derived_field and add_field, in that they only apply to pfs instantiated *after* the call happens. * There will be a way to dynamically add fields, with field dependencies and all, by using pf attributes. * Adding fields *specific to an output type* will no longer be possible outside of field plugins. This means there will be the ability to add pf-specific and universal fields dynamically in a script, but all fields that are specific to an output type will need to be added in the fields.py file for that frontend. So this means that anything you might have used add_enzo_field or add_orion_field on *in your scripts* (not in anything inside yt/frontends/*/fields.py) will no longer work.
I'm going to be implementing the uncontroversial part of this, which is the dynamic field adding to a "custom" plugin, and unless I hear otherwise I'll continue on this track with the dynamic pf field addition.
-Matt _______________________________________________ yt-dev mailing list yt-dev@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
On Thu, Jan 23, 2014 at 5:13 PM, Nathan Goldbaum
I think I have a use case that this design might make more difficult.
Let's say I'm adding new fields to an existing code for some reason (like radiation, chemistry, or MHD). I'd like to make it so users can read in my fields with yt and then make derived fields of their own using my fields.
Yes, that's an ideal use case we need to support.
Under your proposal, would I need to modify the yt source to be able to read the fields in properly and assign them units? If so, that seems a bit awkward to me since there would need to be upstream changes to yt that only make sense relative to someone's private fork of a hydro code.
Hmm, reading in units for a field that yt does not know about, that is actually an outstanding problem with units that predates my changes to the field system. I don't know what to do about it.
On Thu, Jan 23, 2014 at 1:34 PM, Matthew Turk
wrote: Hi all,
The method for adding derived fields has somewhat changed in the unitrefactor of 3.0, and we have the opportunity to make it a bit easier to add new fields -- especially after a parameter file has been created, which previously has been a pain. Below is a brief summary.
The current way we do this is with:
add_field(...)
or
@derived_field def some_function(...
The field system is now plugin based, which means we load plugins for different things and fields are dynamically added. One big advantage is that logic for things like strides and whatnot doesn't need to happen inside the field functions, but it also provides some compartmentalization of things.
I haven't implemented @derived_field and add_field yet in this system, but my plan is to do so by creating a "custom" plugin, to which these will add field definitions, which will always get loaded. But if you do:
pf = ... pf.h.whatever...
@derived_field def func(...)...
currently, and in the proposed implementation, this will not let them be detected. But I want to make it so that we *can* do this:
pf.add_field( ... ) and @pf.derived_field, which will both dynamically add fields to existing parameter files. What this means:
* This will not change behavior of @derived_field and add_field, in that they only apply to pfs instantiated *after* the call happens. * There will be a way to dynamically add fields, with field dependencies and all, by using pf attributes. * Adding fields *specific to an output type* will no longer be possible outside of field plugins. This means there will be the ability to add pf-specific and universal fields dynamically in a script, but all fields that are specific to an output type will need to be added in the fields.py file for that frontend. So this means that anything you might have used add_enzo_field or add_orion_field on *in your scripts* (not in anything inside yt/frontends/*/fields.py) will no longer work.
I'm going to be implementing the uncontroversial part of this, which is the dynamic field adding to a "custom" plugin, and unless I hear otherwise I'll continue on this track with the dynamic pf field addition.
-Matt _______________________________________________ yt-dev mailing list yt-dev@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
_______________________________________________ yt-dev mailing list yt-dev@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
Yes, I think it would be impossible or at least very hard to support that
in general.
In the current system, the user can specify the field units in the
add_enzo_field call (for example). Right now a user can do that in a
plugin file or the top of a script. Would something similar be possible in
the new system?
On Thu, Jan 23, 2014 at 2:23 PM, Matthew Turk
On Thu, Jan 23, 2014 at 5:13 PM, Nathan Goldbaum
wrote: I think I have a use case that this design might make more difficult.
Let's say I'm adding new fields to an existing code for some reason (like radiation, chemistry, or MHD). I'd like to make it so users can read in my fields with yt and then make derived fields of their own using my fields.
Yes, that's an ideal use case we need to support.
Under your proposal, would I need to modify the yt source to be able to
read
the fields in properly and assign them units? If so, that seems a bit awkward to me since there would need to be upstream changes to yt that only make sense relative to someone's private fork of a hydro code.
Hmm, reading in units for a field that yt does not know about, that is actually an outstanding problem with units that predates my changes to the field system. I don't know what to do about it.
On Thu, Jan 23, 2014 at 1:34 PM, Matthew Turk
wrote:
Hi all,
The method for adding derived fields has somewhat changed in the unitrefactor of 3.0, and we have the opportunity to make it a bit easier to add new fields -- especially after a parameter file has been created, which previously has been a pain. Below is a brief summary.
The current way we do this is with:
add_field(...)
or
@derived_field def some_function(...
The field system is now plugin based, which means we load plugins for different things and fields are dynamically added. One big advantage is that logic for things like strides and whatnot doesn't need to happen inside the field functions, but it also provides some compartmentalization of things.
I haven't implemented @derived_field and add_field yet in this system, but my plan is to do so by creating a "custom" plugin, to which these will add field definitions, which will always get loaded. But if you do:
pf = ... pf.h.whatever...
@derived_field def func(...)...
currently, and in the proposed implementation, this will not let them be detected. But I want to make it so that we *can* do this:
pf.add_field( ... ) and @pf.derived_field, which will both dynamically add fields to existing parameter files. What this means:
* This will not change behavior of @derived_field and add_field, in that they only apply to pfs instantiated *after* the call happens. * There will be a way to dynamically add fields, with field dependencies and all, by using pf attributes. * Adding fields *specific to an output type* will no longer be possible outside of field plugins. This means there will be the ability to add pf-specific and universal fields dynamically in a script, but all fields that are specific to an output type will need to be added in the fields.py file for that frontend. So this means that anything you might have used add_enzo_field or add_orion_field on *in your scripts* (not in anything inside yt/frontends/*/fields.py) will no longer work.
I'm going to be implementing the uncontroversial part of this, which is the dynamic field adding to a "custom" plugin, and unless I hear otherwise I'll continue on this track with the dynamic pf field addition.
-Matt _______________________________________________ yt-dev mailing list yt-dev@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
_______________________________________________ yt-dev mailing list yt-dev@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
_______________________________________________ yt-dev mailing list yt-dev@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
On Jan 23, 2014 5:36 PM, "Nathan Goldbaum"
Yes, I think it would be impossible or at least very hard to support that
in general.
In the current system, the user can specify the field units in the
add_enzo_field call (for example). Right now a user can do that in a plugin file or the top of a script. Would something similar be possible in the new system?
Yes.
On Thu, Jan 23, 2014 at 2:23 PM, Matthew Turk
wrote:
On Thu, Jan 23, 2014 at 5:13 PM, Nathan Goldbaum
wrote:
I think I have a use case that this design might make more difficult.
Let's say I'm adding new fields to an existing code for some reason (like radiation, chemistry, or MHD). I'd like to make it so users can read in my fields with yt and then make derived fields of their own using my fields.
Yes, that's an ideal use case we need to support.
Under your proposal, would I need to modify the yt source to be able
to read
the fields in properly and assign them units? If so, that seems a bit awkward to me since there would need to be upstream changes to yt that only make sense relative to someone's private fork of a hydro code.
Hmm, reading in units for a field that yt does not know about, that is actually an outstanding problem with units that predates my changes to the field system. I don't know what to do about it.
On Thu, Jan 23, 2014 at 1:34 PM, Matthew Turk
wrote:
Hi all,
The method for adding derived fields has somewhat changed in the unitrefactor of 3.0, and we have the opportunity to make it a bit easier to add new fields -- especially after a parameter file has been created, which previously has been a pain. Below is a brief summary.
The current way we do this is with:
add_field(...)
or
@derived_field def some_function(...
The field system is now plugin based, which means we load plugins for different things and fields are dynamically added. One big advantage is that logic for things like strides and whatnot doesn't need to happen inside the field functions, but it also provides some compartmentalization of things.
I haven't implemented @derived_field and add_field yet in this system, but my plan is to do so by creating a "custom" plugin, to which these will add field definitions, which will always get loaded. But if you do:
pf = ... pf.h.whatever...
@derived_field def func(...)...
currently, and in the proposed implementation, this will not let them be detected. But I want to make it so that we *can* do this:
pf.add_field( ... ) and @pf.derived_field, which will both dynamically add fields to existing parameter files. What this means:
* This will not change behavior of @derived_field and add_field, in that they only apply to pfs instantiated *after* the call happens. * There will be a way to dynamically add fields, with field dependencies and all, by using pf attributes. * Adding fields *specific to an output type* will no longer be possible outside of field plugins. This means there will be the ability to add pf-specific and universal fields dynamically in a script, but all fields that are specific to an output type will need to be added in the fields.py file for that frontend. So this means that anything you might have used add_enzo_field or add_orion_field on *in your scripts* (not in anything inside yt/frontends/*/fields.py) will no longer work.
I'm going to be implementing the uncontroversial part of this, which is the dynamic field adding to a "custom" plugin, and unless I hear otherwise I'll continue on this track with the dynamic pf field addition.
-Matt _______________________________________________ yt-dev mailing list yt-dev@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
_______________________________________________ yt-dev mailing list yt-dev@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
_______________________________________________ yt-dev mailing list yt-dev@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
_______________________________________________ yt-dev mailing list yt-dev@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
On Jan 23, 2014 5:39 PM, "Matthew Turk"
On Jan 23, 2014 5:36 PM, "Nathan Goldbaum"
wrote: Yes, I think it would be impossible or at least very hard to support
In the current system, the user can specify the field units in the
add_enzo_field call (for example). Right now a user can do that in a
that in general. plugin file or the top of a script. Would something similar be possible in the new system?
Yes.
Sorry for replying again too quickly, but this brought up the (obvious in retrospect) possibility of using plugins for frontend-specific fields. So maybe we don't need to worry about this at all, and I shouldn't have written! [bikeshed joke here]
On Thu, Jan 23, 2014 at 2:23 PM, Matthew Turk
On Thu, Jan 23, 2014 at 5:13 PM, Nathan Goldbaum
wrote:
I think I have a use case that this design might make more difficult.
Let's say I'm adding new fields to an existing code for some reason (like radiation, chemistry, or MHD). I'd like to make it so users can read in my fields with yt and then make derived fields of their own using my fields.
Yes, that's an ideal use case we need to support.
Under your proposal, would I need to modify the yt source to be able
to read
the fields in properly and assign them units? If so, that seems a bit awkward to me since there would need to be upstream changes to yt
make sense relative to someone's private fork of a hydro code.
Hmm, reading in units for a field that yt does not know about, that is actually an outstanding problem with units that predates my changes to the field system. I don't know what to do about it.
On Thu, Jan 23, 2014 at 1:34 PM, Matthew Turk
wrote:
Hi all,
The method for adding derived fields has somewhat changed in the unitrefactor of 3.0, and we have the opportunity to make it a bit easier to add new fields -- especially after a parameter file has
been
created, which previously has been a pain. Below is a brief
The current way we do this is with:
add_field(...)
or
@derived_field def some_function(...
The field system is now plugin based, which means we load plugins
for
different things and fields are dynamically added. One big advantage is that logic for things like strides and whatnot doesn't need to happen inside the field functions, but it also provides some compartmentalization of things.
I haven't implemented @derived_field and add_field yet in this system, but my plan is to do so by creating a "custom" plugin, to which
will add field definitions, which will always get loaded. But if you do:
pf = ... pf.h.whatever...
@derived_field def func(...)...
currently, and in the proposed implementation, this will not let
wrote: that only summary. these them
be detected. But I want to make it so that we *can* do this:
pf.add_field( ... ) and @pf.derived_field, which will both dynamically add fields to existing parameter files. What this means:
* This will not change behavior of @derived_field and add_field, in that they only apply to pfs instantiated *after* the call happens. * There will be a way to dynamically add fields, with field dependencies and all, by using pf attributes. * Adding fields *specific to an output type* will no longer be possible outside of field plugins. This means there will be the ability to add pf-specific and universal fields dynamically in a script, but all fields that are specific to an output type will need to be added in the fields.py file for that frontend. So this means that anything you might have used add_enzo_field or add_orion_field on *in your scripts* (not in anything inside yt/frontends/*/fields.py) will no longer work.
I'm going to be implementing the uncontroversial part of this, which is the dynamic field adding to a "custom" plugin, and unless I hear otherwise I'll continue on this track with the dynamic pf field addition.
-Matt _______________________________________________ yt-dev mailing list yt-dev@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
_______________________________________________ yt-dev mailing list yt-dev@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
_______________________________________________ yt-dev mailing list yt-dev@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
_______________________________________________ yt-dev mailing list yt-dev@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
participants (2)
-
Matthew Turk
-
Nathan Goldbaum