Mailman 3 Field Parameters Bug - yt-dev

Field Parameters Bug

Hilary Egan

1 Oct 2014 1 Oct '14

3:22 p.m.

Hi all, I've encountered a bug with this example http://yt-project.org/doc/developing/creating_derived_fields.html#a-more-com... in the docs of getting/setting field parameters of a dataset for use in a derived field. The example works fine for known yt field parameters like bulk_velocity, but if you change every instance of "bulk_velocity" to "my_bulk_velocity" (sample script http://paste.yt-project.org/show/5142/), then it breaks with this error http://paste.yt-project.org/show/5140/. It seems as though this breaks because `has_field_parameter` always returns True, and there is no way to know ahead of time what units a user-defined field parameter will take, so the mock field data can't be set up correctly. Nathan suggested via IRC that this might necessitate creating a way to register user-defined field parameters, but it's probably worth brain-storming about if anyone else has any other suggestions! -Hilary

Attachments:

attachment.html (text/html — 1.1 KB)

Show replies by date

Nathan Goldbaum

1 Oct 1 Oct

3:25 p.m.

On Wed, Oct 1, 2014 at 1:22 PM, Hilary Egan wrote:

...

Hi all,

I've encountered a bug with this example http://yt-project.org/doc/developing/creating_derived_fields.html#a-more-com... in the docs of getting/setting field parameters of a dataset for use in a derived field. The example works fine for known yt field parameters like bulk_velocity, but if you change every instance of "bulk_velocity" to "my_bulk_velocity" (sample script http://paste.yt-project.org/show/5142/), then it breaks with this error http://paste.yt-project.org/show/5140/.

It seems as though this breaks because `has_field_parameter` always returns True

Note that this is specifically has_field_parameter as defined in the field_detector class: https://bitbucket.org/yt_analysis/yt/src/05de9b2f84be6cc8b933f0af75ac211a57a...

...

, and there is no way to know ahead of time what units a user-defined field parameter will take, so the mock field data can't be set up correctly. Nathan suggested via IRC that this might necessitate creating a way to register user-defined field parameters, but it's probably worth brain-storming about if anyone else has any other suggestions!

The main issue with the way field parameters work right now is that we can't know the units ahead of time for user-defined field parameters. We could either have a field parameter registry where users would encode that before loading their data, or we could temporarily disable the unit system during field detection, which I am loathe to do since it will cause bugs.

...

-Hilary

_______________________________________________ yt-dev mailing list yt-dev@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org

Nathan Goldbaum

3:26 p.m.

On Wed, Oct 1, 2014 at 1:25 PM, Nathan Goldbaum wrote:

...

On Wed, Oct 1, 2014 at 1:22 PM, Hilary Egan wrote:

...
Hi all,

I've encountered a bug with this example http://yt-project.org/doc/developing/creating_derived_fields.html#a-more-com... in the docs of getting/setting field parameters of a dataset for use in a derived field. The example works fine for known yt field parameters like bulk_velocity, but if you change every instance of "bulk_velocity" to "my_bulk_velocity" (sample script http://paste.yt-project.org/show/5142/), then it breaks with this error http://paste.yt-project.org/show/5140/.

It seems as though this breaks because `has_field_parameter` always returns True

Note that this is specifically has_field_parameter as defined in the field_detector class:

https://bitbucket.org/yt_analysis/yt/src/05de9b2f84be6cc8b933f0af75ac211a57a...

...
, and there is no way to know ahead of time what units a user-defined field parameter will take, so the mock field data can't be set up correctly. Nathan suggested via IRC that this might necessitate creating a way to register user-defined field parameters, but it's probably worth brain-storming about if anyone else has any other suggestions!

The main issue with the way field parameters work right now is that we can't know the units ahead of time for user-defined field parameters. We could either have a field parameter registry where users would encode that before loading their data, or we could temporarily disable the unit system during field detection, which I am loathe to do since it will cause bugs.

I meant to say "mask bugs" here.

...

...
-Hilary

_______________________________________________ yt-dev mailing list yt-dev@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org

Matthew Turk

2 Oct 2 Oct

7:36 a.m.

Hi Hilary, Maybe we ought to have a registry of known field parameters, affiliated with dimensions. A while back I looked at the known ones, and there were not many in use. It might be relatively straightforward to set up a simple registry. My concern is then that individuals wanting to define their own are going to run into an issue of the behavior changing, not knowing how to fix it, and so on. Alternately, we could use a default parameter in those failing fields. Or, we could add an argument to the add_field call of the dimensions of the field parameter. -Matt On Wed, Oct 1, 2014 at 3:22 PM, Hilary Egan wrote:

...

Hi all,

I've encountered a bug with this example in the docs of getting/setting field parameters of a dataset for use in a derived field. The example works fine for known yt field parameters like bulk_velocity, but if you change every instance of "bulk_velocity" to "my_bulk_velocity" (sample script), then it breaks with this error.

It seems as though this breaks because `has_field_parameter` always returns True, and there is no way to know ahead of time what units a user-defined field parameter will take, so the mock field data can't be set up correctly. Nathan suggested via IRC that this might necessitate creating a way to register user-defined field parameters, but it's probably worth brain-storming about if anyone else has any other suggestions!

-Hilary

_______________________________________________ yt-dev mailing list yt-dev@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org

Cameron Hummels

9:41 a.m.

I like the option of just adding a "dimension" argument to when you declare/add a field_parameter. That retains the freedom of making your own field_parameters, but seems to fix this dimensional problem. Cameron On Thu, Oct 2, 2014 at 5:36 AM, Matthew Turk wrote:

...

Hi Hilary,

Maybe we ought to have a registry of known field parameters, affiliated with dimensions. A while back I looked at the known ones, and there were not many in use. It might be relatively straightforward to set up a simple registry. My concern is then that individuals wanting to define their own are going to run into an issue of the behavior changing, not knowing how to fix it, and so on.

Alternately, we could use a default parameter in those failing fields. Or, we could add an argument to the add_field call of the dimensions of the field parameter.

-Matt

On Wed, Oct 1, 2014 at 3:22 PM, Hilary Egan wrote:

...
Hi all,

I've encountered a bug with this example in the docs of getting/setting field parameters of a dataset for use in a derived field. The example works fine for known yt field parameters like bulk_velocity, but if you change every instance of "bulk_velocity" to "my_bulk_velocity" (sample script), then it breaks with this error.

It seems as though this breaks because `has_field_parameter` always returns True, and there is no way to know ahead of time what units a user-defined field parameter will take, so the mock field data can't be set up correctly. Nathan suggested via IRC that this might necessitate creating a way to register user-defined field parameters, but it's probably worth brain-storming about if anyone else has any other suggestions!

-Hilary

_______________________________________________ yt-dev mailing list yt-dev@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org

_______________________________________________ yt-dev mailing list yt-dev@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org

-- Cameron Hummels Postdoctoral Researcher Steward Observatory University of Arizona http://chummels.org

Matthew Turk

9:42 a.m.

On Thu, Oct 2, 2014 at 9:41 AM, Cameron Hummels wrote:

...

I like the option of just adding a "dimension" argument to when you declare/add a field_parameter. That retains the freedom of making your own field_parameters, but seems to fix this dimensional problem.

We'd also need to add a step where they are added or declared that isn't inside userspace code.

...

Cameron

On Thu, Oct 2, 2014 at 5:36 AM, Matthew Turk wrote:

...
Hi Hilary,

Maybe we ought to have a registry of known field parameters, affiliated with dimensions. A while back I looked at the known ones, and there were not many in use. It might be relatively straightforward to set up a simple registry. My concern is then that individuals wanting to define their own are going to run into an issue of the behavior changing, not knowing how to fix it, and so on.

Alternately, we could use a default parameter in those failing fields. Or, we could add an argument to the add_field call of the dimensions of the field parameter.

-Matt

On Wed, Oct 1, 2014 at 3:22 PM, Hilary Egan wrote:

...
Hi all,

I've encountered a bug with this example in the docs of getting/setting field parameters of a dataset for use in a derived field. The example works fine for known yt field parameters like bulk_velocity, but if you change every instance of "bulk_velocity" to "my_bulk_velocity" (sample script), then it breaks with this error.

It seems as though this breaks because `has_field_parameter` always returns True, and there is no way to know ahead of time what units a user-defined field parameter will take, so the mock field data can't be set up correctly. Nathan suggested via IRC that this might necessitate creating a way to register user-defined field parameters, but it's probably worth brain-storming about if anyone else has any other suggestions!

-Hilary

_______________________________________________ yt-dev mailing list yt-dev@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org

_______________________________________________ yt-dev mailing list yt-dev@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org

-- Cameron Hummels Postdoctoral Researcher Steward Observatory University of Arizona http://chummels.org

_______________________________________________ yt-dev mailing list yt-dev@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org

Cameron Hummels

10:07 a.m.

So we need something akin to adding particle filters? Define the field_parameter and then add the field_parameter? That seems reasonable to me. On Thu, Oct 2, 2014 at 7:42 AM, Matthew Turk wrote:

...

On Thu, Oct 2, 2014 at 9:41 AM, Cameron Hummels wrote:

...
I like the option of just adding a "dimension" argument to when you declare/add a field_parameter. That retains the freedom of making your own field_parameters, but seems to fix this dimensional problem.

We'd also need to add a step where they are added or declared that isn't inside userspace code.

...
Cameron

On Thu, Oct 2, 2014 at 5:36 AM, Matthew Turk

...
...
Hi Hilary,

Maybe we ought to have a registry of known field parameters, affiliated with dimensions. A while back I looked at the known ones, and there were not many in use. It might be relatively straightforward to set up a simple registry. My concern is then that individuals wanting to define their own are going to run into an issue of the behavior changing, not knowing how to fix it, and so on.

Alternately, we could use a default parameter in those failing fields. Or, we could add an argument to the add_field call of the dimensions of the field parameter.

-Matt

On Wed, Oct 1, 2014 at 3:22 PM, Hilary Egan wrote:

...
Hi all,

I've encountered a bug with this example in the docs of

getting/setting

...
...
field parameters of a dataset for use in a derived field. The example works fine for known yt field parameters like bulk_velocity, but if you change every instance of "bulk_velocity" to "my_bulk_velocity" (sample

wrote: script),

...
...
...
then it breaks with this error.

It seems as though this breaks because `has_field_parameter` always returns True, and there is no way to know ahead of time what units a user-defined field parameter will take, so the mock field data can't be set up correctly. Nathan suggested via IRC that this might necessitate creating a way to register user-defined field parameters, but it's probably worth brain-storming about if anyone else has any other suggestions!

-Hilary

_______________________________________________ yt-dev mailing list yt-dev@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org

_______________________________________________ yt-dev mailing list yt-dev@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org

-- Cameron Hummels Postdoctoral Researcher Steward Observatory University of Arizona http://chummels.org

_______________________________________________ yt-dev mailing list yt-dev@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org

_______________________________________________ yt-dev mailing list yt-dev@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org

-- Cameron Hummels Postdoctoral Researcher Steward Observatory University of Arizona http://chummels.org

Matthew Turk

10:20 a.m.

I'd like to avoid that. What I was thinking was more along the lines of adding an argument to get_field_parameter that optionally specified the dimensions. On Oct 2, 2014 10:08 AM, "Cameron Hummels" wrote:

...

So we need something akin to adding particle filters? Define the field_parameter and then add the field_parameter? That seems reasonable to me.

On Thu, Oct 2, 2014 at 7:42 AM, Matthew Turk wrote:

...
On Thu, Oct 2, 2014 at 9:41 AM, Cameron Hummels wrote:

...
I like the option of just adding a "dimension" argument to when you declare/add a field_parameter. That retains the freedom of making your own field_parameters, but seems to fix this dimensional problem.

We'd also need to add a step where they are added or declared that isn't inside userspace code.

...
Cameron

On Thu, Oct 2, 2014 at 5:36 AM, Matthew Turk

...
...
Hi Hilary,

Maybe we ought to have a registry of known field parameters, affiliated with dimensions. A while back I looked at the known ones, and there were not many in use. It might be relatively straightforward to set up a simple registry. My concern is then that individuals wanting to define their own are going to run into an issue of the behavior changing, not knowing how to fix it, and so on.

Alternately, we could use a default parameter in those failing fields. Or, we could add an argument to the add_field call of the dimensions of the field parameter.

-Matt

On Wed, Oct 1, 2014 at 3:22 PM, Hilary Egan

wrote:

...
...
Hi all,

I've encountered a bug with this example in the docs of getting/setting field parameters of a dataset for use in a derived field. The example works fine for known yt field parameters like bulk_velocity, but if you change every instance of "bulk_velocity" to "my_bulk_velocity" (sample

wrote: script),

...
...
...
then it breaks with this error.

It seems as though this breaks because `has_field_parameter` always returns True, and there is no way to know ahead of time what units a user-defined field parameter will take, so the mock field data can't be set up correctly. Nathan suggested via IRC that this might necessitate creating a way to register user-defined field parameters, but it's probably worth brain-storming about if anyone else has any other suggestions!

-Hilary

_______________________________________________ yt-dev mailing list yt-dev@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org

_______________________________________________ yt-dev mailing list yt-dev@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org

-- Cameron Hummels Postdoctoral Researcher Steward Observatory University of Arizona http://chummels.org

_______________________________________________ yt-dev mailing list yt-dev@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org

_______________________________________________ yt-dev mailing list yt-dev@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org

-- Cameron Hummels Postdoctoral Researcher Steward Observatory University of Arizona http://chummels.org

_______________________________________________ yt-dev mailing list yt-dev@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org

Cameron Hummels

10:23 a.m.

I thought that's what I originally suggested. But wouldn't we need to do it in "set_field_parameter" instead? and wouldn't it be a required argument? On Thu, Oct 2, 2014 at 8:20 AM, Matthew Turk wrote:

...

I'd like to avoid that.

What I was thinking was more along the lines of adding an argument to get_field_parameter that optionally specified the dimensions. On Oct 2, 2014 10:08 AM, "Cameron Hummels" wrote:

...
So we need something akin to adding particle filters? Define the field_parameter and then add the field_parameter? That seems reasonable to me.

On Thu, Oct 2, 2014 at 7:42 AM, Matthew Turk wrote:

...
On Thu, Oct 2, 2014 at 9:41 AM, Cameron Hummels wrote:

...
I like the option of just adding a "dimension" argument to when you declare/add a field_parameter. That retains the freedom of making your own field_parameters, but seems to fix this dimensional problem.

We'd also need to add a step where they are added or declared that isn't inside userspace code.

...
Cameron

On Thu, Oct 2, 2014 at 5:36 AM, Matthew Turk

...
...
Hi Hilary,

Maybe we ought to have a registry of known field parameters, affiliated with dimensions. A while back I looked at the known ones, and there were not many in use. It might be relatively straightforward to set up a simple registry. My concern is then that individuals wanting to define their own are going to run into an issue of the behavior changing, not knowing how to fix it, and so on.

Alternately, we could use a default parameter in those failing fields. Or, we could add an argument to the add_field call of the dimensions of the field parameter.

-Matt

On Wed, Oct 1, 2014 at 3:22 PM, Hilary Egan

wrote:

...
...
Hi all,

I've encountered a bug with this example in the docs of getting/setting field parameters of a dataset for use in a derived field. The example works fine for known yt field parameters like bulk_velocity, but if you change every instance of "bulk_velocity" to "my_bulk_velocity" (sample

wrote: script),

...
...
...
then it breaks with this error.

It seems as though this breaks because `has_field_parameter` always returns True, and there is no way to know ahead of time what units a user-defined field parameter will take, so the mock field data can't be set up correctly. Nathan suggested via IRC that this might necessitate creating a way to register user-defined field parameters, but it's probably worth brain-storming about if anyone else has any other suggestions!

-Hilary

_______________________________________________ yt-dev mailing list yt-dev@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org

_______________________________________________ yt-dev mailing list yt-dev@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org

-- Cameron Hummels Postdoctoral Researcher Steward Observatory University of Arizona http://chummels.org

_______________________________________________ yt-dev mailing list yt-dev@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org

_______________________________________________ yt-dev mailing list yt-dev@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org

-- Cameron Hummels Postdoctoral Researcher Steward Observatory University of Arizona http://chummels.org

_______________________________________________ yt-dev mailing list yt-dev@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org

_______________________________________________ yt-dev mailing list yt-dev@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org

-- Cameron Hummels Postdoctoral Researcher Steward Observatory University of Arizona http://chummels.org

Nathan Goldbaum

12:06 p.m.

On Thu, Oct 2, 2014 at 8:23 AM, Cameron Hummels wrote:

...

I thought that's what I originally suggested. But wouldn't we need to do it in "set_field_parameter" instead? and wouldn't it be a required argument?

Since calls to get_field_parameter generally happen inside field definitions and field definitions can happen *before* datasets are loaded, this can't happen inside set_field_parameter, where that would make sense. It would actually *almost* work, but we would need to figure out a way to mock up appropriate units for field parameters during field detection.

...

On Thu, Oct 2, 2014 at 8:20 AM, Matthew Turk wrote:

...
I'd like to avoid that.

What I was thinking was more along the lines of adding an argument to get_field_parameter that optionally specified the dimensions. On Oct 2, 2014 10:08 AM, "Cameron Hummels" wrote:

...
So we need something akin to adding particle filters? Define the field_parameter and then add the field_parameter? That seems reasonable to me.

On Thu, Oct 2, 2014 at 7:42 AM, Matthew Turk wrote:

...
On Thu, Oct 2, 2014 at 9:41 AM, Cameron Hummels wrote:

...
I like the option of just adding a "dimension" argument to when you declare/add a field_parameter. That retains the freedom of making your own field_parameters, but seems to fix this dimensional problem.

We'd also need to add a step where they are added or declared that isn't inside userspace code.

...
Cameron

On Thu, Oct 2, 2014 at 5:36 AM, Matthew Turk

...
...
Hi Hilary,

Maybe we ought to have a registry of known field parameters, affiliated with dimensions. A while back I looked at the known ones, and there were not many in use. It might be relatively straightforward to set up a simple registry. My concern is then that individuals wanting to define their own are going to run into an

issue

...
of the behavior changing, not knowing how to fix it, and so on.

Alternately, we could use a default parameter in those failing fields. Or, we could add an argument to the add_field call of the dimensions of the field parameter.

-Matt

On Wed, Oct 1, 2014 at 3:22 PM, Hilary Egan wrote: > Hi all, > > I've encountered a bug with this example in the docs of getting/setting > field parameters of a dataset for use in a derived field. The example > works > fine for known yt field parameters like bulk_velocity, but if you change > every instance of "bulk_velocity" to "my_bulk_velocity" (sample

wrote: script),

...
...
> then it breaks with this error. > > It seems as though this breaks because `has_field_parameter` always > returns > True, and there is no way to know ahead of time what units a > user-defined > field parameter will take, so the mock field data can't be set up > correctly. > Nathan suggested via IRC that this might necessitate creating a way to > register user-defined field parameters, but it's probably worth > brain-storming about if anyone else has any other suggestions! > > -Hilary > > _______________________________________________ > yt-dev mailing list > yt-dev@lists.spacepope.org > http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org > _______________________________________________ yt-dev mailing list yt-dev@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org

-- Cameron Hummels Postdoctoral Researcher Steward Observatory University of Arizona http://chummels.org

_______________________________________________ yt-dev mailing list yt-dev@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org

_______________________________________________ yt-dev mailing list yt-dev@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org

-- Cameron Hummels Postdoctoral Researcher Steward Observatory University of Arizona http://chummels.org

_______________________________________________ yt-dev mailing list yt-dev@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org

_______________________________________________ yt-dev mailing list yt-dev@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org

-- Cameron Hummels Postdoctoral Researcher Steward Observatory University of Arizona http://chummels.org

_______________________________________________ yt-dev mailing list yt-dev@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org

3493

Age (days ago)

3494

Last active (days ago)

List overview

Download

9 comments

4 participants

participants (4)

Cameron Hummels
Hilary Egan
Matthew Turk
Nathan Goldbaum

Field Parameters Bug

tags

participants (4)