Mailman 3 non-converted fields? - yt-users - python.org

newer
Release of yt-2.5.1

non-converted fields?

older
can I install yt on my laptop

David Collins

28 Mar 2013 28 Mar '13

12:07 p.m.

Hi! Is there a way to get yt to not convert data to CGS? I'm doing some debugging, and its easier to see the code units. I had a work around for this a while ago, but there has been a bunch of code development since then. Thanks! d.

Attachments:

attachment.html (text/html — 362 bytes)

Reply

Sign in to reply online Use email software

Show replies by thread

Matthew Turk

28 Mar 28 Mar

12:09 p.m.

On Thu, Mar 28, 2013 at 3:07 PM, David Collins wrote:

Hi!

Is there a way to get yt to not convert data to CGS? I'm doing some debugging, and its easier to see the code units.

Read the data directly. pf.h.io._read_data_set(grid, field)

I had a work around for this a while ago, but there has been a bunch of code development since then.

What was your old way?

Thanks! d.

_______________________________________________ yt-users mailing list yt-users@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org

Reply

Sign in to reply online Use email software

David Collins

12:18 p.m.

That reads a single grid from the hierarchy? Can I get yt to do projections and whatnot in those units? Previously I had subclassed EnzoStaticOutput with customized conversion factors. It's a bit outdated. Thanks! d. On Thu, Mar 28, 2013 at 1:09 PM, Matthew Turk wrote:

On Thu, Mar 28, 2013 at 3:07 PM, David Collins wrote:

...
Hi!

Is there a way to get yt to not convert data to CGS? I'm doing some debugging, and its easier to see the code units.

Read the data directly.

pf.h.io._read_data_set(grid, field)

...
I had a work around for this a while ago, but there has been a bunch of

code

...
development since then.

What was your old way?

...
Thanks! d.

_______________________________________________ yt-users mailing list yt-users@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org

_______________________________________________ yt-users mailing list yt-users@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org

Reply

Sign in to reply online Use email software

Nathan Goldbaum

12:21 p.m.

Hi David, Another option would be to temporarily define a new field that has a null or trivial conversion function. This field could live in your ~/.yt/my_plugins.py file so no need to touch the yt codebase. -Nathan On Mar 28, 2013, at 12:18 PM, David Collins wrote:

That reads a single grid from the hierarchy? Can I get yt to do projections and whatnot in those units?

Previously I had subclassed EnzoStaticOutput with customized conversion factors. It's a bit outdated.

Thanks! d.

On Thu, Mar 28, 2013 at 1:09 PM, Matthew Turk wrote: On Thu, Mar 28, 2013 at 3:07 PM, David Collins wrote:

...
Hi!

Is there a way to get yt to not convert data to CGS? I'm doing some debugging, and its easier to see the code units.

Read the data directly.

pf.h.io._read_data_set(grid, field)

...
I had a work around for this a while ago, but there has been a bunch of code development since then.

What was your old way?

...
Thanks! d.

_______________________________________________ yt-users mailing list yt-users@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org

_______________________________________________ yt-users mailing list yt-users@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org

_______________________________________________ yt-users mailing list yt-users@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org

Reply

Sign in to reply online Use email software

Matthew Turk

1:05 p.m.

Nathan's suggestion is a good one, as well. In principle I don't know why the conversion_factors option to the Enzo Static Output wouldn't work for this as well. If we had your subclass or some type of test incorporated into yt, we would be able to verify that this functionality that you rely on continues to work. It can be difficult to anticipate all use cases for the code, but this seems like one that could be valuable. On Thu, Mar 28, 2013 at 3:21 PM, Nathan Goldbaum wrote:

Hi David,

Another option would be to temporarily define a new field that has a null or trivial conversion function.

This field could live in your ~/.yt/my_plugins.py file so no need to touch the yt codebase.

-Nathan

On Mar 28, 2013, at 12:18 PM, David Collins wrote:

That reads a single grid from the hierarchy? Can I get yt to do projections and whatnot in those units?

Previously I had subclassed EnzoStaticOutput with customized conversion factors. It's a bit outdated.

Thanks! d.

On Thu, Mar 28, 2013 at 1:09 PM, Matthew Turk wrote:

...
On Thu, Mar 28, 2013 at 3:07 PM, David Collins wrote:

...
Hi!

Is there a way to get yt to not convert data to CGS? I'm doing some debugging, and its easier to see the code units.

Read the data directly.

pf.h.io._read_data_set(grid, field)

...
I had a work around for this a while ago, but there has been a bunch of code development since then.

What was your old way?

...
Thanks! d.

_______________________________________________ yt-users mailing list yt-users@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org

_______________________________________________ yt-users mailing list yt-users@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org

_______________________________________________ yt-users mailing list yt-users@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org

_______________________________________________ yt-users mailing list yt-users@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org

Reply

Sign in to reply online Use email software

David Collins

1:18 p.m.

I tried Nathan's suggestion, but I'm still getting fields in CGS. I did the following, and just ran it in a script (not messing with the plugins for now). Is there anything else I need to do? What I think is happening is that in D2, data['Density'] is already converted to cgs when it gets returned by D2. Is there an easy way to get the density conversion function that's getting used, so I can have the converter return the inverse? I added the ValidateGridType in order to try to force it to be read before the convert, that didn't work. def D2(field,data): return data['Density'] def no_convert(data): return 1 add_field('D2',function=D2,convert_function=no_convert, validators=[ValidateGridType()])

Nathan's suggestion is a good one, as well. In principle I don't know why the conversion_factors option to the Enzo Static Output wouldn't work for this as well.

I dropped this method a while ago when other yt things started to break. I was doing something like this, but when I do it other things like "load" stop working. Even just creating this class, not actually doing anything with it, causes load to fail silently. class EnzoStaticOutputMHD(EnzoStaticOutput): def __init__(self,*args,**kwargs): Conversion=2.5e-9/4.32e-7 EnzoStaticOutput.__init__(self,*args,conversion_override={ 'MagneticField_C_1': Conversion, 'MagneticField_C_2': Conversion, 'MagneticField_C_3': Conversion, 'MagneticField_F_1': Conversion, 'MagneticField_F_3': Conversion, 'MagneticField_F_2': Conversion}, **kwargs)

If we had your subclass or some type of test incorporated into yt, we would be able to verify that this functionality that you rely on continues to work. It can be difficult to anticipate all use cases for the code, but this seems like one that could be valuable.

On Thu, Mar 28, 2013 at 3:21 PM, Nathan Goldbaum wrote:

...
Hi David,

Another option would be to temporarily define a new field that has a null or trivial conversion function.

This field could live in your ~/.yt/my_plugins.py file so no need to touch the yt codebase.

-Nathan

On Mar 28, 2013, at 12:18 PM, David Collins wrote:

That reads a single grid from the hierarchy? Can I get yt to do projections and whatnot in those units?

Previously I had subclassed EnzoStaticOutput with customized conversion factors. It's a bit outdated.

Thanks! d.

On Thu, Mar 28, 2013 at 1:09 PM, Matthew Turk wrote:

...
On Thu, Mar 28, 2013 at 3:07 PM, David Collins wrote:

...
Hi!

Is there a way to get yt to not convert data to CGS? I'm doing some debugging, and its easier to see the code units.

Read the data directly.

pf.h.io._read_data_set(grid, field)

...
I had a work around for this a while ago, but there has been a bunch of code development since then.

What was your old way?

...
Thanks! d.

_______________________________________________ yt-users mailing list yt-users@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org

_______________________________________________ yt-users mailing list yt-users@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org

_______________________________________________ yt-users mailing list yt-users@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org

_______________________________________________ yt-users mailing list yt-users@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org

_______________________________________________ yt-users mailing list yt-users@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org

-- Sent from my computer.

Reply

Sign in to reply online Use email software

Nathan Goldbaum

1:24 p.m.

Hi Dave, You can see the conversion factors yt applies by inspecting pf.conversion_factors. -Nathan On Mar 28, 2013, at 1:18 PM, David Collins wrote:

I tried Nathan's suggestion, but I'm still getting fields in CGS. I did the following, and just ran it in a script (not messing with the plugins for now). Is there anything else I need to do? What I think is happening is that in D2, data['Density'] is already converted to cgs when it gets returned by D2. Is there an easy way to get the density conversion function that's getting used, so I can have the converter return the inverse? I added the ValidateGridType in order to try to force it to be read before the convert, that didn't work.

def D2(field,data): return data['Density'] def no_convert(data): return 1 add_field('D2',function=D2,convert_function=no_convert, validators=[ValidateGridType()])

...
Nathan's suggestion is a good one, as well. In principle I don't know why the conversion_factors option to the Enzo Static Output wouldn't work for this as well.

I dropped this method a while ago when other yt things started to break. I was doing something like this, but when I do it other things like "load" stop working. Even just creating this class, not actually doing anything with it, causes load to fail silently.

class EnzoStaticOutputMHD(EnzoStaticOutput): def __init__(self,*args,**kwargs): Conversion=2.5e-9/4.32e-7 EnzoStaticOutput.__init__(self,*args,conversion_override={

'MagneticField_C_1': Conversion,

'MagneticField_C_2': Conversion,

'MagneticField_C_3': Conversion,

'MagneticField_F_1': Conversion,

'MagneticField_F_3': Conversion,

'MagneticField_F_2': Conversion}, **kwargs)

...
If we had your subclass or some type of test incorporated into yt, we would be able to verify that this functionality that you rely on continues to work. It can be difficult to anticipate all use cases for the code, but this seems like one that could be valuable.

On Thu, Mar 28, 2013 at 3:21 PM, Nathan Goldbaum wrote:

...
Hi David,

Another option would be to temporarily define a new field that has a null or trivial conversion function.

This field could live in your ~/.yt/my_plugins.py file so no need to touch the yt codebase.

-Nathan

On Mar 28, 2013, at 12:18 PM, David Collins wrote:

That reads a single grid from the hierarchy? Can I get yt to do projections and whatnot in those units?

Previously I had subclassed EnzoStaticOutput with customized conversion factors. It's a bit outdated.

Thanks! d.

On Thu, Mar 28, 2013 at 1:09 PM, Matthew Turk wrote:

...
On Thu, Mar 28, 2013 at 3:07 PM, David Collins wrote:

...
Hi!

Is there a way to get yt to not convert data to CGS? I'm doing some debugging, and its easier to see the code units.

Read the data directly.

pf.h.io._read_data_set(grid, field)

...
I had a work around for this a while ago, but there has been a bunch of code development since then.

What was your old way?

...
Thanks! d.

_______________________________________________ yt-users mailing list yt-users@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org

_______________________________________________ yt-users mailing list yt-users@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org

_______________________________________________ yt-users mailing list yt-users@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org

_______________________________________________ yt-users mailing list yt-users@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org

_______________________________________________ yt-users mailing list yt-users@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org

-- Sent from my computer. _______________________________________________ yt-users mailing list yt-users@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org

Reply

Sign in to reply online Use email software

Matthew Turk

1:29 p.m.

On Thu, Mar 28, 2013 at 4:18 PM, David Collins wrote:

I tried Nathan's suggestion, but I'm still getting fields in CGS. I did the following, and just ran it in a script (not messing with the plugins for now). Is there anything else I need to do? What I think is happening is that in D2, data['Density'] is already converted to cgs when it gets returned by D2. Is there an easy way to get the density conversion function that's getting used, so I can have the converter return the inverse? I added the ValidateGridType in order to try to force it to be read before the convert, that didn't work.

def D2(field,data): return data['Density'] def no_convert(data): return 1 add_field('D2',function=D2,convert_function=no_convert, validators=[ValidateGridType()])

...
Nathan's suggestion is a good one, as well. In principle I don't know why the conversion_factors option to the Enzo Static Output wouldn't work for this as well.

I dropped this method a while ago when other yt things started to break.

Break how?

I was doing something like this, but when I do it other things like "load" stop working. Even just creating this class, not actually doing anything with it, causes load to fail silently.

I suspect that what's happening is that it's probably reporting via the log (loglevel of error) that too many _is_valid routines succeeded, so it could not distinguish. Everything for Enzo is mediated by two functions: parse_parameter_file, which indirectly modifies conversion_factors when it stumbles across a Units entry in the parameter file (as long as it's not TemperatureUnits). This is overriden by conversion_override. However, note that this occurs *before* parsing the datalabels, which looks like an error to me, as that will then subsequently override conversion_factors again. Following this, _set_units is called. This will determine it's cosmological, and if so, call _setup_comoving_units. If the parameter LengthUnit is in the parameter file it calls _setup_getunits_units. If neither of those two things happens, it calls setup_nounits_units. This sets everything to 1.0, except for the length/time conversions, where everything is set to 1.0 cm == 1.0 in code units and 1.0 s == 1.0 in code units. So my guess is your dataset has a #CGSConversionFactor for Density.

class EnzoStaticOutputMHD(EnzoStaticOutput): def __init__(self,*args,**kwargs): Conversion=2.5e-9/4.32e-7 EnzoStaticOutput.__init__(self,*args,conversion_override={

'MagneticField_C_1': Conversion,

'MagneticField_C_2': Conversion,

'MagneticField_C_3': Conversion,

'MagneticField_F_1': Conversion,

'MagneticField_F_3': Conversion,

'MagneticField_F_2': Conversion}, **kwargs)

...
If we had your subclass or some type of test incorporated into yt, we would be able to verify that this functionality that you rely on continues to work. It can be difficult to anticipate all use cases for the code, but this seems like one that could be valuable.

On Thu, Mar 28, 2013 at 3:21 PM, Nathan Goldbaum wrote:

...
Hi David,

Another option would be to temporarily define a new field that has a null or trivial conversion function.

This field could live in your ~/.yt/my_plugins.py file so no need to touch the yt codebase.

-Nathan

On Mar 28, 2013, at 12:18 PM, David Collins wrote:

That reads a single grid from the hierarchy? Can I get yt to do projections and whatnot in those units?

Previously I had subclassed EnzoStaticOutput with customized conversion factors. It's a bit outdated.

Thanks! d.

On Thu, Mar 28, 2013 at 1:09 PM, Matthew Turk wrote:

...
On Thu, Mar 28, 2013 at 3:07 PM, David Collins wrote:

...
Hi!

Is there a way to get yt to not convert data to CGS? I'm doing some debugging, and its easier to see the code units.

Read the data directly.

pf.h.io._read_data_set(grid, field)

...
I had a work around for this a while ago, but there has been a bunch of code development since then.

What was your old way?

...
Thanks! d.

_______________________________________________ yt-users mailing list yt-users@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org

_______________________________________________ yt-users mailing list yt-users@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org

_______________________________________________ yt-users mailing list yt-users@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org

_______________________________________________ yt-users mailing list yt-users@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org

_______________________________________________ yt-users mailing list yt-users@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org

-- Sent from my computer. _______________________________________________ yt-users mailing list yt-users@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org

Reply

Sign in to reply online Use email software

David Collins

1:33 p.m.

Ah, the conversion factors change with time, so that "inverse conversion factor" idea won't work. Is there a way to get to the grid, through add_field, before the convsersion is applied? On Thu, Mar 28, 2013 at 2:24 PM, Nathan Goldbaum wrote:

Hi Dave,

You can see the conversion factors yt applies by inspecting pf.conversion_factors.

-Nathan

On Mar 28, 2013, at 1:18 PM, David Collins wrote:

...
I tried Nathan's suggestion, but I'm still getting fields in CGS. I did the following, and just ran it in a script (not messing with the plugins for now). Is there anything else I need to do? What I think is happening is that in D2, data['Density'] is already converted to cgs when it gets returned by D2. Is there an easy way to get the density conversion function that's getting used, so I can have the converter return the inverse? I added the ValidateGridType in order to try to force it to be read before the convert, that didn't work.

def D2(field,data): return data['Density'] def no_convert(data): return 1 add_field('D2',function=D2,convert_function=no_convert, validators=[ValidateGridType()])

...
Nathan's suggestion is a good one, as well. In principle I don't know why the conversion_factors option to the Enzo Static Output wouldn't work for this as well.

I dropped this method a while ago when other yt things started to break. I was doing something like this, but when I do it other things like "load" stop working. Even just creating this class, not actually doing anything with it, causes load to fail silently.

class EnzoStaticOutputMHD(EnzoStaticOutput): def __init__(self,*args,**kwargs): Conversion=2.5e-9/4.32e-7 EnzoStaticOutput.__init__(self,*args,conversion_override={

'MagneticField_C_1': Conversion,

'MagneticField_C_2': Conversion,

'MagneticField_C_3': Conversion,

'MagneticField_F_1': Conversion,

'MagneticField_F_3': Conversion,

'MagneticField_F_2': Conversion}, **kwargs)

...
If we had your subclass or some type of test incorporated into yt, we would be able to verify that this functionality that you rely on continues to work. It can be difficult to anticipate all use cases for the code, but this seems like one that could be valuable.

On Thu, Mar 28, 2013 at 3:21 PM, Nathan Goldbaum

...
...
Hi David,

Another option would be to temporarily define a new field that has a null or trivial conversion function.

This field could live in your ~/.yt/my_plugins.py file so no need to touch the yt codebase.

-Nathan

On Mar 28, 2013, at 12:18 PM, David Collins wrote:

That reads a single grid from the hierarchy? Can I get yt to do

wrote: projections

...
...
and whatnot in those units?

Previously I had subclassed EnzoStaticOutput with customized conversion factors. It's a bit outdated.

Thanks! d.

On Thu, Mar 28, 2013 at 1:09 PM, Matthew Turk wrote:

...
On Thu, Mar 28, 2013 at 3:07 PM, David Collins <

dcollins4096@gmail.com>

...
wrote:

...
Hi!

Is there a way to get yt to not convert data to CGS? I'm doing some debugging, and its easier to see the code units.

Read the data directly.

pf.h.io._read_data_set(grid, field)

...
I had a work around for this a while ago, but there has been a bunch

of

...
code development since then.

What was your old way?

...
Thanks! d.

_______________________________________________ yt-users mailing list yt-users@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org

_______________________________________________ yt-users mailing list yt-users@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org

_______________________________________________ yt-users mailing list yt-users@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org

_______________________________________________ yt-users mailing list yt-users@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org

_______________________________________________ yt-users mailing list yt-users@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org

-- Sent from my computer. _______________________________________________ yt-users mailing list yt-users@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org

_______________________________________________ yt-users mailing list yt-users@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org

Reply

Sign in to reply online Use email software

David Collins

1:37 p.m.

So my guess is your dataset has a #CGSConversionFactor for Density.

That's right, this is a cosmology sim. Is there a way to hijack setup_comoving_units to get it to not do anything? d.

Reply

Sign in to reply online Use email software

Matthew Turk

1:45 p.m.

On Thu, Mar 28, 2013 at 4:37 PM, David Collins wrote:

...
So my guess is your dataset has a #CGSConversionFactor for Density.

That's right, this is a cosmology sim. Is there a way to hijack setup_comoving_units to get it to not do anything?

I don't know. Just change conversion_factors after you've instantiated the pf. I do believe in ensuring yt can support debugging of raw values, which is where reading the datasets and the grids comes in, but I don't think it's necessarily correct that we need to overlay on enough complexity to make this any easier than just dropping the conversion factors after the parameter file is instantiated. I mean, the dataset says what its units are, and yt is set up to know how to interpret that. 1.59542869378e-32 2.5694638476e-25 0.00631190510467 101654.257812 These are the two outputs for this script: http://paste.yt-project.org/show/3316/ with the conversion_factors line commented out and uncommented, respectively. Note that conversion_factors should not be considered final API, as the YTEP governing units will likely replace it with something else. Draft of YTEP: https://bitbucket.org/yt_analysis/ytep/pull-request/13/created-new-ytep-for-... -Matt

d. _______________________________________________ yt-users mailing list yt-users@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org

Reply

Sign in to reply online Use email software

David Collins

1:54 p.m.

http://paste.yt-project.org/show/3316/

Ah, that's perfect. Not sure why that didn't occur to me first. Thanks! d.

Reply

Sign in to reply online Use email software

4044

Age (days ago)

4044

Last active (days ago)

Download

11 comments

4 participants

tags

participants (4)

David Collins
David Collins
Matthew Turk
Nathan Goldbaum