Hi Brian and Nick, Can you share with us the errors you're seeing? It's likely that no one has tried to do what you're doing before. To get it to work, it might just be necessary to make some adjustments to the absorption spectrum machinery in the case when we detect a non-cosmological dataset. That will require making some modifications to yt, but that's ok! It looks like the absorption spectrum analysis module *has* seen some updates for yt-3.0, so you may also want to try again in the latest version of the codebase. I haven't personally used it - it looks like Hillary Egan was the last person to touch the code. -Nathan On Mon Dec 08 2014 at 1:52:15 PM Brian O'Shea wrote:

Hi folks,

Just to send a quick followup on this: Nick is referring specifically to the *absorption spectrum* generator, and he is trying to make spectra of idealized, non-cosmological simulations (think a blob in an ambient medium) that has no redshift as a simulation parameter. The question is, how does one do this?

Cheers, Brian

On Mon, Dec 8, 2014 at 11:59 AM, Nicholas Earl wrote:

...

Hey everyone,

Is there a way to fake the cosmological parameters that the synthetic spectrum generator looks for in a simulation? Or rather, a "sanctioned" way? Having the package reference hardcoded values when the parameters themselves are not found yields other missing fields like 'creation_time'. I am wondering if this is a consequence of the simulation not being inherently cosmological?

I am using yt 2.x currently, as I'm aware that there seems to be an issue with the generator in the 3.0 release.

Thanks for your time, Nick

_______________________________________________ yt-users mailing list yt-users@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org

_______________________________________________ yt-users mailing list yt-users@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org

Thanks, everyone, for the responses. It is as Brian has said: I am dealing specifically with the case of an absorption spectrum generator. The simulation is indeed idealized and non-cosmological. Attempting to load the data and pass it to SpectrumBuilder yields the 'CosmologyInitialRedshift' KeyError seen here http://paste.yt-project.org/show/5248/. Editing the sfr_spectrum.py file to accept hard-coded values (for the omegas and initial redshift) yields a second KeyError seen here http://paste.yt-project.org/show/5249/. As per Nathan's suggestion, I will give the yt-3.0 version of the generator a go in the mean time. Thanks again for your guys' attention, Nick On Mon Dec 08 2014 at 5:19:21 PM Cameron Hummels wrote:

I've used the code on individual blobs of gas in an ambient medium with short sightlines, but I used it on sim outputs that were cosmological. I think it would be relatively straightforward to generalize the code to allow one to *set* the cosmological parameters, instead of reading those values from the dataset. I think Britton would be the main person to answer these questions, but I'll take a look at it this evening and see how hard this might be.

Cameron

On Mon, Dec 8, 2014 at 2:59 PM, Nathan Goldbaum wrote:

...

Hi Brian and Nick,

Can you share with us the errors you're seeing?

It's likely that no one has tried to do what you're doing before. To get it to work, it might just be necessary to make some adjustments to the absorption spectrum machinery in the case when we detect a non-cosmological dataset. That will require making some modifications to yt, but that's ok!

It looks like the absorption spectrum analysis module *has* seen some updates for yt-3.0, so you may also want to try again in the latest version of the codebase. I haven't personally used it - it looks like Hillary Egan was the last person to touch the code.

-Nathan

On Mon Dec 08 2014 at 1:52:15 PM Brian O'Shea wrote:

...

Hi folks,

Just to send a quick followup on this: Nick is referring specifically to the *absorption spectrum* generator, and he is trying to make spectra of idealized, non-cosmological simulations (think a blob in an ambient medium) that has no redshift as a simulation parameter. The question is, how does one do this?

Cheers, Brian

On Mon, Dec 8, 2014 at 11:59 AM, Nicholas Earl wrote:

...

Hey everyone,

Is there a way to fake the cosmological parameters that the synthetic spectrum generator looks for in a simulation? Or rather, a "sanctioned" way? Having the package reference hardcoded values when the parameters themselves are not found yields other missing fields like 'creation_time'. I am wondering if this is a consequence of the simulation not being inherently cosmological?

I am using yt 2.x currently, as I'm aware that there seems to be an issue with the generator in the 3.0 release.

Thanks for your time, Nick

_______________________________________________ yt-users mailing list yt-users@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org

_______________________________________________ yt-users mailing list yt-users@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org

_______________________________________________ yt-users mailing list yt-users@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org

-- Cameron Hummels Postdoctoral Researcher Steward Observatory University of Arizona http://chummels.org _______________________________________________ yt-users mailing list yt-users@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org

Hey all, Britton: thanks a lot for the run down -- I really appreciate it! I seem to be running into an issue, however: the spectrum builder doesn't want to give me anything other than a flat line for the flux. Testing the min/max values the ray returns seems to indicate everything is pretty much in working order; the temperature, densities, etc seem reasonable. I'm hoping perhaps you guys may be able to spot something I've missed. For reference, here's is the current script I'm using to try and generate the spectrum: https://gist.github.com/nmearl/85c70d6f8dc8e9e092a6. And here is a single output of the test data I'm playing with: https://www.dropbox.com/s/ldq4xyfnbzbbcj0/test_run.tar.gz?dl=0. Thanks, Nick On Tue Dec 09 2014 at 9:51:00 AM Britton Smith wrote:

Hi Nick,

There are a number of places in the LightRay tool that assume that the simulation data being used is cosmological. This is certainly something that could and should be improved, but it will be a non-trivial upgrade. Your best bet is to bypass the LightRay tool and manually create files that can be used by the absorption spectrum generator. This should be quite straightforward.

The LightRay tool works by creating ray data containers, gathering field data for all the cells intersected by the ray, and saving them to an hdf5 file. If you have some dataset loaded, you can do:

import yt ds = yt.load(...) ray = ds.ray(start_position, end_position)

Then, you can get fields you want to use in your spectrum. You will want the density fields for whatever species you want to make an absorption line for. You will also want to get the temperature field, since that controls the width of the line. You will also need to know the physical path length of the ray through each cell. This is multiplied by the density fields to get column densities that are then used to generate the Voigt profiles that make the absorption lines. The ray object has an extra field called "dts" which is that same length normalized such that the total sum over the ray is 1.0. Just multiply that by the physical length of the ray and you'll have what you need.

If you want the lines to be shifted via the Doppler effect, you will have to calculate the line of sight velocity, which is just the dot product of the velocity field with your line of sight vector.

Once you have all of these arrays created for your ray, just write them out to an hdf5 file. Name the line of sight velocity field "los_velocity" and the path length field "dl" (lowercase L). The absorption spectrum generator will also look for a redshift field, so just write out an array of zeros to the hdf5 file with the name "redshift". If you have made the hdf5 file in this way, you can feed it to the absorption spectrum generator and it will work.

To summarize, you need to write out the following fields: - densities (for each species you want to make lines for) - temperature - line of sight velocity "los_velocity" (optional) - path length "dl" - array of zeros called "redshift"

Finally, I highly suggest you do this in yt-3.0. While there may be bugs (although I believe all are fixed as of now), it will be much easier for us to fix them in 3.0. The code has changed significantly since version 2.x, so porting fixes back is not easy.

Britton

On Tue, Dec 9, 2014 at 2:06 AM, Nicholas Earl wrote:

...

Thanks, everyone, for the responses.

It is as Brian has said: I am dealing specifically with the case of an absorption spectrum generator. The simulation is indeed idealized and non-cosmological.

Attempting to load the data and pass it to SpectrumBuilder yields the 'CosmologyInitialRedshift' KeyError seen here http://paste.yt-project.org/show/5248/. Editing the sfr_spectrum.py file to accept hard-coded values (for the omegas and initial redshift) yields a second KeyError seen here http://paste.yt-project.org/show/5249/.

As per Nathan's suggestion, I will give the yt-3.0 version of the generator a go in the mean time.

Thanks again for your guys' attention, Nick

On Mon Dec 08 2014 at 5:19:21 PM Cameron Hummels wrote:

...

I've used the code on individual blobs of gas in an ambient medium with short sightlines, but I used it on sim outputs that were cosmological. I think it would be relatively straightforward to generalize the code to allow one to *set* the cosmological parameters, instead of reading those values from the dataset. I think Britton would be the main person to answer these questions, but I'll take a look at it this evening and see how hard this might be.

Cameron

On Mon, Dec 8, 2014 at 2:59 PM, Nathan Goldbaum wrote:

...

Hi Brian and Nick,

Can you share with us the errors you're seeing?

It's likely that no one has tried to do what you're doing before. To get it to work, it might just be necessary to make some adjustments to the absorption spectrum machinery in the case when we detect a non-cosmological dataset. That will require making some modifications to yt, but that's ok!

It looks like the absorption spectrum analysis module *has* seen some updates for yt-3.0, so you may also want to try again in the latest version of the codebase. I haven't personally used it - it looks like Hillary Egan was the last person to touch the code.

-Nathan

On Mon Dec 08 2014 at 1:52:15 PM Brian O'Shea wrote:

...

Hi folks,

Just to send a quick followup on this: Nick is referring specifically to the *absorption spectrum* generator, and he is trying to make spectra of idealized, non-cosmological simulations (think a blob in an ambient medium) that has no redshift as a simulation parameter. The question is, how does one do this?

Cheers, Brian

On Mon, Dec 8, 2014 at 11:59 AM, Nicholas Earl wrote:

...

Hey everyone,

Is there a way to fake the cosmological parameters that the synthetic spectrum generator looks for in a simulation? Or rather, a "sanctioned" way? Having the package reference hardcoded values when the parameters themselves are not found yields other missing fields like 'creation_time'. I am wondering if this is a consequence of the simulation not being inherently cosmological?

I am using yt 2.x currently, as I'm aware that there seems to be an issue with the generator in the 3.0 release.

Thanks for your time, Nick

_______________________________________________ yt-users mailing list yt-users@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org

_______________________________________________ yt-users mailing list yt-users@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org

_______________________________________________ yt-users mailing list yt-users@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org

-- Cameron Hummels Postdoctoral Researcher Steward Observatory University of Arizona http://chummels.org _______________________________________________ yt-users mailing list yt-users@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org

_______________________________________________ yt-users mailing list yt-users@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org

_______________________________________________ yt-users mailing list yt-users@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org

Well done! This is excellent, it works great. I'm glad you were able to work around some of my naive assumptions -- sorry about those. Beautiful absorption lines, indeed! Thanks for taking the time to help out like this -- you rock! On Fri Dec 12 2014 at 8:10:48 AM Britton Smith wrote:

Hi Nick,

Ok, I think we're now sorted. There was a combination of issues at play here.

In your script, some of the fields were not in the correct units, so the column densities were not coming out right. Since yt is now unit aware, one has to make sure that field arrays have the correct units. For example, the density and dl arrays here were being written in code units while the spectrum generator was expecting CGS. Additionally, I told you to write out the density field, when I really mean number density. Sorry about that!

On top of that, it appears that the absorption spectrum generator itself was not truly yt-3.0 ready. I apologize for that as I was the one who signed off on it. I have issued a pull request that makes things work for real. You can check it out here:

https://bitbucket.org/yt_analysis/yt/pull-request/1356/bugfix-updating-absor...

Finally, after fixing all of this, I was still getting a spectrum free of absorption. I determined that at the spectral resolution given, all of the line components were unresolved, meaning that each individual line was less than the size of one spectral element. Previously, the behavior was to quietly ignore these components, which I can see now is the wrong thing to do. I have added a check for this, and when encountered, the code will issue a warning and tell you to increase the resolution.

After all this, you now have a beautiful absorption line: [image: Inline image 1]

Here is a copy of your script with the few changes I made.

http://paste.yt-project.org/show/5258/

That and the changes from the pull request should take care of it. Sorry for inconvenience!

Britton

On Thu, Dec 11, 2014 at 1:51 AM, Nicholas Earl wrote:

...

Hey all,

Britton: thanks a lot for the run down -- I really appreciate it! I seem to be running into an issue, however: the spectrum builder doesn't want to give me anything other than a flat line for the flux.

Testing the min/max values the ray returns seems to indicate everything is pretty much in working order; the temperature, densities, etc seem reasonable. I'm hoping perhaps you guys may be able to spot something I've missed.

For reference, here's is the current script I'm using to try and generate the spectrum: https://gist.github.com/nmearl/85c70d6f8dc8e9e092a6. And here is a single output of the test data I'm playing with: https://www.dropbox.com/s/ldq4xyfnbzbbcj0/test_run.tar.gz?dl=0.

Thanks, Nick

On Tue Dec 09 2014 at 9:51:00 AM Britton Smith wrote:

...

Hi Nick,

There are a number of places in the LightRay tool that assume that the simulation data being used is cosmological. This is certainly something that could and should be improved, but it will be a non-trivial upgrade. Your best bet is to bypass the LightRay tool and manually create files that can be used by the absorption spectrum generator. This should be quite straightforward.

The LightRay tool works by creating ray data containers, gathering field data for all the cells intersected by the ray, and saving them to an hdf5 file. If you have some dataset loaded, you can do:

import yt ds = yt.load(...) ray = ds.ray(start_position, end_position)

Then, you can get fields you want to use in your spectrum. You will want the density fields for whatever species you want to make an absorption line for. You will also want to get the temperature field, since that controls the width of the line. You will also need to know the physical path length of the ray through each cell. This is multiplied by the density fields to get column densities that are then used to generate the Voigt profiles that make the absorption lines. The ray object has an extra field called "dts" which is that same length normalized such that the total sum over the ray is 1.0. Just multiply that by the physical length of the ray and you'll have what you need.

If you want the lines to be shifted via the Doppler effect, you will have to calculate the line of sight velocity, which is just the dot product of the velocity field with your line of sight vector.

Once you have all of these arrays created for your ray, just write them out to an hdf5 file. Name the line of sight velocity field "los_velocity" and the path length field "dl" (lowercase L). The absorption spectrum generator will also look for a redshift field, so just write out an array of zeros to the hdf5 file with the name "redshift". If you have made the hdf5 file in this way, you can feed it to the absorption spectrum generator and it will work.

To summarize, you need to write out the following fields: - densities (for each species you want to make lines for) - temperature - line of sight velocity "los_velocity" (optional) - path length "dl" - array of zeros called "redshift"

Finally, I highly suggest you do this in yt-3.0. While there may be bugs (although I believe all are fixed as of now), it will be much easier for us to fix them in 3.0. The code has changed significantly since version 2.x, so porting fixes back is not easy.

Britton

On Tue, Dec 9, 2014 at 2:06 AM, Nicholas Earl wrote:

...

Thanks, everyone, for the responses.

It is as Brian has said: I am dealing specifically with the case of an absorption spectrum generator. The simulation is indeed idealized and non-cosmological.

Attempting to load the data and pass it to SpectrumBuilder yields the 'CosmologyInitialRedshift' KeyError seen here http://paste.yt-project.org/show/5248/. Editing the sfr_spectrum.py file to accept hard-coded values (for the omegas and initial redshift) yields a second KeyError seen here http://paste.yt-project.org/show/5249/.

As per Nathan's suggestion, I will give the yt-3.0 version of the generator a go in the mean time.

Thanks again for your guys' attention, Nick

On Mon Dec 08 2014 at 5:19:21 PM Cameron Hummels wrote:

...

I've used the code on individual blobs of gas in an ambient medium with short sightlines, but I used it on sim outputs that were cosmological. I think it would be relatively straightforward to generalize the code to allow one to *set* the cosmological parameters, instead of reading those values from the dataset. I think Britton would be the main person to answer these questions, but I'll take a look at it this evening and see how hard this might be.

Cameron

On Mon, Dec 8, 2014 at 2:59 PM, Nathan Goldbaum

...

wrote:

...

Hi Brian and Nick,

Can you share with us the errors you're seeing?

It's likely that no one has tried to do what you're doing before. To get it to work, it might just be necessary to make some adjustments to the absorption spectrum machinery in the case when we detect a non-cosmological dataset. That will require making some modifications to yt, but that's ok!

It looks like the absorption spectrum analysis module *has* seen some updates for yt-3.0, so you may also want to try again in the latest version of the codebase. I haven't personally used it - it looks like Hillary Egan was the last person to touch the code.

-Nathan

On Mon Dec 08 2014 at 1:52:15 PM Brian O'Shea wrote:

> Hi folks, > > Just to send a quick followup on this: Nick is referring > specifically to the *absorption spectrum* generator, and he is trying to > make spectra of idealized, non-cosmological simulations (think a blob in an > ambient medium) that has no redshift as a simulation parameter. The > question is, how does one do this? > > Cheers, > Brian > > > On Mon, Dec 8, 2014 at 11:59 AM, Nicholas Earl > wrote: > >> Hey everyone, >> >> Is there a way to fake the cosmological parameters that the >> synthetic spectrum generator looks for in a simulation? Or rather, a >> "sanctioned" way? Having the package reference hardcoded values when the >> parameters themselves are not found yields other missing fields like >> 'creation_time'. I am wondering if this is a consequence of the simulation >> not being inherently cosmological? >> >> I am using yt 2.x currently, as I'm aware that there seems to be an >> issue with the generator in the 3.0 release. >> >> Thanks for your time, >> Nick >> >> _______________________________________________ >> yt-users mailing list >> yt-users@lists.spacepope.org >> http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org >> >> > _______________________________________________ > yt-users mailing list > yt-users@lists.spacepope.org > http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org >

_______________________________________________ yt-users mailing list yt-users@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org

-- Cameron Hummels Postdoctoral Researcher Steward Observatory University of Arizona http://chummels.org _______________________________________________ yt-users mailing list yt-users@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org

_______________________________________________ yt-users mailing list yt-users@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org

_______________________________________________ yt-users mailing list yt-users@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org

_______________________________________________ yt-users mailing list yt-users@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org

_______________________________________________ yt-users mailing list yt-users@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org