Light Cone Projection
Hi all, I downloaded the example script of light cone projectoin( http://yt-project.org/doc/cookbook/cosmological_analysis.html#light-cone-pro...) to my laptop and changed the Enzo simulation parameter file and redshifts. But when I ran it the following error occurred: File "light_cone_projection.py", line 19, in <module> find_outputs=False) File "/Users/madcpf/Documents/dev_yt/src/yt-hg/yt/analysis_modules/cosmological_observation/light_cone/light_cone.py", line 158, in __init__ find_outputs=find_outputs) File "/Users/madcpf/Documents/dev_yt/src/yt-hg/yt/analysis_modules/cosmological_observation/cosmology_splice.py", line 44, in __init__ find_outputs=find_outputs) File "/Users/madcpf/Documents/dev_yt/src/yt-hg/yt/convenience.py", line 131, in simulation find_outputs=find_outputs) File "/Users/madcpf/Documents/dev_yt/src/yt-hg/yt/frontends/enzo/simulation_handling.py", line 81, in __init__ find_outputs=find_outputs) File "/Users/madcpf/Documents/dev_yt/src/yt-hg/yt/data_objects/time_series.py", line 343, in __init__ self._parse_parameter_file() File "/Users/madcpf/Documents/dev_yt/src/yt-hg/yt/frontends/enzo/simulation_handling.py", line 273, in _parse_parameter_file param, vals = (i.strip() for i in line.split("=")) ValueError: need more than 1 value to unpack Does the parameters "near_redshift" and "far_redshift" in LightCone need to be the same with "CosmologyFinalRedshift" and "CosmologyInitialRedshift" in enzo parameter file? Does anyone know how to solve this? Thank you all! Pengfei
Hi Pengfei, The near_redshift and far_redshift are free parameters for you to set, and just control the redshift interval over which to make the light cone projection. You might also want to look at the narrative documentation on the light cone generator, which you can find here: http://yt-project.org/doc/analysis_modules/light_cone_generator.html Note, that you need to use this with a simulation that made enough data outputs to connect the redshift interval you specify. The "enzo_tiny_cosmology" dataset available at http://yt-project.org/data/ was configured to work with the light cone recipe in the cookbook, so I would suggest downloading that. As for the error, this is happening because there is a line in your enzo parameter file that contains more than one "=" sign. I have just submitted a pull request that fixes it, so this should be fixed soon. In the mean time, you can simply comment out that line in your parameter file and everything should work fine. Britton On Fri, Sep 6, 2013 at 3:02 AM, Pengfei Chen <madcpf@gmail.com> wrote:
Hi all,
I downloaded the example script of light cone projectoin( http://yt-project.org/doc/cookbook/cosmological_analysis.html#light-cone-pro...) to my laptop and changed the Enzo simulation parameter file and redshifts. But when I ran it the following error occurred:
File "light_cone_projection.py", line 19, in <module> find_outputs=False) File "/Users/madcpf/Documents/dev_yt/src/yt-hg/yt/analysis_modules/cosmological_observation/light_cone/light_cone.py", line 158, in __init__ find_outputs=find_outputs) File "/Users/madcpf/Documents/dev_yt/src/yt-hg/yt/analysis_modules/cosmological_observation/cosmology_splice.py", line 44, in __init__ find_outputs=find_outputs) File "/Users/madcpf/Documents/dev_yt/src/yt-hg/yt/convenience.py", line 131, in simulation find_outputs=find_outputs) File "/Users/madcpf/Documents/dev_yt/src/yt-hg/yt/frontends/enzo/simulation_handling.py", line 81, in __init__ find_outputs=find_outputs) File "/Users/madcpf/Documents/dev_yt/src/yt-hg/yt/data_objects/time_series.py", line 343, in __init__ self._parse_parameter_file() File "/Users/madcpf/Documents/dev_yt/src/yt-hg/yt/frontends/enzo/simulation_handling.py", line 273, in _parse_parameter_file param, vals = (i.strip() for i in line.split("=")) ValueError: need more than 1 value to unpack
Does the parameters "near_redshift" and "far_redshift" in LightCone need to be the same with "CosmologyFinalRedshift" and "CosmologyInitialRedshift" in enzo parameter file? Does anyone know how to solve this?
Thank you all!
Pengfei
_______________________________________________ yt-users mailing list yt-users@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
Hi Britton, Thank you so much for your detailed reply! I've read documentation about Light Cone Generator, Light Ray Generator and Absorption Spectrum. I also downloaded the dataset and ran the example scripts with it. There might be two typos here: (1) In Section "Adding Features to the Spectrum" on http://yt-project.org/doc/analysis_modules/absorption_spectrum.html gamme should be gamma. (2) When I ran make_light_ray.py I got an error "OSError: [Errno 2] No such file or directory: 'halo_analysis/enzo_tiny_cosmology/RD0000'" I need to do "cp -r enzo_tiny_cosmology halo_analysis/" to make it work. Back to my enzo parameter file, which is attached. I correct that line and know that I don't have enough dataset, so I ran plan_cosmology_splice function to it but I got the following error: Traceback (most recent call last): File "plan_cosmology_splice.py", line 2, in <module> my_splice = CosmologySplice('enzo_run/32cube.enzo', 'Enzo') File "/Users/madcpf/Documents/dev_yt/src/yt-hg/yt/analysis_modules/cosmological_observation/cosmology_splice.py", line 44, in __init__ find_outputs=find_outputs) File "/Users/madcpf/Documents/dev_yt/src/yt-hg/yt/convenience.py", line 131, in simulation find_outputs=find_outputs) File "/Users/madcpf/Documents/dev_yt/src/yt-hg/yt/frontends/enzo/simulation_handling.py", line 81, in __init__ find_outputs=find_outputs) File "/Users/madcpf/Documents/dev_yt/src/yt-hg/yt/data_objects/time_series.py", line 350, in __init__ self._get_all_outputs(find_outputs=find_outputs) File "/Users/madcpf/Documents/dev_yt/src/yt-hg/yt/frontends/enzo/simulation_handling.py", line 434, in _get_all_outputs self.all_outputs = self.all_time_outputs + self.all_redshift_outputs AttributeError: 'EnzoSimulation' object has no attribute 'all_time_outputs' Since the Enzo I need to use is a modified version of Enzo 1.5, I don't know if that's the problem and how I could fix it. Could you please give me some advice? Also I am a little bit confused about how light ray and absorption spectrum works and have a few questions: (1) What's the initial spectrum before any absorption? (2) Where does the light ray originate from? Is it shoot from a random position in the first dataset? Can we set it manually like on the surface of the box? Does it go out of the box and back into it until the next output redshift? (3) In the output file lightraysolution.txt, what do the parameters(dl/box, Start x/y/z, End x/y/z) mean? Sorry that I have so many questions... I wonder if there is some existing documents or paper that have a more detailed description. Thank you again for your help! Have a nice weekend! Pengfei 2013/9/6 Britton Smith <brittonsmith@gmail.com>
Hi Pengfei,
The near_redshift and far_redshift are free parameters for you to set, and just control the redshift interval over which to make the light cone projection. You might also want to look at the narrative documentation on the light cone generator, which you can find here: http://yt-project.org/doc/analysis_modules/light_cone_generator.html Note, that you need to use this with a simulation that made enough data outputs to connect the redshift interval you specify. The "enzo_tiny_cosmology" dataset available at http://yt-project.org/data/was configured to work with the light cone recipe in the cookbook, so I would suggest downloading that.
As for the error, this is happening because there is a line in your enzo parameter file that contains more than one "=" sign. I have just submitted a pull request that fixes it, so this should be fixed soon. In the mean time, you can simply comment out that line in your parameter file and everything should work fine.
Britton
On Fri, Sep 6, 2013 at 3:02 AM, Pengfei Chen <madcpf@gmail.com> wrote:
Hi all,
I downloaded the example script of light cone projectoin( http://yt-project.org/doc/cookbook/cosmological_analysis.html#light-cone-pro...) to my laptop and changed the Enzo simulation parameter file and redshifts. But when I ran it the following error occurred:
File "light_cone_projection.py", line 19, in <module> find_outputs=False) File "/Users/madcpf/Documents/dev_yt/src/yt-hg/yt/analysis_modules/cosmological_observation/light_cone/light_cone.py", line 158, in __init__ find_outputs=find_outputs) File "/Users/madcpf/Documents/dev_yt/src/yt-hg/yt/analysis_modules/cosmological_observation/cosmology_splice.py", line 44, in __init__ find_outputs=find_outputs) File "/Users/madcpf/Documents/dev_yt/src/yt-hg/yt/convenience.py", line 131, in simulation find_outputs=find_outputs) File "/Users/madcpf/Documents/dev_yt/src/yt-hg/yt/frontends/enzo/simulation_handling.py", line 81, in __init__ find_outputs=find_outputs) File "/Users/madcpf/Documents/dev_yt/src/yt-hg/yt/data_objects/time_series.py", line 343, in __init__ self._parse_parameter_file() File "/Users/madcpf/Documents/dev_yt/src/yt-hg/yt/frontends/enzo/simulation_handling.py", line 273, in _parse_parameter_file param, vals = (i.strip() for i in line.split("=")) ValueError: need more than 1 value to unpack
Does the parameters "near_redshift" and "far_redshift" in LightCone need to be the same with "CosmologyFinalRedshift" and "CosmologyInitialRedshift" in enzo parameter file? Does anyone know how to solve this?
Thank you all!
Pengfei
_______________________________________________ yt-users mailing list yt-users@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
_______________________________________________ yt-users mailing list yt-users@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
Hi Pengfei, Thank you for catching all of those errors in the source and documentation. As of now, they have all been fixed and if you do "yt update" the recipe should run as normal without the need to alter anything. I can answer your additional questions here. (1) What's the initial spectrum before any absorption?
The spectrum generator only creates absorption features. Essentially, it assumes that you have already subtracted off the continuum, so effectively the input spectrum is just f(lambda) = 1.0.
(2) Where does the light ray originate from? Is it shoot from a random position in the first dataset? Can we set it manually like on the surface of the box? Does it go out of the box and back into it until the next output redshift?
The light ray originates in a random position inside the first dataset with a random trajectory. This code assumes a periodic domain, so the ray will wrap around if it leaves the box. Once it has traversed a distance equivalent to the comoving distance to the next dataset, it terminates and begins a new ray there. Currently, you can not manually set the initial position of a LightRay, but you can also using the ray object ( http://yt-project.org/doc/analyzing/objects.html#available-objects). If you are interested in adding support to the LightRay object for setting a fixed initial position (and presumably a fixed end position), let me know and I can help you get started. If you were to do this, we would be very pleased do have you contribute this code back to the main code base.
(3) In the output file lightraysolution.txt, what do the parameters(dl/box, Start x/y/z, End x/y/z) mean?
Since the start and stop positions are randomly chosen, this file shows what those random positions are for each dataset, in case you want to know that. dl/box is the fraction of the box length that was used for a ray in that dataset. For more information on this, I would suggest you check out this page: http://yt-project.org/doc/analysis_modules/light_ray_generator.html#light-ra... Britton
Sorry that I have so many questions... I wonder if there is some existing documents or paper that have a more detailed description.
Thank you again for your help! Have a nice weekend!
Pengfei
2013/9/6 Britton Smith <brittonsmith@gmail.com>
Hi Pengfei,
The near_redshift and far_redshift are free parameters for you to set, and just control the redshift interval over which to make the light cone projection. You might also want to look at the narrative documentation on the light cone generator, which you can find here: http://yt-project.org/doc/analysis_modules/light_cone_generator.html Note, that you need to use this with a simulation that made enough data outputs to connect the redshift interval you specify. The "enzo_tiny_cosmology" dataset available at http://yt-project.org/data/was configured to work with the light cone recipe in the cookbook, so I would suggest downloading that.
As for the error, this is happening because there is a line in your enzo parameter file that contains more than one "=" sign. I have just submitted a pull request that fixes it, so this should be fixed soon. In the mean time, you can simply comment out that line in your parameter file and everything should work fine.
Britton
On Fri, Sep 6, 2013 at 3:02 AM, Pengfei Chen <madcpf@gmail.com> wrote:
Hi all,
I downloaded the example script of light cone projectoin( http://yt-project.org/doc/cookbook/cosmological_analysis.html#light-cone-pro...) to my laptop and changed the Enzo simulation parameter file and redshifts. But when I ran it the following error occurred:
File "light_cone_projection.py", line 19, in <module> find_outputs=False) File "/Users/madcpf/Documents/dev_yt/src/yt-hg/yt/analysis_modules/cosmological_observation/light_cone/light_cone.py", line 158, in __init__ find_outputs=find_outputs) File "/Users/madcpf/Documents/dev_yt/src/yt-hg/yt/analysis_modules/cosmological_observation/cosmology_splice.py", line 44, in __init__ find_outputs=find_outputs) File "/Users/madcpf/Documents/dev_yt/src/yt-hg/yt/convenience.py", line 131, in simulation find_outputs=find_outputs) File "/Users/madcpf/Documents/dev_yt/src/yt-hg/yt/frontends/enzo/simulation_handling.py", line 81, in __init__ find_outputs=find_outputs) File "/Users/madcpf/Documents/dev_yt/src/yt-hg/yt/data_objects/time_series.py", line 343, in __init__ self._parse_parameter_file() File "/Users/madcpf/Documents/dev_yt/src/yt-hg/yt/frontends/enzo/simulation_handling.py", line 273, in _parse_parameter_file param, vals = (i.strip() for i in line.split("=")) ValueError: need more than 1 value to unpack
Does the parameters "near_redshift" and "far_redshift" in LightCone need to be the same with "CosmologyFinalRedshift" and "CosmologyInitialRedshift" in enzo parameter file? Does anyone know how to solve this?
Thank you all!
Pengfei
_______________________________________________ yt-users mailing list yt-users@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
_______________________________________________ yt-users mailing list yt-users@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
_______________________________________________ yt-users mailing list yt-users@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
Hi Britton, Thank you so much for your reply! I think I understand how LightRay works. What plan_cosmology_splice() does is to choose output redshifts so that comoving distance between them is approximately equal to but less than the comoving box size. So the light ray would not pass through the same structure in one dataset no matter how it travels. Is it right? Also I would like to add support to the LightRay object for setting a fixed initial position. Since I never modified yt or any other large code before, could you please give me some advice? Is there any documentation or tutorials I could read? Have a nice day! Pengfei 2013/9/9 Britton Smith <brittonsmith@gmail.com>
Hi Pengfei,
Thank you for catching all of those errors in the source and documentation. As of now, they have all been fixed and if you do "yt update" the recipe should run as normal without the need to alter anything.
I can answer your additional questions here.
(1) What's the initial spectrum before any absorption?
The spectrum generator only creates absorption features. Essentially, it assumes that you have already subtracted off the continuum, so effectively the input spectrum is just f(lambda) = 1.0.
(2) Where does the light ray originate from? Is it shoot from a random position in the first dataset? Can we set it manually like on the surface of the box? Does it go out of the box and back into it until the next output redshift?
The light ray originates in a random position inside the first dataset with a random trajectory. This code assumes a periodic domain, so the ray will wrap around if it leaves the box. Once it has traversed a distance equivalent to the comoving distance to the next dataset, it terminates and begins a new ray there. Currently, you can not manually set the initial position of a LightRay, but you can also using the ray object ( http://yt-project.org/doc/analyzing/objects.html#available-objects). If you are interested in adding support to the LightRay object for setting a fixed initial position (and presumably a fixed end position), let me know and I can help you get started. If you were to do this, we would be very pleased do have you contribute this code back to the main code base.
(3) In the output file lightraysolution.txt, what do the parameters(dl/box, Start x/y/z, End x/y/z) mean?
Since the start and stop positions are randomly chosen, this file shows what those random positions are for each dataset, in case you want to know that. dl/box is the fraction of the box length that was used for a ray in that dataset. For more information on this, I would suggest you check out this page:
http://yt-project.org/doc/analysis_modules/light_ray_generator.html#light-ra...
Britton
Sorry that I have so many questions... I wonder if there is some existing documents or paper that have a more detailed description.
Thank you again for your help! Have a nice weekend!
Pengfei
2013/9/6 Britton Smith <brittonsmith@gmail.com>
Hi Pengfei,
The near_redshift and far_redshift are free parameters for you to set, and just control the redshift interval over which to make the light cone projection. You might also want to look at the narrative documentation on the light cone generator, which you can find here: http://yt-project.org/doc/analysis_modules/light_cone_generator.html Note, that you need to use this with a simulation that made enough data outputs to connect the redshift interval you specify. The "enzo_tiny_cosmology" dataset available at http://yt-project.org/data/was configured to work with the light cone recipe in the cookbook, so I would suggest downloading that.
As for the error, this is happening because there is a line in your enzo parameter file that contains more than one "=" sign. I have just submitted a pull request that fixes it, so this should be fixed soon. In the mean time, you can simply comment out that line in your parameter file and everything should work fine.
Britton
On Fri, Sep 6, 2013 at 3:02 AM, Pengfei Chen <madcpf@gmail.com> wrote:
Hi all,
I downloaded the example script of light cone projectoin( http://yt-project.org/doc/cookbook/cosmological_analysis.html#light-cone-pro...) to my laptop and changed the Enzo simulation parameter file and redshifts. But when I ran it the following error occurred:
File "light_cone_projection.py", line 19, in <module> find_outputs=False) File "/Users/madcpf/Documents/dev_yt/src/yt-hg/yt/analysis_modules/cosmological_observation/light_cone/light_cone.py", line 158, in __init__ find_outputs=find_outputs) File "/Users/madcpf/Documents/dev_yt/src/yt-hg/yt/analysis_modules/cosmological_observation/cosmology_splice.py", line 44, in __init__ find_outputs=find_outputs) File "/Users/madcpf/Documents/dev_yt/src/yt-hg/yt/convenience.py", line 131, in simulation find_outputs=find_outputs) File "/Users/madcpf/Documents/dev_yt/src/yt-hg/yt/frontends/enzo/simulation_handling.py", line 81, in __init__ find_outputs=find_outputs) File "/Users/madcpf/Documents/dev_yt/src/yt-hg/yt/data_objects/time_series.py", line 343, in __init__ self._parse_parameter_file() File "/Users/madcpf/Documents/dev_yt/src/yt-hg/yt/frontends/enzo/simulation_handling.py", line 273, in _parse_parameter_file param, vals = (i.strip() for i in line.split("=")) ValueError: need more than 1 value to unpack
Does the parameters "near_redshift" and "far_redshift" in LightCone need to be the same with "CosmologyFinalRedshift" and "CosmologyInitialRedshift" in enzo parameter file? Does anyone know how to solve this?
Thank you all!
Pengfei
_______________________________________________ yt-users mailing list yt-users@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
_______________________________________________ yt-users mailing list yt-users@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
_______________________________________________ yt-users mailing list yt-users@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
_______________________________________________ yt-users mailing list yt-users@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
Hi Pengfei, That is right. Using plan_cosmology_splice will allow you to minimize the number of outputs needed to traverse a given redshift interval. Note, that since the start and stop positions in each dataset are random, there is no guarantee that you will not pass through the same structure more than once, although the odds are quite low. I'm very glad to hear you're interested in working on this. For starters, you might want to read our documentation page on developing for yt, which is here: http://yt-project.org/doc/advanced/developing.html This will explain some basics about how to work in a way that will make it easiest for your contributions to be pulled into the main repository. As for specifics on what you want to do, you should check out the yt source file: yt/analysis_modules/cosmological_observation/light_ray/light_ray.py Have a look at the routine called _calculate_light_ray_solution. This routine is called by make_light_ray, and is responsible for creating the list of starting and ending positions for each ray segment. Starting on line 165, you can see where the random initial position for a segment is calculated. Here is where you would want to add the ability to set a fixed position. A few lines down, a random trajectory is calculated, which is used along with the length of the ray segment to determine the end point. This should get you started. Please, let us know if you have any more questions. Additionally, you might consider joining the developers mailing list if you think you would like to continue working on yt, which would be great! Good luck! Britton On Wed, Sep 11, 2013 at 12:59 AM, Pengfei Chen <madcpf@gmail.com> wrote:
Hi Britton,
Thank you so much for your reply! I think I understand how LightRay works. What plan_cosmology_splice() does is to choose output redshifts so that comoving distance between them is approximately equal to but less than the comoving box size. So the light ray would not pass through the same structure in one dataset no matter how it travels. Is it right?
Also I would like to add support to the LightRay object for setting a fixed initial position. Since I never modified yt or any other large code before, could you please give me some advice? Is there any documentation or tutorials I could read?
Have a nice day!
Pengfei
2013/9/9 Britton Smith <brittonsmith@gmail.com>
Hi Pengfei,
Thank you for catching all of those errors in the source and documentation. As of now, they have all been fixed and if you do "yt update" the recipe should run as normal without the need to alter anything.
I can answer your additional questions here.
(1) What's the initial spectrum before any absorption?
The spectrum generator only creates absorption features. Essentially, it assumes that you have already subtracted off the continuum, so effectively the input spectrum is just f(lambda) = 1.0.
(2) Where does the light ray originate from? Is it shoot from a random position in the first dataset? Can we set it manually like on the surface of the box? Does it go out of the box and back into it until the next output redshift?
The light ray originates in a random position inside the first dataset with a random trajectory. This code assumes a periodic domain, so the ray will wrap around if it leaves the box. Once it has traversed a distance equivalent to the comoving distance to the next dataset, it terminates and begins a new ray there. Currently, you can not manually set the initial position of a LightRay, but you can also using the ray object ( http://yt-project.org/doc/analyzing/objects.html#available-objects). If you are interested in adding support to the LightRay object for setting a fixed initial position (and presumably a fixed end position), let me know and I can help you get started. If you were to do this, we would be very pleased do have you contribute this code back to the main code base.
(3) In the output file lightraysolution.txt, what do the parameters(dl/box, Start x/y/z, End x/y/z) mean?
Since the start and stop positions are randomly chosen, this file shows what those random positions are for each dataset, in case you want to know that. dl/box is the fraction of the box length that was used for a ray in that dataset. For more information on this, I would suggest you check out this page:
http://yt-project.org/doc/analysis_modules/light_ray_generator.html#light-ra...
Britton
Sorry that I have so many questions... I wonder if there is some existing documents or paper that have a more detailed description.
Thank you again for your help! Have a nice weekend!
Pengfei
2013/9/6 Britton Smith <brittonsmith@gmail.com>
Hi Pengfei,
The near_redshift and far_redshift are free parameters for you to set, and just control the redshift interval over which to make the light cone projection. You might also want to look at the narrative documentation on the light cone generator, which you can find here: http://yt-project.org/doc/analysis_modules/light_cone_generator.html Note, that you need to use this with a simulation that made enough data outputs to connect the redshift interval you specify. The "enzo_tiny_cosmology" dataset available at http://yt-project.org/data/was configured to work with the light cone recipe in the cookbook, so I would suggest downloading that.
As for the error, this is happening because there is a line in your enzo parameter file that contains more than one "=" sign. I have just submitted a pull request that fixes it, so this should be fixed soon. In the mean time, you can simply comment out that line in your parameter file and everything should work fine.
Britton
On Fri, Sep 6, 2013 at 3:02 AM, Pengfei Chen <madcpf@gmail.com> wrote:
Hi all,
I downloaded the example script of light cone projectoin( http://yt-project.org/doc/cookbook/cosmological_analysis.html#light-cone-pro...) to my laptop and changed the Enzo simulation parameter file and redshifts. But when I ran it the following error occurred:
File "light_cone_projection.py", line 19, in <module> find_outputs=False) File "/Users/madcpf/Documents/dev_yt/src/yt-hg/yt/analysis_modules/cosmological_observation/light_cone/light_cone.py", line 158, in __init__ find_outputs=find_outputs) File "/Users/madcpf/Documents/dev_yt/src/yt-hg/yt/analysis_modules/cosmological_observation/cosmology_splice.py", line 44, in __init__ find_outputs=find_outputs) File "/Users/madcpf/Documents/dev_yt/src/yt-hg/yt/convenience.py", line 131, in simulation find_outputs=find_outputs) File "/Users/madcpf/Documents/dev_yt/src/yt-hg/yt/frontends/enzo/simulation_handling.py", line 81, in __init__ find_outputs=find_outputs) File "/Users/madcpf/Documents/dev_yt/src/yt-hg/yt/data_objects/time_series.py", line 343, in __init__ self._parse_parameter_file() File "/Users/madcpf/Documents/dev_yt/src/yt-hg/yt/frontends/enzo/simulation_handling.py", line 273, in _parse_parameter_file param, vals = (i.strip() for i in line.split("=")) ValueError: need more than 1 value to unpack
Does the parameters "near_redshift" and "far_redshift" in LightCone need to be the same with "CosmologyFinalRedshift" and "CosmologyInitialRedshift" in enzo parameter file? Does anyone know how to solve this?
Thank you all!
Pengfei
_______________________________________________ yt-users mailing list yt-users@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
_______________________________________________ yt-users mailing list yt-users@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
_______________________________________________ yt-users mailing list yt-users@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
_______________________________________________ yt-users mailing list yt-users@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
_______________________________________________ yt-users mailing list yt-users@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
Hi Britton, Thank you very much for your detailed instructions. I've added two parameters in _calculate_light_ray_solution in light_ray.py. They are 'start_point' and 'start_direction', each of which is supposed to be a 3-element list/array. Their default values are None and they can be used together or individually. The code is attached and it works. But I still have two problems: (1) I am confused about the parameter "minimum_coherent_box_fraction", and what the following statement is trying to do: if (q == 0) or (self.minimum_coherent_box_fraction == 0) or \ (box_fraction_used > self.minimum_coherent_box_fraction) or \ (box_fraction_used + self.light_ray_solution[q]['traversal_box_fraction'] > 1.0): What does "coherent large scale structure" and "box coherence" mean? (2) When I am using minimum datasets the start point of each dataset is not same as the end point of the last dataset even if I set minimum_coherent_box_fraction = 1. Because box_fraction_used + self.light_ray_solution[q]['traversal_box_fraction'] > 1.0 is always satisfied when using minimum datasets. I don't know the right way to get start point to be the same as the end point of the last dataset, so I tried to change the statements in (1) to if (q == 0): I find it works for(parameters when call make_light_ray): start_point=None, start_direction=None, and start_point=None, start_direction=[1,1,0], and start_point=[0.5,0.5,0.5], start_direction=[1,1,0], But not start_point=[0.5,0.5,0.5], start_direction=None, For the last case it shows the following error: yt : [INFO ] 2013-09-25 04:07:18,162 Getting the binary hierarchy Parsing Hierarchy100% |||||||||||||||||||||||||||||||||||||||||||||||||||||||||| Time: 00:00:00 yt : [INFO ] 2013-09-25 04:07:18,185 Gathering a field list (this may take a moment.) yt : [INFO ] 2013-09-25 04:07:18,443 Getting field t from 0 Traceback (most recent call last): File "make_light_ray.py", line 68, in <module> njobs=-1) File "/Users/madcpf/Documents/yt/yt/analysis_modules/cosmological_observation/light_ray/light_ray.py", line 388, in make_light_ray (sub_ray['dts'] * File "/Users/madcpf/Documents/yt/yt/data_objects/data_containers.py", line 318, in __getitem__ self.get_data(key) File "/Users/madcpf/Documents/yt/yt/data_objects/data_containers.py", line 525, in get_data for grid in self._grids]) ValueError: need at least one array to concatenate Could you please take a look when you have time? Thank you so much! Pengfei 2013/9/11 Britton Smith <brittonsmith@gmail.com>
Hi Pengfei,
That is right. Using plan_cosmology_splice will allow you to minimize the number of outputs needed to traverse a given redshift interval. Note, that since the start and stop positions in each dataset are random, there is no guarantee that you will not pass through the same structure more than once, although the odds are quite low.
I'm very glad to hear you're interested in working on this. For starters, you might want to read our documentation page on developing for yt, which is here: http://yt-project.org/doc/advanced/developing.html This will explain some basics about how to work in a way that will make it easiest for your contributions to be pulled into the main repository.
As for specifics on what you want to do, you should check out the yt source file: yt/analysis_modules/cosmological_observation/light_ray/light_ray.py
Have a look at the routine called _calculate_light_ray_solution. This routine is called by make_light_ray, and is responsible for creating the list of starting and ending positions for each ray segment. Starting on line 165, you can see where the random initial position for a segment is calculated. Here is where you would want to add the ability to set a fixed position. A few lines down, a random trajectory is calculated, which is used along with the length of the ray segment to determine the end point.
This should get you started. Please, let us know if you have any more questions. Additionally, you might consider joining the developers mailing list if you think you would like to continue working on yt, which would be great! Good luck!
Britton
On Wed, Sep 11, 2013 at 12:59 AM, Pengfei Chen <madcpf@gmail.com> wrote:
Hi Britton,
Thank you so much for your reply! I think I understand how LightRay works. What plan_cosmology_splice() does is to choose output redshifts so that comoving distance between them is approximately equal to but less than the comoving box size. So the light ray would not pass through the same structure in one dataset no matter how it travels. Is it right?
Also I would like to add support to the LightRay object for setting a fixed initial position. Since I never modified yt or any other large code before, could you please give me some advice? Is there any documentation or tutorials I could read?
Have a nice day!
Pengfei
2013/9/9 Britton Smith <brittonsmith@gmail.com>
Hi Pengfei,
Thank you for catching all of those errors in the source and documentation. As of now, they have all been fixed and if you do "yt update" the recipe should run as normal without the need to alter anything.
I can answer your additional questions here.
(1) What's the initial spectrum before any absorption?
The spectrum generator only creates absorption features. Essentially, it assumes that you have already subtracted off the continuum, so effectively the input spectrum is just f(lambda) = 1.0.
(2) Where does the light ray originate from? Is it shoot from a random position in the first dataset? Can we set it manually like on the surface of the box? Does it go out of the box and back into it until the next output redshift?
The light ray originates in a random position inside the first dataset with a random trajectory. This code assumes a periodic domain, so the ray will wrap around if it leaves the box. Once it has traversed a distance equivalent to the comoving distance to the next dataset, it terminates and begins a new ray there. Currently, you can not manually set the initial position of a LightRay, but you can also using the ray object ( http://yt-project.org/doc/analyzing/objects.html#available-objects). If you are interested in adding support to the LightRay object for setting a fixed initial position (and presumably a fixed end position), let me know and I can help you get started. If you were to do this, we would be very pleased do have you contribute this code back to the main code base.
(3) In the output file lightraysolution.txt, what do the parameters(dl/box, Start x/y/z, End x/y/z) mean?
Since the start and stop positions are randomly chosen, this file shows what those random positions are for each dataset, in case you want to know that. dl/box is the fraction of the box length that was used for a ray in that dataset. For more information on this, I would suggest you check out this page:
http://yt-project.org/doc/analysis_modules/light_ray_generator.html#light-ra...
Britton
Sorry that I have so many questions... I wonder if there is some existing documents or paper that have a more detailed description.
Thank you again for your help! Have a nice weekend!
Pengfei
2013/9/6 Britton Smith <brittonsmith@gmail.com>
Hi Pengfei,
The near_redshift and far_redshift are free parameters for you to set, and just control the redshift interval over which to make the light cone projection. You might also want to look at the narrative documentation on the light cone generator, which you can find here: http://yt-project.org/doc/analysis_modules/light_cone_generator.html Note, that you need to use this with a simulation that made enough data outputs to connect the redshift interval you specify. The "enzo_tiny_cosmology" dataset available at http://yt-project.org/data/was configured to work with the light cone recipe in the cookbook, so I would suggest downloading that.
As for the error, this is happening because there is a line in your enzo parameter file that contains more than one "=" sign. I have just submitted a pull request that fixes it, so this should be fixed soon. In the mean time, you can simply comment out that line in your parameter file and everything should work fine.
Britton
On Fri, Sep 6, 2013 at 3:02 AM, Pengfei Chen <madcpf@gmail.com> wrote:
Hi all,
I downloaded the example script of light cone projectoin( http://yt-project.org/doc/cookbook/cosmological_analysis.html#light-cone-pro...) to my laptop and changed the Enzo simulation parameter file and redshifts. But when I ran it the following error occurred:
File "light_cone_projection.py", line 19, in <module> find_outputs=False) File "/Users/madcpf/Documents/dev_yt/src/yt-hg/yt/analysis_modules/cosmological_observation/light_cone/light_cone.py", line 158, in __init__ find_outputs=find_outputs) File "/Users/madcpf/Documents/dev_yt/src/yt-hg/yt/analysis_modules/cosmological_observation/cosmology_splice.py", line 44, in __init__ find_outputs=find_outputs) File "/Users/madcpf/Documents/dev_yt/src/yt-hg/yt/convenience.py", line 131, in simulation find_outputs=find_outputs) File "/Users/madcpf/Documents/dev_yt/src/yt-hg/yt/frontends/enzo/simulation_handling.py", line 81, in __init__ find_outputs=find_outputs) File "/Users/madcpf/Documents/dev_yt/src/yt-hg/yt/data_objects/time_series.py", line 343, in __init__ self._parse_parameter_file() File "/Users/madcpf/Documents/dev_yt/src/yt-hg/yt/frontends/enzo/simulation_handling.py", line 273, in _parse_parameter_file param, vals = (i.strip() for i in line.split("=")) ValueError: need more than 1 value to unpack
Does the parameters "near_redshift" and "far_redshift" in LightCone need to be the same with "CosmologyFinalRedshift" and "CosmologyInitialRedshift" in enzo parameter file? Does anyone know how to solve this?
Thank you all!
Pengfei
_______________________________________________ yt-users mailing list yt-users@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
_______________________________________________ yt-users mailing list yt-users@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
_______________________________________________ yt-users mailing list yt-users@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
_______________________________________________ yt-users mailing list yt-users@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
_______________________________________________ yt-users mailing list yt-users@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
_______________________________________________ yt-users mailing list yt-users@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
Hi Pengfei, To answer your first question, the minimum_coherent_box_fraction option is a bit of a failed experiment in which one is using as many datasets as possible in the light cone, instead of using as few as possible. In this case, we use the same ray trajectory through multiple datasets until the ray has extended the length of one box. This was to avoid making cuts in halos or filaments by using thin slices of each dataset. Functionally, this works by waiting to re-randomize the ray trajectory until you've gone through a full box length. In practice, doing this produced a number of artifacts in the light cone projections so should probably be used with caution in the light rays. I think in reality, one should stick with the option of using the smallest number of datasets as possible to traverse a redshift interval. On your second question, the problem is actually that, if you start from a random position in a periodic cube of box length L, you only return to that position after moving a distance of L if you move exactly along the axis directions. As another example, if you started in the corner of the box and went toward the opposite corner (which is the same point since it's periodic) you would have to travel a distance of sqrt(3) * L to get there. In reality, though, you probably do not want to be starting from the same point in each dataset, since you will then just be resampling the same structure over and over. Other than that, the code itself looks just fine. Please, let me know if I haven't answered your questions. Britton On Wed, Sep 25, 2013 at 12:19 PM, Pengfei Chen <madcpf@gmail.com> wrote:
Hi Britton,
Thank you very much for your detailed instructions. I've added two parameters in _calculate_light_ray_solution in light_ray.py. They are 'start_point' and 'start_direction', each of which is supposed to be a 3-element list/array. Their default values are None and they can be used together or individually. The code is attached and it works. But I still have two problems:
(1) I am confused about the parameter "minimum_coherent_box_fraction", and what the following statement is trying to do:
if (q == 0) or (self.minimum_coherent_box_fraction == 0) or \ (box_fraction_used > self.minimum_coherent_box_fraction) or \ (box_fraction_used + self.light_ray_solution[q]['traversal_box_fraction']
1.0):
What does "coherent large scale structure" and "box coherence" mean?
(2) When I am using minimum datasets the start point of each dataset is not same as the end point of the last dataset even if I set minimum_coherent_box_fraction = 1. Because
box_fraction_used + self.light_ray_solution[q]['traversal_box_fraction']
1.0
is always satisfied when using minimum datasets. I don't know the right way to get start point to be the same as the end point of the last dataset, so I tried to change the statements in (1) to if (q == 0): I find it works for(parameters when call make_light_ray): start_point=None, start_direction=None, and start_point=None, start_direction=[1,1,0], and start_point=[0.5,0.5,0.5], start_direction=[1,1,0], But not start_point=[0.5,0.5,0.5], start_direction=None,
For the last case it shows the following error:
yt : [INFO ] 2013-09-25 04:07:18,162 Getting the binary hierarchy Parsing Hierarchy100% |||||||||||||||||||||||||||||||||||||||||||||||||||||||||| Time: 00:00:00 yt : [INFO ] 2013-09-25 04:07:18,185 Gathering a field list (this may take a moment.) yt : [INFO ] 2013-09-25 04:07:18,443 Getting field t from 0 Traceback (most recent call last): File "make_light_ray.py", line 68, in <module> njobs=-1) File "/Users/madcpf/Documents/yt/yt/analysis_modules/cosmological_observation/light_ray/light_ray.py", line 388, in make_light_ray (sub_ray['dts'] * File "/Users/madcpf/Documents/yt/yt/data_objects/data_containers.py", line 318, in __getitem__ self.get_data(key) File "/Users/madcpf/Documents/yt/yt/data_objects/data_containers.py", line 525, in get_data for grid in self._grids]) ValueError: need at least one array to concatenate
Could you please take a look when you have time?
Thank you so much!
Pengfei
2013/9/11 Britton Smith <brittonsmith@gmail.com>
Hi Pengfei,
That is right. Using plan_cosmology_splice will allow you to minimize the number of outputs needed to traverse a given redshift interval. Note, that since the start and stop positions in each dataset are random, there is no guarantee that you will not pass through the same structure more than once, although the odds are quite low.
I'm very glad to hear you're interested in working on this. For starters, you might want to read our documentation page on developing for yt, which is here: http://yt-project.org/doc/advanced/developing.html This will explain some basics about how to work in a way that will make it easiest for your contributions to be pulled into the main repository.
As for specifics on what you want to do, you should check out the yt source file: yt/analysis_modules/cosmological_observation/light_ray/light_ray.py
Have a look at the routine called _calculate_light_ray_solution. This routine is called by make_light_ray, and is responsible for creating the list of starting and ending positions for each ray segment. Starting on line 165, you can see where the random initial position for a segment is calculated. Here is where you would want to add the ability to set a fixed position. A few lines down, a random trajectory is calculated, which is used along with the length of the ray segment to determine the end point.
This should get you started. Please, let us know if you have any more questions. Additionally, you might consider joining the developers mailing list if you think you would like to continue working on yt, which would be great! Good luck!
Britton
On Wed, Sep 11, 2013 at 12:59 AM, Pengfei Chen <madcpf@gmail.com> wrote:
Hi Britton,
Thank you so much for your reply! I think I understand how LightRay works. What plan_cosmology_splice() does is to choose output redshifts so that comoving distance between them is approximately equal to but less than the comoving box size. So the light ray would not pass through the same structure in one dataset no matter how it travels. Is it right?
Also I would like to add support to the LightRay object for setting a fixed initial position. Since I never modified yt or any other large code before, could you please give me some advice? Is there any documentation or tutorials I could read?
Have a nice day!
Pengfei
2013/9/9 Britton Smith <brittonsmith@gmail.com>
Hi Pengfei,
Thank you for catching all of those errors in the source and documentation. As of now, they have all been fixed and if you do "yt update" the recipe should run as normal without the need to alter anything.
I can answer your additional questions here.
(1) What's the initial spectrum before any absorption?
The spectrum generator only creates absorption features. Essentially, it assumes that you have already subtracted off the continuum, so effectively the input spectrum is just f(lambda) = 1.0.
(2) Where does the light ray originate from? Is it shoot from a random position in the first dataset? Can we set it manually like on the surface of the box? Does it go out of the box and back into it until the next output redshift?
The light ray originates in a random position inside the first dataset with a random trajectory. This code assumes a periodic domain, so the ray will wrap around if it leaves the box. Once it has traversed a distance equivalent to the comoving distance to the next dataset, it terminates and begins a new ray there. Currently, you can not manually set the initial position of a LightRay, but you can also using the ray object ( http://yt-project.org/doc/analyzing/objects.html#available-objects). If you are interested in adding support to the LightRay object for setting a fixed initial position (and presumably a fixed end position), let me know and I can help you get started. If you were to do this, we would be very pleased do have you contribute this code back to the main code base.
(3) In the output file lightraysolution.txt, what do the parameters(dl/box, Start x/y/z, End x/y/z) mean?
Since the start and stop positions are randomly chosen, this file shows what those random positions are for each dataset, in case you want to know that. dl/box is the fraction of the box length that was used for a ray in that dataset. For more information on this, I would suggest you check out this page:
http://yt-project.org/doc/analysis_modules/light_ray_generator.html#light-ra...
Britton
Sorry that I have so many questions... I wonder if there is some existing documents or paper that have a more detailed description.
Thank you again for your help! Have a nice weekend!
Pengfei
2013/9/6 Britton Smith <brittonsmith@gmail.com>
Hi Pengfei,
The near_redshift and far_redshift are free parameters for you to set, and just control the redshift interval over which to make the light cone projection. You might also want to look at the narrative documentation on the light cone generator, which you can find here: http://yt-project.org/doc/analysis_modules/light_cone_generator.html Note, that you need to use this with a simulation that made enough data outputs to connect the redshift interval you specify. The "enzo_tiny_cosmology" dataset available at http://yt-project.org/data/ was configured to work with the light cone recipe in the cookbook, so I would suggest downloading that.
As for the error, this is happening because there is a line in your enzo parameter file that contains more than one "=" sign. I have just submitted a pull request that fixes it, so this should be fixed soon. In the mean time, you can simply comment out that line in your parameter file and everything should work fine.
Britton
On Fri, Sep 6, 2013 at 3:02 AM, Pengfei Chen <madcpf@gmail.com>wrote:
> Hi all, > > I downloaded the example script of light cone projectoin( > http://yt-project.org/doc/cookbook/cosmological_analysis.html#light-cone-pro...) > to my laptop and changed the Enzo simulation parameter file and redshifts. > But when I ran it the following error occurred: > > File "light_cone_projection.py", line 19, in <module> > find_outputs=False) > File > "/Users/madcpf/Documents/dev_yt/src/yt-hg/yt/analysis_modules/cosmological_observation/light_cone/light_cone.py", > line 158, in __init__ > find_outputs=find_outputs) > File > "/Users/madcpf/Documents/dev_yt/src/yt-hg/yt/analysis_modules/cosmological_observation/cosmology_splice.py", > line 44, in __init__ > find_outputs=find_outputs) > File "/Users/madcpf/Documents/dev_yt/src/yt-hg/yt/convenience.py", > line 131, in simulation > find_outputs=find_outputs) > File > "/Users/madcpf/Documents/dev_yt/src/yt-hg/yt/frontends/enzo/simulation_handling.py", > line 81, in __init__ > find_outputs=find_outputs) > File > "/Users/madcpf/Documents/dev_yt/src/yt-hg/yt/data_objects/time_series.py", > line 343, in __init__ > self._parse_parameter_file() > File > "/Users/madcpf/Documents/dev_yt/src/yt-hg/yt/frontends/enzo/simulation_handling.py", > line 273, in _parse_parameter_file > param, vals = (i.strip() for i in line.split("=")) > ValueError: need more than 1 value to unpack > > Does the parameters "near_redshift" and "far_redshift" in LightCone > need to be the same with "CosmologyFinalRedshift" and > "CosmologyInitialRedshift" in enzo parameter file? > Does anyone know how to solve this? > > Thank you all! > > Pengfei > > _______________________________________________ > yt-users mailing list > yt-users@lists.spacepope.org > http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org > >
_______________________________________________ yt-users mailing list yt-users@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
_______________________________________________ yt-users mailing list yt-users@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
_______________________________________________ yt-users mailing list yt-users@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
_______________________________________________ yt-users mailing list yt-users@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
_______________________________________________ yt-users mailing list yt-users@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
_______________________________________________ yt-users mailing list yt-users@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
Hi Britton, Last time when I asked you about minimum_coherent_box_fraction in light_ray in this email, you told me that "one should stick with the option of using the smallest number of datasets as possible to traverse a redshift interval." Now I have some datasets expanding from redshift 5 to 3, which have enough outputs but didn't use the subroutine plan_cosmology_splice to plan outputs. Now when I use the subroutine Lightray I got some problems: If I set use_minimum_datasets=True, it would have errors like this: P014 yt : [ERROR ] 2014-02-27 01:24:21,405 Warning: box fraction required to go from z = 3.684226 to 3.659368 is 1.001284 P013 yt : [ERROR ] 2014-02-27 01:24:21,405 Full box delta z is 0.024826, but it is 0.024857 to the next data dump. I don't know why there are such errors given that I have enough datasets. The generated light solution file is like this: http://paste.yt-project.org/show/4335/ But I'm not sure if this lightray is good to use with the above error message. If I set use_minimum_datasets=False, all the datasets would be used and no errors are shown. The output light solution file is like this: http://paste.yt-project.org/show/4334/ Also I'm not sure if I should use this with the potential problem about "minimum_coherent_box_fraction". Another thing I could do is to keep use_minimum_datasets=False but reduce the datasets used manually until the minimum number of datasets are used with no error messages. Could you please give me some advice about what I should do? Thank you! Pengfei On Mon, Sep 30, 2013 at 4:36 AM, Britton Smith <brittonsmith@gmail.com>wrote:
Hi Pengfei,
To answer your first question, the minimum_coherent_box_fraction option is a bit of a failed experiment in which one is using as many datasets as possible in the light cone, instead of using as few as possible. In this case, we use the same ray trajectory through multiple datasets until the ray has extended the length of one box. This was to avoid making cuts in halos or filaments by using thin slices of each dataset. Functionally, this works by waiting to re-randomize the ray trajectory until you've gone through a full box length. In practice, doing this produced a number of artifacts in the light cone projections so should probably be used with caution in the light rays. I think in reality, one should stick with the option of using the smallest number of datasets as possible to traverse a redshift interval.
On your second question, the problem is actually that, if you start from a random position in a periodic cube of box length L, you only return to that position after moving a distance of L if you move exactly along the axis directions. As another example, if you started in the corner of the box and went toward the opposite corner (which is the same point since it's periodic) you would have to travel a distance of sqrt(3) * L to get there. In reality, though, you probably do not want to be starting from the same point in each dataset, since you will then just be resampling the same structure over and over.
Other than that, the code itself looks just fine. Please, let me know if I haven't answered your questions.
Britton
On Wed, Sep 25, 2013 at 12:19 PM, Pengfei Chen <madcpf@gmail.com> wrote:
Hi Britton,
Thank you very much for your detailed instructions. I've added two parameters in _calculate_light_ray_solution in light_ray.py. They are 'start_point' and 'start_direction', each of which is supposed to be a 3-element list/array. Their default values are None and they can be used together or individually. The code is attached and it works. But I still have two problems:
(1) I am confused about the parameter "minimum_coherent_box_fraction", and what the following statement is trying to do:
if (q == 0) or (self.minimum_coherent_box_fraction == 0) or \ (box_fraction_used > self.minimum_coherent_box_fraction) or \ (box_fraction_used + self.light_ray_solution[q]['traversal_box_fraction']
1.0):
What does "coherent large scale structure" and "box coherence" mean?
(2) When I am using minimum datasets the start point of each dataset is not same as the end point of the last dataset even if I set minimum_coherent_box_fraction = 1. Because
box_fraction_used + self.light_ray_solution[q]['traversal_box_fraction']
1.0
is always satisfied when using minimum datasets. I don't know the right way to get start point to be the same as the end point of the last dataset, so I tried to change the statements in (1) to if (q == 0): I find it works for(parameters when call make_light_ray): start_point=None, start_direction=None, and start_point=None, start_direction=[1,1,0], and start_point=[0.5,0.5,0.5], start_direction=[1,1,0], But not start_point=[0.5,0.5,0.5], start_direction=None,
For the last case it shows the following error:
yt : [INFO ] 2013-09-25 04:07:18,162 Getting the binary hierarchy Parsing Hierarchy100% |||||||||||||||||||||||||||||||||||||||||||||||||||||||||| Time: 00:00:00 yt : [INFO ] 2013-09-25 04:07:18,185 Gathering a field list (this may take a moment.) yt : [INFO ] 2013-09-25 04:07:18,443 Getting field t from 0 Traceback (most recent call last): File "make_light_ray.py", line 68, in <module> njobs=-1) File "/Users/madcpf/Documents/yt/yt/analysis_modules/cosmological_observation/light_ray/light_ray.py", line 388, in make_light_ray (sub_ray['dts'] * File "/Users/madcpf/Documents/yt/yt/data_objects/data_containers.py", line 318, in __getitem__ self.get_data(key) File "/Users/madcpf/Documents/yt/yt/data_objects/data_containers.py", line 525, in get_data for grid in self._grids]) ValueError: need at least one array to concatenate
Could you please take a look when you have time?
Thank you so much!
Pengfei
2013/9/11 Britton Smith <brittonsmith@gmail.com>
Hi Pengfei,
That is right. Using plan_cosmology_splice will allow you to minimize the number of outputs needed to traverse a given redshift interval. Note, that since the start and stop positions in each dataset are random, there is no guarantee that you will not pass through the same structure more than once, although the odds are quite low.
I'm very glad to hear you're interested in working on this. For starters, you might want to read our documentation page on developing for yt, which is here: http://yt-project.org/doc/advanced/developing.html This will explain some basics about how to work in a way that will make it easiest for your contributions to be pulled into the main repository.
As for specifics on what you want to do, you should check out the yt source file: yt/analysis_modules/cosmological_observation/light_ray/light_ray.py
Have a look at the routine called _calculate_light_ray_solution. This routine is called by make_light_ray, and is responsible for creating the list of starting and ending positions for each ray segment. Starting on line 165, you can see where the random initial position for a segment is calculated. Here is where you would want to add the ability to set a fixed position. A few lines down, a random trajectory is calculated, which is used along with the length of the ray segment to determine the end point.
This should get you started. Please, let us know if you have any more questions. Additionally, you might consider joining the developers mailing list if you think you would like to continue working on yt, which would be great! Good luck!
Britton
On Wed, Sep 11, 2013 at 12:59 AM, Pengfei Chen <madcpf@gmail.com> wrote:
Hi Britton,
Thank you so much for your reply! I think I understand how LightRay works. What plan_cosmology_splice() does is to choose output redshifts so that comoving distance between them is approximately equal to but less than the comoving box size. So the light ray would not pass through the same structure in one dataset no matter how it travels. Is it right?
Also I would like to add support to the LightRay object for setting a fixed initial position. Since I never modified yt or any other large code before, could you please give me some advice? Is there any documentation or tutorials I could read?
Have a nice day!
Pengfei
2013/9/9 Britton Smith <brittonsmith@gmail.com>
Hi Pengfei,
Thank you for catching all of those errors in the source and documentation. As of now, they have all been fixed and if you do "yt update" the recipe should run as normal without the need to alter anything.
I can answer your additional questions here.
(1) What's the initial spectrum before any absorption?
The spectrum generator only creates absorption features. Essentially, it assumes that you have already subtracted off the continuum, so effectively the input spectrum is just f(lambda) = 1.0.
(2) Where does the light ray originate from? Is it shoot from a random position in the first dataset? Can we set it manually like on the surface of the box? Does it go out of the box and back into it until the next output redshift?
The light ray originates in a random position inside the first dataset with a random trajectory. This code assumes a periodic domain, so the ray will wrap around if it leaves the box. Once it has traversed a distance equivalent to the comoving distance to the next dataset, it terminates and begins a new ray there. Currently, you can not manually set the initial position of a LightRay, but you can also using the ray object ( http://yt-project.org/doc/analyzing/objects.html#available-objects). If you are interested in adding support to the LightRay object for setting a fixed initial position (and presumably a fixed end position), let me know and I can help you get started. If you were to do this, we would be very pleased do have you contribute this code back to the main code base.
(3) In the output file lightraysolution.txt, what do the parameters(dl/box, Start x/y/z, End x/y/z) mean?
Since the start and stop positions are randomly chosen, this file shows what those random positions are for each dataset, in case you want to know that. dl/box is the fraction of the box length that was used for a ray in that dataset. For more information on this, I would suggest you check out this page:
http://yt-project.org/doc/analysis_modules/light_ray_generator.html#light-ra...
Britton
Sorry that I have so many questions... I wonder if there is some existing documents or paper that have a more detailed description.
Thank you again for your help! Have a nice weekend!
Pengfei
2013/9/6 Britton Smith <brittonsmith@gmail.com>
> Hi Pengfei, > > The near_redshift and far_redshift are free parameters for you to > set, and just control the redshift interval over which to make the light > cone projection. You might also want to look at the narrative > documentation on the light cone generator, which you can find here: > http://yt-project.org/doc/analysis_modules/light_cone_generator.html > Note, that you need to use this with a simulation that made enough > data outputs to connect the redshift interval you specify. The > "enzo_tiny_cosmology" dataset available at > http://yt-project.org/data/ was configured to work with the light > cone recipe in the cookbook, so I would suggest downloading that. > > As for the error, this is happening because there is a line in your > enzo parameter file that contains more than one "=" sign. I have just > submitted a pull request that fixes it, so this should be fixed soon. In > the mean time, you can simply comment out that line in your parameter file > and everything should work fine. > > Britton > > > On Fri, Sep 6, 2013 at 3:02 AM, Pengfei Chen <madcpf@gmail.com>wrote: > >> Hi all, >> >> I downloaded the example script of light cone projectoin( >> http://yt-project.org/doc/cookbook/cosmological_analysis.html#light-cone-pro...) >> to my laptop and changed the Enzo simulation parameter file and redshifts. >> But when I ran it the following error occurred: >> >> File "light_cone_projection.py", line 19, in <module> >> find_outputs=False) >> File >> "/Users/madcpf/Documents/dev_yt/src/yt-hg/yt/analysis_modules/cosmological_observation/light_cone/light_cone.py", >> line 158, in __init__ >> find_outputs=find_outputs) >> File >> "/Users/madcpf/Documents/dev_yt/src/yt-hg/yt/analysis_modules/cosmological_observation/cosmology_splice.py", >> line 44, in __init__ >> find_outputs=find_outputs) >> File >> "/Users/madcpf/Documents/dev_yt/src/yt-hg/yt/convenience.py", line 131, in >> simulation >> find_outputs=find_outputs) >> File >> "/Users/madcpf/Documents/dev_yt/src/yt-hg/yt/frontends/enzo/simulation_handling.py", >> line 81, in __init__ >> find_outputs=find_outputs) >> File >> "/Users/madcpf/Documents/dev_yt/src/yt-hg/yt/data_objects/time_series.py", >> line 343, in __init__ >> self._parse_parameter_file() >> File >> "/Users/madcpf/Documents/dev_yt/src/yt-hg/yt/frontends/enzo/simulation_handling.py", >> line 273, in _parse_parameter_file >> param, vals = (i.strip() for i in line.split("=")) >> ValueError: need more than 1 value to unpack >> >> Does the parameters "near_redshift" and "far_redshift" in LightCone >> need to be the same with "CosmologyFinalRedshift" and >> "CosmologyInitialRedshift" in enzo parameter file? >> Does anyone know how to solve this? >> >> Thank you all! >> >> Pengfei >> >> _______________________________________________ >> yt-users mailing list >> yt-users@lists.spacepope.org >> http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org >> >> > > _______________________________________________ > yt-users mailing list > yt-users@lists.spacepope.org > http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org > >
_______________________________________________ yt-users mailing list yt-users@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
_______________________________________________ yt-users mailing list yt-users@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
_______________________________________________ yt-users mailing list yt-users@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
_______________________________________________ yt-users mailing list yt-users@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
_______________________________________________ yt-users mailing list yt-users@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
_______________________________________________ yt-users mailing list yt-users@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
Hi Pengfei, This can happen sometime just due to roundoff error or error that crops up when converting back and forth between redshift and time. Since the box fraction is only marginally greater than 1 for that segment of the light ray, it shouldn't really be a problem to use these datasets. You should still be able to make the light ray and just ignore the error message. Britton On Thu, Feb 27, 2014 at 8:20 AM, Pengfei Chen <madcpf@gmail.com> wrote:
Hi Britton,
Last time when I asked you about minimum_coherent_box_fraction in light_ray in this email, you told me that "one should stick with the option of using the smallest number of datasets as possible to traverse a redshift interval." Now I have some datasets expanding from redshift 5 to 3, which have enough outputs but didn't use the subroutine plan_cosmology_splice to plan outputs. Now when I use the subroutine Lightray I got some problems:
If I set use_minimum_datasets=True, it would have errors like this:
P014 yt : [ERROR ] 2014-02-27 01:24:21,405 Warning: box fraction required to go from z = 3.684226 to 3.659368 is 1.001284 P013 yt : [ERROR ] 2014-02-27 01:24:21,405 Full box delta z is 0.024826, but it is 0.024857 to the next data dump.
I don't know why there are such errors given that I have enough datasets. The generated light solution file is like this: http://paste.yt-project.org/show/4335/ But I'm not sure if this lightray is good to use with the above error message.
If I set use_minimum_datasets=False, all the datasets would be used and no errors are shown. The output light solution file is like this: http://paste.yt-project.org/show/4334/ Also I'm not sure if I should use this with the potential problem about "minimum_coherent_box_fraction".
Another thing I could do is to keep use_minimum_datasets=False but reduce the datasets used manually until the minimum number of datasets are used with no error messages. Could you please give me some advice about what I should do?
Thank you! Pengfei
On Mon, Sep 30, 2013 at 4:36 AM, Britton Smith <brittonsmith@gmail.com>wrote:
Hi Pengfei,
To answer your first question, the minimum_coherent_box_fraction option is a bit of a failed experiment in which one is using as many datasets as possible in the light cone, instead of using as few as possible. In this case, we use the same ray trajectory through multiple datasets until the ray has extended the length of one box. This was to avoid making cuts in halos or filaments by using thin slices of each dataset. Functionally, this works by waiting to re-randomize the ray trajectory until you've gone through a full box length. In practice, doing this produced a number of artifacts in the light cone projections so should probably be used with caution in the light rays. I think in reality, one should stick with the option of using the smallest number of datasets as possible to traverse a redshift interval.
On your second question, the problem is actually that, if you start from a random position in a periodic cube of box length L, you only return to that position after moving a distance of L if you move exactly along the axis directions. As another example, if you started in the corner of the box and went toward the opposite corner (which is the same point since it's periodic) you would have to travel a distance of sqrt(3) * L to get there. In reality, though, you probably do not want to be starting from the same point in each dataset, since you will then just be resampling the same structure over and over.
Other than that, the code itself looks just fine. Please, let me know if I haven't answered your questions.
Britton
On Wed, Sep 25, 2013 at 12:19 PM, Pengfei Chen <madcpf@gmail.com> wrote:
Hi Britton,
Thank you very much for your detailed instructions. I've added two parameters in _calculate_light_ray_solution in light_ray.py. They are 'start_point' and 'start_direction', each of which is supposed to be a 3-element list/array. Their default values are None and they can be used together or individually. The code is attached and it works. But I still have two problems:
(1) I am confused about the parameter "minimum_coherent_box_fraction", and what the following statement is trying to do:
if (q == 0) or (self.minimum_coherent_box_fraction == 0) or \ (box_fraction_used > self.minimum_coherent_box_fraction) or \ (box_fraction_used +
self.light_ray_solution[q]['traversal_box_fraction'] > 1.0):
What does "coherent large scale structure" and "box coherence" mean?
(2) When I am using minimum datasets the start point of each dataset is not same as the end point of the last dataset even if I set minimum_coherent_box_fraction = 1. Because
box_fraction_used +
self.light_ray_solution[q]['traversal_box_fraction'] > 1.0
is always satisfied when using minimum datasets. I don't know the right way to get start point to be the same as the end point of the last dataset, so I tried to change the statements in (1) to if (q == 0): I find it works for(parameters when call make_light_ray): start_point=None, start_direction=None, and start_point=None, start_direction=[1,1,0], and start_point=[0.5,0.5,0.5], start_direction=[1,1,0], But not start_point=[0.5,0.5,0.5], start_direction=None,
For the last case it shows the following error:
yt : [INFO ] 2013-09-25 04:07:18,162 Getting the binary hierarchy Parsing Hierarchy100% |||||||||||||||||||||||||||||||||||||||||||||||||||||||||| Time: 00:00:00 yt : [INFO ] 2013-09-25 04:07:18,185 Gathering a field list (this may take a moment.) yt : [INFO ] 2013-09-25 04:07:18,443 Getting field t from 0 Traceback (most recent call last): File "make_light_ray.py", line 68, in <module> njobs=-1) File "/Users/madcpf/Documents/yt/yt/analysis_modules/cosmological_observation/light_ray/light_ray.py", line 388, in make_light_ray (sub_ray['dts'] * File "/Users/madcpf/Documents/yt/yt/data_objects/data_containers.py", line 318, in __getitem__ self.get_data(key) File "/Users/madcpf/Documents/yt/yt/data_objects/data_containers.py", line 525, in get_data for grid in self._grids]) ValueError: need at least one array to concatenate
Could you please take a look when you have time?
Thank you so much!
Pengfei
2013/9/11 Britton Smith <brittonsmith@gmail.com>
Hi Pengfei,
That is right. Using plan_cosmology_splice will allow you to minimize the number of outputs needed to traverse a given redshift interval. Note, that since the start and stop positions in each dataset are random, there is no guarantee that you will not pass through the same structure more than once, although the odds are quite low.
I'm very glad to hear you're interested in working on this. For starters, you might want to read our documentation page on developing for yt, which is here: http://yt-project.org/doc/advanced/developing.html This will explain some basics about how to work in a way that will make it easiest for your contributions to be pulled into the main repository.
As for specifics on what you want to do, you should check out the yt source file: yt/analysis_modules/cosmological_observation/light_ray/light_ray.py
Have a look at the routine called _calculate_light_ray_solution. This routine is called by make_light_ray, and is responsible for creating the list of starting and ending positions for each ray segment. Starting on line 165, you can see where the random initial position for a segment is calculated. Here is where you would want to add the ability to set a fixed position. A few lines down, a random trajectory is calculated, which is used along with the length of the ray segment to determine the end point.
This should get you started. Please, let us know if you have any more questions. Additionally, you might consider joining the developers mailing list if you think you would like to continue working on yt, which would be great! Good luck!
Britton
On Wed, Sep 11, 2013 at 12:59 AM, Pengfei Chen <madcpf@gmail.com>wrote:
Hi Britton,
Thank you so much for your reply! I think I understand how LightRay works. What plan_cosmology_splice() does is to choose output redshifts so that comoving distance between them is approximately equal to but less than the comoving box size. So the light ray would not pass through the same structure in one dataset no matter how it travels. Is it right?
Also I would like to add support to the LightRay object for setting a fixed initial position. Since I never modified yt or any other large code before, could you please give me some advice? Is there any documentation or tutorials I could read?
Have a nice day!
Pengfei
2013/9/9 Britton Smith <brittonsmith@gmail.com>
Hi Pengfei,
Thank you for catching all of those errors in the source and documentation. As of now, they have all been fixed and if you do "yt update" the recipe should run as normal without the need to alter anything.
I can answer your additional questions here.
(1) What's the initial spectrum before any absorption? >
The spectrum generator only creates absorption features. Essentially, it assumes that you have already subtracted off the continuum, so effectively the input spectrum is just f(lambda) = 1.0.
> (2) Where does the light ray originate from? Is it shoot from a > random position in the first dataset? Can we set it manually like on the > surface of the box? Does it go out of the box and back into it until the > next output redshift? >
The light ray originates in a random position inside the first dataset with a random trajectory. This code assumes a periodic domain, so the ray will wrap around if it leaves the box. Once it has traversed a distance equivalent to the comoving distance to the next dataset, it terminates and begins a new ray there. Currently, you can not manually set the initial position of a LightRay, but you can also using the ray object ( http://yt-project.org/doc/analyzing/objects.html#available-objects). If you are interested in adding support to the LightRay object for setting a fixed initial position (and presumably a fixed end position), let me know and I can help you get started. If you were to do this, we would be very pleased do have you contribute this code back to the main code base.
> (3) In the output file lightraysolution.txt, what do the > parameters(dl/box, Start x/y/z, End x/y/z) mean? >
Since the start and stop positions are randomly chosen, this file shows what those random positions are for each dataset, in case you want to know that. dl/box is the fraction of the box length that was used for a ray in that dataset. For more information on this, I would suggest you check out this page:
http://yt-project.org/doc/analysis_modules/light_ray_generator.html#light-ra...
Britton
> > Sorry that I have so many questions... I wonder if there is some > existing documents or paper that have a more detailed description. > > Thank you again for your help! Have a nice weekend! > > Pengfei > > > > 2013/9/6 Britton Smith <brittonsmith@gmail.com> > >> Hi Pengfei, >> >> The near_redshift and far_redshift are free parameters for you to >> set, and just control the redshift interval over which to make the light >> cone projection. You might also want to look at the narrative >> documentation on the light cone generator, which you can find here: >> http://yt-project.org/doc/analysis_modules/light_cone_generator.html >> Note, that you need to use this with a simulation that made enough >> data outputs to connect the redshift interval you specify. The >> "enzo_tiny_cosmology" dataset available at >> http://yt-project.org/data/ was configured to work with the light >> cone recipe in the cookbook, so I would suggest downloading that. >> >> As for the error, this is happening because there is a line in your >> enzo parameter file that contains more than one "=" sign. I have just >> submitted a pull request that fixes it, so this should be fixed soon. In >> the mean time, you can simply comment out that line in your parameter file >> and everything should work fine. >> >> Britton >> >> >> On Fri, Sep 6, 2013 at 3:02 AM, Pengfei Chen <madcpf@gmail.com>wrote: >> >>> Hi all, >>> >>> I downloaded the example script of light cone projectoin( >>> http://yt-project.org/doc/cookbook/cosmological_analysis.html#light-cone-pro...) >>> to my laptop and changed the Enzo simulation parameter file and redshifts. >>> But when I ran it the following error occurred: >>> >>> File "light_cone_projection.py", line 19, in <module> >>> find_outputs=False) >>> File >>> "/Users/madcpf/Documents/dev_yt/src/yt-hg/yt/analysis_modules/cosmological_observation/light_cone/light_cone.py", >>> line 158, in __init__ >>> find_outputs=find_outputs) >>> File >>> "/Users/madcpf/Documents/dev_yt/src/yt-hg/yt/analysis_modules/cosmological_observation/cosmology_splice.py", >>> line 44, in __init__ >>> find_outputs=find_outputs) >>> File >>> "/Users/madcpf/Documents/dev_yt/src/yt-hg/yt/convenience.py", line 131, in >>> simulation >>> find_outputs=find_outputs) >>> File >>> "/Users/madcpf/Documents/dev_yt/src/yt-hg/yt/frontends/enzo/simulation_handling.py", >>> line 81, in __init__ >>> find_outputs=find_outputs) >>> File >>> "/Users/madcpf/Documents/dev_yt/src/yt-hg/yt/data_objects/time_series.py", >>> line 343, in __init__ >>> self._parse_parameter_file() >>> File >>> "/Users/madcpf/Documents/dev_yt/src/yt-hg/yt/frontends/enzo/simulation_handling.py", >>> line 273, in _parse_parameter_file >>> param, vals = (i.strip() for i in line.split("=")) >>> ValueError: need more than 1 value to unpack >>> >>> Does the parameters "near_redshift" and "far_redshift" in >>> LightCone need to be the same with "CosmologyFinalRedshift" and >>> "CosmologyInitialRedshift" in enzo parameter file? >>> Does anyone know how to solve this? >>> >>> Thank you all! >>> >>> Pengfei >>> >>> _______________________________________________ >>> yt-users mailing list >>> yt-users@lists.spacepope.org >>> http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org >>> >>> >> >> _______________________________________________ >> yt-users mailing list >> yt-users@lists.spacepope.org >> http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org >> >> > > _______________________________________________ > yt-users mailing list > yt-users@lists.spacepope.org > http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org > >
_______________________________________________ yt-users mailing list yt-users@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
_______________________________________________ yt-users mailing list yt-users@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
_______________________________________________ yt-users mailing list yt-users@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
_______________________________________________ yt-users mailing list yt-users@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
_______________________________________________ yt-users mailing list yt-users@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
_______________________________________________ yt-users mailing list yt-users@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
OK. Thank you, Britton! Pengfei On Sat, Mar 1, 2014 at 8:35 AM, Britton Smith <brittonsmith@gmail.com>wrote:
Hi Pengfei,
This can happen sometime just due to roundoff error or error that crops up when converting back and forth between redshift and time. Since the box fraction is only marginally greater than 1 for that segment of the light ray, it shouldn't really be a problem to use these datasets. You should still be able to make the light ray and just ignore the error message.
Britton
On Thu, Feb 27, 2014 at 8:20 AM, Pengfei Chen <madcpf@gmail.com> wrote:
Hi Britton,
Last time when I asked you about minimum_coherent_box_fraction in light_ray in this email, you told me that "one should stick with the option of using the smallest number of datasets as possible to traverse a redshift interval." Now I have some datasets expanding from redshift 5 to 3, which have enough outputs but didn't use the subroutine plan_cosmology_splice to plan outputs. Now when I use the subroutine Lightray I got some problems:
If I set use_minimum_datasets=True, it would have errors like this:
P014 yt : [ERROR ] 2014-02-27 01:24:21,405 Warning: box fraction required to go from z = 3.684226 to 3.659368 is 1.001284 P013 yt : [ERROR ] 2014-02-27 01:24:21,405 Full box delta z is 0.024826, but it is 0.024857 to the next data dump.
I don't know why there are such errors given that I have enough datasets. The generated light solution file is like this: http://paste.yt-project.org/show/4335/ But I'm not sure if this lightray is good to use with the above error message.
If I set use_minimum_datasets=False, all the datasets would be used and no errors are shown. The output light solution file is like this: http://paste.yt-project.org/show/4334/ Also I'm not sure if I should use this with the potential problem about "minimum_coherent_box_fraction".
Another thing I could do is to keep use_minimum_datasets=False but reduce the datasets used manually until the minimum number of datasets are used with no error messages. Could you please give me some advice about what I should do?
Thank you! Pengfei
On Mon, Sep 30, 2013 at 4:36 AM, Britton Smith <brittonsmith@gmail.com>wrote:
Hi Pengfei,
To answer your first question, the minimum_coherent_box_fraction option is a bit of a failed experiment in which one is using as many datasets as possible in the light cone, instead of using as few as possible. In this case, we use the same ray trajectory through multiple datasets until the ray has extended the length of one box. This was to avoid making cuts in halos or filaments by using thin slices of each dataset. Functionally, this works by waiting to re-randomize the ray trajectory until you've gone through a full box length. In practice, doing this produced a number of artifacts in the light cone projections so should probably be used with caution in the light rays. I think in reality, one should stick with the option of using the smallest number of datasets as possible to traverse a redshift interval.
On your second question, the problem is actually that, if you start from a random position in a periodic cube of box length L, you only return to that position after moving a distance of L if you move exactly along the axis directions. As another example, if you started in the corner of the box and went toward the opposite corner (which is the same point since it's periodic) you would have to travel a distance of sqrt(3) * L to get there. In reality, though, you probably do not want to be starting from the same point in each dataset, since you will then just be resampling the same structure over and over.
Other than that, the code itself looks just fine. Please, let me know if I haven't answered your questions.
Britton
On Wed, Sep 25, 2013 at 12:19 PM, Pengfei Chen <madcpf@gmail.com> wrote:
Hi Britton,
Thank you very much for your detailed instructions. I've added two parameters in _calculate_light_ray_solution in light_ray.py. They are 'start_point' and 'start_direction', each of which is supposed to be a 3-element list/array. Their default values are None and they can be used together or individually. The code is attached and it works. But I still have two problems:
(1) I am confused about the parameter "minimum_coherent_box_fraction", and what the following statement is trying to do:
if (q == 0) or (self.minimum_coherent_box_fraction == 0) or \ (box_fraction_used > self.minimum_coherent_box_fraction) or \ (box_fraction_used +
self.light_ray_solution[q]['traversal_box_fraction'] > 1.0):
What does "coherent large scale structure" and "box coherence" mean?
(2) When I am using minimum datasets the start point of each dataset is not same as the end point of the last dataset even if I set minimum_coherent_box_fraction = 1. Because
box_fraction_used +
self.light_ray_solution[q]['traversal_box_fraction'] > 1.0
is always satisfied when using minimum datasets. I don't know the right way to get start point to be the same as the end point of the last dataset, so I tried to change the statements in (1) to if (q == 0): I find it works for(parameters when call make_light_ray): start_point=None, start_direction=None, and start_point=None, start_direction=[1,1,0], and start_point=[0.5,0.5,0.5], start_direction=[1,1,0], But not start_point=[0.5,0.5,0.5], start_direction=None,
For the last case it shows the following error:
yt : [INFO ] 2013-09-25 04:07:18,162 Getting the binary hierarchy Parsing Hierarchy100% |||||||||||||||||||||||||||||||||||||||||||||||||||||||||| Time: 00:00:00 yt : [INFO ] 2013-09-25 04:07:18,185 Gathering a field list (this may take a moment.) yt : [INFO ] 2013-09-25 04:07:18,443 Getting field t from 0 Traceback (most recent call last): File "make_light_ray.py", line 68, in <module> njobs=-1) File "/Users/madcpf/Documents/yt/yt/analysis_modules/cosmological_observation/light_ray/light_ray.py", line 388, in make_light_ray (sub_ray['dts'] * File "/Users/madcpf/Documents/yt/yt/data_objects/data_containers.py", line 318, in __getitem__ self.get_data(key) File "/Users/madcpf/Documents/yt/yt/data_objects/data_containers.py", line 525, in get_data for grid in self._grids]) ValueError: need at least one array to concatenate
Could you please take a look when you have time?
Thank you so much!
Pengfei
2013/9/11 Britton Smith <brittonsmith@gmail.com>
Hi Pengfei,
That is right. Using plan_cosmology_splice will allow you to minimize the number of outputs needed to traverse a given redshift interval. Note, that since the start and stop positions in each dataset are random, there is no guarantee that you will not pass through the same structure more than once, although the odds are quite low.
I'm very glad to hear you're interested in working on this. For starters, you might want to read our documentation page on developing for yt, which is here: http://yt-project.org/doc/advanced/developing.html This will explain some basics about how to work in a way that will make it easiest for your contributions to be pulled into the main repository.
As for specifics on what you want to do, you should check out the yt source file: yt/analysis_modules/cosmological_observation/light_ray/light_ray.py
Have a look at the routine called _calculate_light_ray_solution. This routine is called by make_light_ray, and is responsible for creating the list of starting and ending positions for each ray segment. Starting on line 165, you can see where the random initial position for a segment is calculated. Here is where you would want to add the ability to set a fixed position. A few lines down, a random trajectory is calculated, which is used along with the length of the ray segment to determine the end point.
This should get you started. Please, let us know if you have any more questions. Additionally, you might consider joining the developers mailing list if you think you would like to continue working on yt, which would be great! Good luck!
Britton
On Wed, Sep 11, 2013 at 12:59 AM, Pengfei Chen <madcpf@gmail.com>wrote:
Hi Britton,
Thank you so much for your reply! I think I understand how LightRay works. What plan_cosmology_splice() does is to choose output redshifts so that comoving distance between them is approximately equal to but less than the comoving box size. So the light ray would not pass through the same structure in one dataset no matter how it travels. Is it right?
Also I would like to add support to the LightRay object for setting a fixed initial position. Since I never modified yt or any other large code before, could you please give me some advice? Is there any documentation or tutorials I could read?
Have a nice day!
Pengfei
2013/9/9 Britton Smith <brittonsmith@gmail.com>
> Hi Pengfei, > > Thank you for catching all of those errors in the source and > documentation. As of now, they have all been fixed and if you do "yt > update" the recipe should run as normal without the need to alter anything. > > I can answer your additional questions here. > > (1) What's the initial spectrum before any absorption? >> > > The spectrum generator only creates absorption features. > Essentially, it assumes that you have already subtracted off the continuum, > so effectively the input spectrum is just f(lambda) = 1.0. > > >> (2) Where does the light ray originate from? Is it shoot from a >> random position in the first dataset? Can we set it manually like on the >> surface of the box? Does it go out of the box and back into it until the >> next output redshift? >> > > The light ray originates in a random position inside the first > dataset with a random trajectory. This code assumes a periodic domain, so > the ray will wrap around if it leaves the box. Once it has traversed a > distance equivalent to the comoving distance to the next dataset, it > terminates and begins a new ray there. Currently, you can not manually set > the initial position of a LightRay, but you can also using the ray object ( > http://yt-project.org/doc/analyzing/objects.html#available-objects). > If you are interested in adding support to the LightRay object for setting > a fixed initial position (and presumably a fixed end position), let me know > and I can help you get started. If you were to do this, we would be very > pleased do have you contribute this code back to the main code base. > > >> (3) In the output file lightraysolution.txt, what do the >> parameters(dl/box, Start x/y/z, End x/y/z) mean? >> > > Since the start and stop positions are randomly chosen, this file > shows what those random positions are for each dataset, in case you want to > know that. dl/box is the fraction of the box length that was used for a > ray in that dataset. For more information on this, I would suggest you > check out this page: > > http://yt-project.org/doc/analysis_modules/light_ray_generator.html#light-ra... > > Britton > > >> >> Sorry that I have so many questions... I wonder if there is some >> existing documents or paper that have a more detailed description. >> >> Thank you again for your help! Have a nice weekend! >> >> Pengfei >> >> >> >> 2013/9/6 Britton Smith <brittonsmith@gmail.com> >> >>> Hi Pengfei, >>> >>> The near_redshift and far_redshift are free parameters for you to >>> set, and just control the redshift interval over which to make the light >>> cone projection. You might also want to look at the narrative >>> documentation on the light cone generator, which you can find here: >>> >>> http://yt-project.org/doc/analysis_modules/light_cone_generator.html >>> Note, that you need to use this with a simulation that made enough >>> data outputs to connect the redshift interval you specify. The >>> "enzo_tiny_cosmology" dataset available at >>> http://yt-project.org/data/ was configured to work with the light >>> cone recipe in the cookbook, so I would suggest downloading that. >>> >>> As for the error, this is happening because there is a line in >>> your enzo parameter file that contains more than one "=" sign. I have just >>> submitted a pull request that fixes it, so this should be fixed soon. In >>> the mean time, you can simply comment out that line in your parameter file >>> and everything should work fine. >>> >>> Britton >>> >>> >>> On Fri, Sep 6, 2013 at 3:02 AM, Pengfei Chen <madcpf@gmail.com>wrote: >>> >>>> Hi all, >>>> >>>> I downloaded the example script of light cone projectoin( >>>> http://yt-project.org/doc/cookbook/cosmological_analysis.html#light-cone-pro...) >>>> to my laptop and changed the Enzo simulation parameter file and redshifts. >>>> But when I ran it the following error occurred: >>>> >>>> File "light_cone_projection.py", line 19, in <module> >>>> find_outputs=False) >>>> File >>>> "/Users/madcpf/Documents/dev_yt/src/yt-hg/yt/analysis_modules/cosmological_observation/light_cone/light_cone.py", >>>> line 158, in __init__ >>>> find_outputs=find_outputs) >>>> File >>>> "/Users/madcpf/Documents/dev_yt/src/yt-hg/yt/analysis_modules/cosmological_observation/cosmology_splice.py", >>>> line 44, in __init__ >>>> find_outputs=find_outputs) >>>> File >>>> "/Users/madcpf/Documents/dev_yt/src/yt-hg/yt/convenience.py", line 131, in >>>> simulation >>>> find_outputs=find_outputs) >>>> File >>>> "/Users/madcpf/Documents/dev_yt/src/yt-hg/yt/frontends/enzo/simulation_handling.py", >>>> line 81, in __init__ >>>> find_outputs=find_outputs) >>>> File >>>> "/Users/madcpf/Documents/dev_yt/src/yt-hg/yt/data_objects/time_series.py", >>>> line 343, in __init__ >>>> self._parse_parameter_file() >>>> File >>>> "/Users/madcpf/Documents/dev_yt/src/yt-hg/yt/frontends/enzo/simulation_handling.py", >>>> line 273, in _parse_parameter_file >>>> param, vals = (i.strip() for i in line.split("=")) >>>> ValueError: need more than 1 value to unpack >>>> >>>> Does the parameters "near_redshift" and "far_redshift" in >>>> LightCone need to be the same with "CosmologyFinalRedshift" and >>>> "CosmologyInitialRedshift" in enzo parameter file? >>>> Does anyone know how to solve this? >>>> >>>> Thank you all! >>>> >>>> Pengfei >>>> >>>> _______________________________________________ >>>> yt-users mailing list >>>> yt-users@lists.spacepope.org >>>> http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org >>>> >>>> >>> >>> _______________________________________________ >>> yt-users mailing list >>> yt-users@lists.spacepope.org >>> http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org >>> >>> >> >> _______________________________________________ >> yt-users mailing list >> yt-users@lists.spacepope.org >> http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org >> >> > > _______________________________________________ > yt-users mailing list > yt-users@lists.spacepope.org > http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org > >
_______________________________________________ yt-users mailing list yt-users@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
_______________________________________________ yt-users mailing list yt-users@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
_______________________________________________ yt-users mailing list yt-users@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
_______________________________________________ yt-users mailing list yt-users@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
_______________________________________________ yt-users mailing list yt-users@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
_______________________________________________ yt-users mailing list yt-users@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
participants (2)
-
Britton Smith
-
Pengfei Chen