Re: [yt-users] Light Cone Projection

27 Feb 2014

      Hi Britton,

Last time when I asked you about minimum_coherent_box_fraction in light_ray
in this email, you told me that "one should stick with the option of using
the smallest number of datasets as possible to traverse a redshift
interval." Now I have some datasets expanding from redshift 5 to 3, which
have enough outputs but didn't use the subroutine plan_cosmology_splice to
plan outputs. Now when I use the subroutine Lightray I got some problems:

If I set use_minimum_datasets=True, it would have errors like this:

P014 yt : [ERROR    ] 2014-02-27 01:24:21,405 Warning: box fraction
required to go from z = 3.684226 to 3.659368 is 1.001284
P013 yt : [ERROR    ] 2014-02-27 01:24:21,405 Full box delta z is 0.024826,
but it is 0.024857 to the next data dump.

I don't know why there are such errors given that I have enough datasets.
The generated light solution file is like this:
http://paste.yt-project.org/show/4335/
But I'm not sure if this lightray is good to use with the above error
message.

If I set use_minimum_datasets=False, all the datasets would be used and no
errors are shown. The output light solution file is like this:
http://paste.yt-project.org/show/4334/
Also I'm not sure if I should use this with the potential problem about
"minimum_coherent_box_fraction".

Another thing I could do is to keep use_minimum_datasets=False but reduce
the datasets used manually until the minimum number of datasets are used
with no error messages. Could you please give me some advice about what I
should do?

Thank you!
Pengfei

On Mon, Sep 30, 2013 at 4:36 AM, Britton Smith wrote:
...
Hi Pengfei,
To answer your first question, the minimum_coherent_box_fraction option is
a bit of a failed experiment in which one is using as many datasets as
possible in the light cone, instead of using as few as possible.  In this
case, we use the same ray trajectory through multiple datasets until the
ray has extended the length of one box.  This was to avoid making cuts in
halos or filaments by using thin slices of each dataset.  Functionally,
this works by waiting to re-randomize the ray trajectory until you've gone
through a full box length.  In practice, doing this produced a number of
artifacts in the light cone projections so should probably be used with
caution in the light rays.  I think in reality, one should stick with the
option of using the smallest number of datasets as possible to traverse a
redshift interval.
On your second question, the problem is actually that, if you start from a
random position in a periodic cube of box length L, you only return to that
position after moving a distance of L if you move exactly along the axis
directions.  As another example, if you started in the corner of the box
and went toward the opposite corner (which is the same point since it's
periodic) you would have to travel a distance of sqrt(3) * L to get there.
In reality, though, you probably do not want to be starting from the same
point in each dataset, since you will then just be resampling the same
structure over and over.
Other than that, the code itself looks just fine.  Please, let me know if
I haven't answered your questions.
Britton
On Wed, Sep 25, 2013 at 12:19 PM, Pengfei Chen  wrote:
...
Hi Britton,
Thank you very much for your detailed instructions. I've added two
parameters in _calculate_light_ray_solution in light_ray.py. They are
'start_point' and 'start_direction', each of which is supposed to be a
3-element list/array. Their default values are None and they can be used
together or individually. The code is attached and it works. But I still
have two problems:
(1) I am confused about the parameter "minimum_coherent_box_fraction",
and what the following statement is trying to do:
if (q == 0) or (self.minimum_coherent_box_fraction == 0) or \
                    (box_fraction_used >
                     self.minimum_coherent_box_fraction) or \
                    (box_fraction_used +
                     self.light_ray_solution[q]['traversal_box_fraction']
...
1.0):
What does "coherent large scale structure" and "box coherence" mean?
(2) When I am using minimum datasets the start point of each dataset is
not same as the end point of the last dataset even if I
set minimum_coherent_box_fraction = 1. Because
box_fraction_used +
                     self.light_ray_solution[q]['traversal_box_fraction']
...
1.0
is always satisfied when using minimum datasets. I don't know the right
way to get start point to be the same as the end point of the last dataset,
so I tried to change the statements in (1) to
           if (q == 0):
I find it works for(parameters when call make_light_ray):
                  start_point=None,
                  start_direction=None,
and
                  start_point=None,
                  start_direction=[1,1,0],
and
                  start_point=[0.5,0.5,0.5],
                  start_direction=[1,1,0],
But not
                  start_point=[0.5,0.5,0.5],
                  start_direction=None,
For the last case it shows the following error:
yt : [INFO     ] 2013-09-25 04:07:18,162 Getting the binary hierarchy
Parsing Hierarchy100%
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||| Time: 00:00:00
yt : [INFO     ] 2013-09-25 04:07:18,185 Gathering a field list (this may
take a moment.)
yt : [INFO     ] 2013-09-25 04:07:18,443 Getting field t from 0
Traceback (most recent call last):
  File "make_light_ray.py", line 68, in <module>
    njobs=-1)
  File
"/Users/madcpf/Documents/yt/yt/analysis_modules/cosmological_observation/light_ray/light_ray.py",
line 388, in make_light_ray
    (sub_ray['dts'] *
  File "/Users/madcpf/Documents/yt/yt/data_objects/data_containers.py",
line 318, in __getitem__
    self.get_data(key)
  File "/Users/madcpf/Documents/yt/yt/data_objects/data_containers.py",
line 525, in get_data
    for grid in self._grids])
ValueError: need at least one array to concatenate
Could you please take a look when you have time?
Thank you so much!
Pengfei
2013/9/11 Britton Smith 
...
Hi Pengfei,
That is right.  Using plan_cosmology_splice will allow you to minimize
the number of outputs needed to traverse a given redshift interval.  Note,
that since the start and stop positions in each dataset are random, there
is no guarantee that you will not pass through the same structure more than
once, although the odds are quite low.
I'm very glad to hear you're interested in working on this.  For
starters, you might want to read our documentation page on developing for
yt, which is here:
http://yt-project.org/doc/advanced/developing.html
This will explain some basics about how to work in a way that will make
it easiest for your contributions to be pulled into the main repository.
As for specifics on what you want to do, you should check out the yt
source file:
yt/analysis_modules/cosmological_observation/light_ray/light_ray.py
Have a look at the routine called _calculate_light_ray_solution.  This
routine is called by make_light_ray, and is responsible for creating the
list of starting and ending positions for each ray segment.  Starting on
line 165, you can see where the random initial position for a segment is
calculated.  Here is where you would want to add the ability to set a fixed
position.  A few lines down, a random trajectory is calculated, which is
used along with the length of the ray segment to determine the end point.
This should get you started.  Please, let us know if you have any more
questions.  Additionally, you might consider joining the developers mailing
list if you think you would like to continue working on yt, which would be
great!  Good luck!
Britton
On Wed, Sep 11, 2013 at 12:59 AM, Pengfei Chen  wrote:
...
Hi Britton,
Thank you so much for your reply! I think I understand how LightRay
works. What plan_cosmology_splice() does is to choose output redshifts so
that comoving distance between them is approximately equal to but less than
the comoving box size. So the light ray would not pass through the same
structure in one dataset no matter how it travels. Is it right?
Also I would like to add support to the LightRay object for setting a
fixed initial position. Since I never modified yt or any other large code
before, could you please give me some advice? Is there any documentation or
tutorials I could read?
Have a nice day!
Pengfei
2013/9/9 Britton Smith 
...
Hi Pengfei,
Thank you for catching all of those errors in the source and
documentation.  As of now, they have all been fixed and if you do "yt
update" the recipe should run as normal without the need to alter anything.
I can answer your additional questions here.
(1) What's the initial spectrum before any absorption?
...
The spectrum generator only creates absorption features.  Essentially,
it assumes that you have already subtracted off the continuum, so
effectively the input spectrum is just f(lambda) = 1.0.
...
(2) Where does the light ray originate from? Is it shoot from a
random position in the first dataset? Can we set it manually like on the
surface of the box? Does it go out of the box and back into it until the
next output redshift?
The light ray originates in a random position inside the first dataset
with a random trajectory.  This code assumes a periodic domain, so the ray
will wrap around if it leaves the box.  Once it has traversed a distance
equivalent to the comoving distance to the next dataset, it terminates and
begins a new ray there.  Currently, you can not manually set the initial
position of a LightRay, but you can also using the ray object (
http://yt-project.org/doc/analyzing/objects.html#available-objects).
If you are interested in adding support to the LightRay object for setting
a fixed initial position (and presumably a fixed end position), let me know
and I can help you get started.  If you were to do this, we would be very
pleased do have you contribute this code back to the main code base.
...
(3) In the output file lightraysolution.txt, what do the
parameters(dl/box, Start x/y/z, End x/y/z) mean?
Since the start and stop positions are randomly chosen, this file
shows what those random positions are for each dataset, in case you want to
know that.  dl/box is the fraction of the box length that was used for a
ray in that dataset.  For more information on this, I would suggest you
check out this page:
http://yt-project.org/doc/analysis_modules/light_ray_generator.html#light-ra...
Britton
...
Sorry that I have so many questions... I wonder if there is some
existing documents or paper that have a more detailed description.
Thank you again for your help! Have a nice weekend!
Pengfei
2013/9/6 Britton Smith 
> Hi Pengfei,
>
> The near_redshift and far_redshift are free parameters for you to
> set, and just control the redshift interval over which to make the light
> cone projection.  You might also want to look at the narrative
> documentation on the light cone generator, which you can find here:
> http://yt-project.org/doc/analysis_modules/light_cone_generator.html
> Note, that you need to use this with a simulation that made enough
> data outputs to connect the redshift interval you specify.  The
> "enzo_tiny_cosmology" dataset available at
> http://yt-project.org/data/ was configured to work with the light
> cone recipe in the cookbook, so I would suggest downloading that.
>
> As for the error, this is happening because there is a line in your
> enzo parameter file that contains more than one "=" sign.  I have just
> submitted a pull request that fixes it, so this should be fixed soon.  In
> the mean time, you can simply comment out that line in your parameter file
> and everything should work fine.
>
> Britton
>
>
> On Fri, Sep 6, 2013 at 3:02 AM, Pengfei Chen wrote:
>
>> Hi all,
>>
>> I downloaded the example script of light cone projectoin(
>> http://yt-project.org/doc/cookbook/cosmological_analysis.html#light-cone-pro...)
>> to my laptop and changed the Enzo simulation parameter file and redshifts.
>> But when I ran it the following error occurred:
>>
>>   File "light_cone_projection.py", line 19, in <module>
>>     find_outputs=False)
>>   File
>> "/Users/madcpf/Documents/dev_yt/src/yt-hg/yt/analysis_modules/cosmological_observation/light_cone/light_cone.py",
>> line 158, in __init__
>>     find_outputs=find_outputs)
>>   File
>> "/Users/madcpf/Documents/dev_yt/src/yt-hg/yt/analysis_modules/cosmological_observation/cosmology_splice.py",
>> line 44, in __init__
>>     find_outputs=find_outputs)
>>   File
>> "/Users/madcpf/Documents/dev_yt/src/yt-hg/yt/convenience.py", line 131, in
>> simulation
>>     find_outputs=find_outputs)
>>   File
>> "/Users/madcpf/Documents/dev_yt/src/yt-hg/yt/frontends/enzo/simulation_handling.py",
>> line 81, in __init__
>>     find_outputs=find_outputs)
>>   File
>> "/Users/madcpf/Documents/dev_yt/src/yt-hg/yt/data_objects/time_series.py",
>> line 343, in __init__
>>     self._parse_parameter_file()
>>   File
>> "/Users/madcpf/Documents/dev_yt/src/yt-hg/yt/frontends/enzo/simulation_handling.py",
>> line 273, in _parse_parameter_file
>>     param, vals = (i.strip() for i in line.split("="))
>> ValueError: need more than 1 value to unpack
>>
>> Does the parameters "near_redshift" and "far_redshift" in LightCone
>> need to be the same with "CosmologyFinalRedshift" and
>> "CosmologyInitialRedshift" in enzo parameter file?
>> Does anyone know how to solve this?
>>
>> Thank you all!
>>
>> Pengfei
>>
>> _______________________________________________
>> yt-users mailing list
>> yt-users@lists.spacepope.org
>> http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
>>
>>
>
> _______________________________________________
> yt-users mailing list
> yt-users@lists.spacepope.org
> http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
>
>
_______________________________________________
yt-users mailing list
yt-users@lists.spacepope.org
http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
_______________________________________________
yt-users mailing list
yt-users@lists.spacepope.org
http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
_______________________________________________
yt-users mailing list
yt-users@lists.spacepope.org
http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
_______________________________________________
yt-users mailing list
yt-users@lists.spacepope.org
http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
_______________________________________________
yt-users mailing list
yt-users@lists.spacepope.org
http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
_______________________________________________
yt-users mailing list
yt-users@lists.spacepope.org
http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org