Hi Stephen, I read http://lists.spacepope.org/pipermail/yt-users-spacepope.org/2009-March/00011... Do you remember the cosmology parameters for your run: particles, grid, box, refinement, AMR ON or OFF ? I have run a 200 Mpc/h box, 512^3 particle, 512^3 grid, AMR-OFF simulation and I get about 30,000 halos using HOP. When I try to finding virial halos, I get only 500. I may have to increase the grid to 1024^3. Let me know how you got 40,000 virial halos. Thanks Shankar
Hi Shankar, Do you have a line that looks like: hp.add_halo_filter(HP.VirialFilter,must_be_virialized=True, overdensity_field='ActualOverdensity', virial_overdensity=200, virial_filters=[['TotalMassMsun','>=','1e14']], virial_quantities=['TotalMassMsun','RadiusMpc']) If so, please post it. The number of halos > 10^14 Msun is fairly close to 500 in a 200 Mpc box. If you want the halo profiler to use all halos, set virial_filters=[['TotalMassMsun','>=','1e14']] accordingly. Sam On Sun, Feb 28, 2010 at 11:08 AM, Agarwal, Shankar <sagarwal@ku.edu> wrote:
Hi Stephen,
I read http://lists.spacepope.org/pipermail/yt-users-spacepope.org/2009-March/00011...
Do you remember the cosmology parameters for your run: particles, grid, box, refinement, AMR ON or OFF ?
I have run a 200 Mpc/h box, 512^3 particle, 512^3 grid, AMR-OFF simulation and I get about 30,000 halos using HOP. When I try to finding virial halos, I get only 500. I may have to increase the grid to 1024^3.
Let me know how you got 40,000 virial halos.
Thanks Shankar _______________________________________________ yt-users mailing list yt-users@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
-- Samuel W. Skillman DOE Computational Science Graduate Fellow Center for Astrophysics and Space Astronomy University of Colorado at Boulder samuel.skillman[at]colorado.edu
Hi Sam, Here is how I am using it right now... hp.add_halo_filter(HP.VirialFilter,must_be_virialized=True, overdensity_field='ActualOverdensity', virial_overdensity=200, virial_filters=[['TotalMassMsun','>=','5e11']], virial_quantities=['TotalMassMsun','RadiusMpc']) hp.make_profiles(filename="VirialHaloes.out") And i get 500. Do you think it has something to do with the grid spacing (for 512 ^3 grid in 200 Box, it is 390 Kpc) HaloProfiler may be skipping small halos completely since they are smaller than 390 Kpc. Shankar -----Original Message----- From: yt-users-bounces@lists.spacepope.org on behalf of Sam Skillman Sent: Sun 2/28/2010 12:34 PM To: Discussion of the yt analysis package Subject: Re: [yt-users] Dealing with HOP groups Hi Shankar, Do you have a line that looks like: hp.add_halo_filter(HP.VirialFilter,must_be_virialized=True, overdensity_field='ActualOverdensity', virial_overdensity=200, virial_filters=[['TotalMassMsun','>=','1e14']], virial_quantities=['TotalMassMsun','RadiusMpc']) If so, please post it. The number of halos > 10^14 Msun is fairly close to 500 in a 200 Mpc box. If you want the halo profiler to use all halos, set virial_filters=[['TotalMassMsun','>=','1e14']] accordingly. Sam On Sun, Feb 28, 2010 at 11:08 AM, Agarwal, Shankar <sagarwal@ku.edu> wrote:
Hi Stephen,
I read http://lists.spacepope.org/pipermail/yt-users-spacepope.org/2009-March/00011...
Do you remember the cosmology parameters for your run: particles, grid, box, refinement, AMR ON or OFF ?
I have run a 200 Mpc/h box, 512^3 particle, 512^3 grid, AMR-OFF simulation and I get about 30,000 halos using HOP. When I try to finding virial halos, I get only 500. I may have to increase the grid to 1024^3.
Let me know how you got 40,000 virial halos.
Thanks Shankar _______________________________________________ yt-users mailing list yt-users@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
-- Samuel W. Skillman DOE Computational Science Graduate Fellow Center for Astrophysics and Space Astronomy University of Colorado at Boulder samuel.skillman[at]colorado.edu
Actually yes. You could check this yourself by examining the last column in HopAnalysis out. If max_r < 2*(1/512)=3.9e-3 then yes, it will skip it. In one of my simulations, a 5e11 Msun halo has a max_r of 2.64e-03 corresponding to 168 kpc/h. The halo profiler was not built for profiling halos with < 2 cells in radius. In my mind you have at least four options: 1. Run with AMR to resolve the dark matter potential so that you can use the halo profiler as built. Moderate computational cost. I'm guessing this was the case for Stephen's simulation where a decent fraction of the halos in the HopAnalysis were able to be profiled. 2. Run with a gigantic unigrid such that you can resolve these halos with > 2 (and preferably many) cells. This is a very computationally expensive route as you'd need more than a 2048^3 unigrid in a 200 Mpc box. I do not recommend this option. 3. Write a dark matter particle profiler that uses the particle positions (and not grid cells) to bin up the profile. Little computational cost but requires coding on your part. 4. Run the same size unigrid on a smaller physical volume to capture a different range of the mass function. Cheers, Sam On Sun, Feb 28, 2010 at 12:07 PM, Agarwal, Shankar <sagarwal@ku.edu> wrote:
Hi Sam,
Here is how I am using it right now...
hp.add_halo_filter(HP.VirialFilter,must_be_virialized=True, overdensity_field='ActualOverdensity', virial_overdensity=200, virial_filters=[['TotalMassMsun','>=','5e11']], virial_quantities=['TotalMassMsun','RadiusMpc']) hp.make_profiles(filename="VirialHaloes.out")
And i get 500.
Do you think it has something to do with the grid spacing (for 512 ^3 grid in 200 Box, it is 390 Kpc)
HaloProfiler may be skipping small halos completely since they are smaller than 390 Kpc.
Shankar
-----Original Message----- From: yt-users-bounces@lists.spacepope.org on behalf of Sam Skillman Sent: Sun 2/28/2010 12:34 PM To: Discussion of the yt analysis package Subject: Re: [yt-users] Dealing with HOP groups
Hi Shankar,
Do you have a line that looks like: hp.add_halo_filter(HP.VirialFilter,must_be_virialized=True, overdensity_field='ActualOverdensity', virial_overdensity=200, virial_filters=[['TotalMassMsun','>=','1e14']], virial_quantities=['TotalMassMsun','RadiusMpc'])
If so, please post it. The number of halos > 10^14 Msun is fairly close to 500 in a 200 Mpc box. If you want the halo profiler to use all halos, set virial_filters=[['TotalMassMsun','>=','1e14']] accordingly.
Sam
On Sun, Feb 28, 2010 at 11:08 AM, Agarwal, Shankar <sagarwal@ku.edu> wrote:
Hi Stephen,
I read
http://lists.spacepope.org/pipermail/yt-users-spacepope.org/2009-March/00011...
Do you remember the cosmology parameters for your run: particles, grid, box, refinement, AMR ON or OFF ?
I have run a 200 Mpc/h box, 512^3 particle, 512^3 grid, AMR-OFF
simulation
and I get about 30,000 halos using HOP. When I try to finding virial halos, I get only 500. I may have to increase the grid to 1024^3.
Let me know how you got 40,000 virial halos.
Thanks Shankar _______________________________________________ yt-users mailing list yt-users@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
-- Samuel W. Skillman DOE Computational Science Graduate Fellow Center for Astrophysics and Space Astronomy University of Colorado at Boulder samuel.skillman[at]colorado.edu
_______________________________________________ yt-users mailing list yt-users@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
-- Samuel W. Skillman DOE Computational Science Graduate Fellow Center for Astrophysics and Space Astronomy University of Colorado at Boulder samuel.skillman[at]colorado.edu
All,
3. Write a dark matter particle profiler that uses the particle positions (and not grid cells) to bin up the profile. Little computational cost but requires coding on your part.
This is already in yt, as of 1.6, see .virial_mass() or .virial_radius(): http://yt.enzotools.org/doc/extensions/running_halofinder.html#halo-data-acc... _______________________________________________________ sskory@physics.ucsd.edu o__ Stephen Skory http://physics.ucsd.edu/~sskory/ _.>/ _Graduate Student ________________________________(_)_\(_)_______________
Hi, http://yt.enzotools.org/doc/extensions/running_halofinder.html#halo-data-acc... I get the following error while trying to access halos... halos.write_particle_list("Hop_particle_list.dat") AttributeError: 'HOPHaloFinder' object has no attribute 'write_particle_list' Shankar -----Original Message----- From: yt-users-bounces@lists.spacepope.org on behalf of Stephen Skory Sent: Sun 2/28/2010 1:51 PM To: Discussion of the yt analysis package Subject: Re: [yt-users] Dealing with HOP groups All,
3. Write a dark matter particle profiler that uses the particle positions (and not grid cells) to bin up the profile. Little computational cost but requires coding on your part.
This is already in yt, as of 1.6, see .virial_mass() or .virial_radius(): http://yt.enzotools.org/doc/extensions/running_halofinder.html#halo-data-acc... _______________________________________________________ sskory@physics.ucsd.edu o__ Stephen Skory http://physics.ucsd.edu/~sskory/ _.>/ _Graduate Student ________________________________(_)_\(_)_______________ _______________________________________________ yt-users mailing list yt-users@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
Hi Shankar, It works on the individual halos. You could do this with: halos[0].write_particle_list ... or you could do: for i,halo in enumerate(halos): halo.write_particle_list("halo_%08i.dat" % i) -Matt On Mon, Mar 1, 2010 at 3:21 PM, Agarwal, Shankar <sagarwal@ku.edu> wrote:
Hi,
http://yt.enzotools.org/doc/extensions/running_halofinder.html#halo-data-acc...
I get the following error while trying to access halos...
halos.write_particle_list("Hop_particle_list.dat") AttributeError: 'HOPHaloFinder' object has no attribute 'write_particle_list'
Shankar
-----Original Message----- From: yt-users-bounces@lists.spacepope.org on behalf of Stephen Skory Sent: Sun 2/28/2010 1:51 PM To: Discussion of the yt analysis package Subject: Re: [yt-users] Dealing with HOP groups
All,
3. Write a dark matter particle profiler that uses the particle positions (and not grid cells) to bin up the profile. Little computational cost but requires coding on your part.
This is already in yt, as of 1.6, see .virial_mass() or .virial_radius():
http://yt.enzotools.org/doc/extensions/running_halofinder.html#halo-data-acc...
_______________________________________________________ sskory@physics.ucsd.edu o__ Stephen Skory http://physics.ucsd.edu/~sskory/ _.>/ _Graduate Student ________________________________(_)_\(_)_______________
_______________________________________________ yt-users mailing list yt-users@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
_______________________________________________ yt-users mailing list yt-users@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
Shankar,
I get the following error while trying to access halos...
halos.write_particle_list("Hop_particle_list.dat") AttributeError: 'HOPHaloFinder' object has no attribute 'write_particle_list'
It looks like that's a typo in the docs, it should be halos.write_particle_lists("Hop_particle_list.dat") with an 's'. Thanks for pointing that out! _______________________________________________________ sskory@physics.ucsd.edu o__ Stephen Skory http://physics.ucsd.edu/~sskory/ _.>/ _Graduate Student ________________________________(_)_\(_)_______________
Hi Sam, If I choose option 1, I guess I will need a grid of 512*4=2048. That will resolve 200Mpc/2048=97 Kpc In that case, MaximumRefinementLevel = 2 CellFlaggingMethod = 2 4 MinimumOverDensityForRefinement = 2.0 4.0 How important is MinimumOverDensityForRefinement ? I can't decide why I should not use 4.0 8.0 to reduce computing time. And how do these two numbers relate to the VirialOverdensity=200? Is this something that I will have to figure out only by running a simulation? I am hoping somebody already has experience with large box simulations. Shankar -----Original Message----- From: yt-users-bounces@lists.spacepope.org on behalf of Sam Skillman Sent: Sun 2/28/2010 1:39 PM To: Discussion of the yt analysis package Subject: Re: [yt-users] Dealing with HOP groups Actually yes. You could check this yourself by examining the last column in HopAnalysis out. If max_r < 2*(1/512)=3.9e-3 then yes, it will skip it. In one of my simulations, a 5e11 Msun halo has a max_r of 2.64e-03 corresponding to 168 kpc/h. The halo profiler was not built for profiling halos with < 2 cells in radius. In my mind you have at least four options: 1. Run with AMR to resolve the dark matter potential so that you can use the halo profiler as built. Moderate computational cost. I'm guessing this was the case for Stephen's simulation where a decent fraction of the halos in the HopAnalysis were able to be profiled. 2. Run with a gigantic unigrid such that you can resolve these halos with > 2 (and preferably many) cells. This is a very computationally expensive route as you'd need more than a 2048^3 unigrid in a 200 Mpc box. I do not recommend this option. 3. Write a dark matter particle profiler that uses the particle positions (and not grid cells) to bin up the profile. Little computational cost but requires coding on your part. 4. Run the same size unigrid on a smaller physical volume to capture a different range of the mass function. Cheers, Sam On Sun, Feb 28, 2010 at 12:07 PM, Agarwal, Shankar <sagarwal@ku.edu> wrote:
Hi Sam,
Here is how I am using it right now...
hp.add_halo_filter(HP.VirialFilter,must_be_virialized=True, overdensity_field='ActualOverdensity', virial_overdensity=200, virial_filters=[['TotalMassMsun','>=','5e11']], virial_quantities=['TotalMassMsun','RadiusMpc']) hp.make_profiles(filename="VirialHaloes.out")
And i get 500.
Do you think it has something to do with the grid spacing (for 512 ^3 grid in 200 Box, it is 390 Kpc)
HaloProfiler may be skipping small halos completely since they are smaller than 390 Kpc.
Shankar
-----Original Message----- From: yt-users-bounces@lists.spacepope.org on behalf of Sam Skillman Sent: Sun 2/28/2010 12:34 PM To: Discussion of the yt analysis package Subject: Re: [yt-users] Dealing with HOP groups
Hi Shankar,
Do you have a line that looks like: hp.add_halo_filter(HP.VirialFilter,must_be_virialized=True, overdensity_field='ActualOverdensity', virial_overdensity=200, virial_filters=[['TotalMassMsun','>=','1e14']], virial_quantities=['TotalMassMsun','RadiusMpc'])
If so, please post it. The number of halos > 10^14 Msun is fairly close to 500 in a 200 Mpc box. If you want the halo profiler to use all halos, set virial_filters=[['TotalMassMsun','>=','1e14']] accordingly.
Sam
On Sun, Feb 28, 2010 at 11:08 AM, Agarwal, Shankar <sagarwal@ku.edu> wrote:
Hi Stephen,
I read
http://lists.spacepope.org/pipermail/yt-users-spacepope.org/2009-March/00011...
Do you remember the cosmology parameters for your run: particles, grid, box, refinement, AMR ON or OFF ?
I have run a 200 Mpc/h box, 512^3 particle, 512^3 grid, AMR-OFF
simulation
and I get about 30,000 halos using HOP. When I try to finding virial halos, I get only 500. I may have to increase the grid to 1024^3.
Let me know how you got 40,000 virial halos.
Thanks Shankar _______________________________________________ yt-users mailing list yt-users@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
-- Samuel W. Skillman DOE Computational Science Graduate Fellow Center for Astrophysics and Space Astronomy University of Colorado at Boulder samuel.skillman[at]colorado.edu
_______________________________________________ yt-users mailing list yt-users@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
-- Samuel W. Skillman DOE Computational Science Graduate Fellow Center for Astrophysics and Space Astronomy University of Colorado at Boulder samuel.skillman[at]colorado.edu
Since this isn't yt related question anymore, let's move this to a private conversation. Sam On Sun, Feb 28, 2010 at 1:45 PM, Agarwal, Shankar <sagarwal@ku.edu> wrote:
Hi Sam,
If I choose option 1, I guess I will need a grid of 512*4=2048. That will resolve 200Mpc/2048=97 Kpc
In that case,
MaximumRefinementLevel = 2 CellFlaggingMethod = 2 4 MinimumOverDensityForRefinement = 2.0 4.0
How important is MinimumOverDensityForRefinement ? I can't decide why I should not use 4.0 8.0 to reduce computing time. And how do these two numbers relate to the VirialOverdensity=200?
Is this something that I will have to figure out only by running a simulation? I am hoping somebody already has experience with large box simulations.
Shankar
-----Original Message----- From: yt-users-bounces@lists.spacepope.org on behalf of Sam Skillman Sent: Sun 2/28/2010 1:39 PM To: Discussion of the yt analysis package Subject: Re: [yt-users] Dealing with HOP groups
Actually yes. You could check this yourself by examining the last column in HopAnalysis out. If max_r < 2*(1/512)=3.9e-3 then yes, it will skip it. In one of my simulations, a 5e11 Msun halo has a max_r of 2.64e-03 corresponding to 168 kpc/h. The halo profiler was not built for profiling halos with < 2 cells in radius. In my mind you have at least four options:
1. Run with AMR to resolve the dark matter potential so that you can use the halo profiler as built. Moderate computational cost. I'm guessing this was the case for Stephen's simulation where a decent fraction of the halos in the HopAnalysis were able to be profiled. 2. Run with a gigantic unigrid such that you can resolve these halos with
2 (and preferably many) cells. This is a very computationally expensive route as you'd need more than a 2048^3 unigrid in a 200 Mpc box. I do not recommend this option. 3. Write a dark matter particle profiler that uses the particle positions (and not grid cells) to bin up the profile. Little computational cost but requires coding on your part. 4. Run the same size unigrid on a smaller physical volume to capture a different range of the mass function.
Cheers, Sam
On Sun, Feb 28, 2010 at 12:07 PM, Agarwal, Shankar <sagarwal@ku.edu> wrote:
Hi Sam,
Here is how I am using it right now...
hp.add_halo_filter(HP.VirialFilter,must_be_virialized=True, overdensity_field='ActualOverdensity', virial_overdensity=200, virial_filters=[['TotalMassMsun','>=','5e11']], virial_quantities=['TotalMassMsun','RadiusMpc']) hp.make_profiles(filename="VirialHaloes.out")
And i get 500.
Do you think it has something to do with the grid spacing (for 512 ^3 grid in 200 Box, it is 390 Kpc)
HaloProfiler may be skipping small halos completely since they are smaller than 390 Kpc.
Shankar
-----Original Message----- From: yt-users-bounces@lists.spacepope.org on behalf of Sam Skillman Sent: Sun 2/28/2010 12:34 PM To: Discussion of the yt analysis package Subject: Re: [yt-users] Dealing with HOP groups
Hi Shankar,
Do you have a line that looks like: hp.add_halo_filter(HP.VirialFilter,must_be_virialized=True, overdensity_field='ActualOverdensity', virial_overdensity=200, virial_filters=[['TotalMassMsun','>=','1e14']], virial_quantities=['TotalMassMsun','RadiusMpc'])
If so, please post it. The number of halos > 10^14 Msun is fairly close to 500 in a 200 Mpc box. If you want the halo profiler to use all halos, set virial_filters=[['TotalMassMsun','>=','1e14']] accordingly.
Sam
On Sun, Feb 28, 2010 at 11:08 AM, Agarwal, Shankar <sagarwal@ku.edu> wrote:
Hi Stephen,
I read
http://lists.spacepope.org/pipermail/yt-users-spacepope.org/2009-March/00011...
Do you remember the cosmology parameters for your run: particles, grid, box, refinement, AMR ON or OFF ?
I have run a 200 Mpc/h box, 512^3 particle, 512^3 grid, AMR-OFF
simulation
and I get about 30,000 halos using HOP. When I try to finding virial halos, I get only 500. I may have to increase the grid to 1024^3.
Let me know how you got 40,000 virial halos.
Thanks Shankar _______________________________________________ yt-users mailing list yt-users@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
-- Samuel W. Skillman DOE Computational Science Graduate Fellow Center for Astrophysics and Space Astronomy University of Colorado at Boulder samuel.skillman[at]colorado.edu
_______________________________________________ yt-users mailing list yt-users@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
-- Samuel W. Skillman DOE Computational Science Graduate Fellow Center for Astrophysics and Space Astronomy University of Colorado at Boulder samuel.skillman[at]colorado.edu
_______________________________________________ yt-users mailing list yt-users@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
-- Samuel W. Skillman DOE Computational Science Graduate Fellow Center for Astrophysics and Space Astronomy University of Colorado at Boulder samuel.skillman[at]colorado.edu
participants (4)
-
Agarwal, Shankar -
Matthew Turk -
Sam Skillman -
Stephen Skory