Hi guys,
I had an email exchange with Brian a few days ago about trying to reproduce some data he made on L7. In particular, he gave me some settings for HOP hop.c and regroup.c that cannot be changed using the current setup. Happily, I was able to reproduce the list of haloes he found using his three step serial process with parallel yt-hop once I made some hard-coded changes to HOP.
If we want to have as many options as the serial version Brian used, there would have to be about 10 optional inputs to yt-hop, which I know will kind of go against the aesthetic that Matt has for the current yt-hop. Unless there are any questions or protests, I'll try to add these options tomorrow.
_______________________________________________________ sskory@physics.ucsd.edu o__ Stephen Skory http://physics.ucsd.edu/%7Esskory/ _.>/ _Graduate Student ________________________________(_)_\(_)_______________
I'll try to add these options tomorrow.
I'll post the changes to codereview before committing...
_______________________________________________________ sskory@physics.ucsd.edu o__ Stephen Skory http://physics.ucsd.edu/%7Esskory/ _.>/ _Graduate Student ________________________________(_)_\(_)_______________
Stephen, why don't you post the list of parameters here, and I'll add them all as keywords inside the C function? I don't mind doing it. :)
-Matt
On Sun, Mar 22, 2009 at 10:04 PM, Stephen Skory stephenskory@yahoo.com wrote:
I'll try to add these options tomorrow.
I'll post the changes to codereview before committing...
_______________________________________________________ sskory@physics.ucsd.edu o__ Stephen Skory http://physics.ucsd.edu/%7Esskory/ _.>/ _Graduate Student ________________________________(_)_(_)_______________
Yt-dev mailing list Yt-dev@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
I agree that those parameters should be tunable by the C function. I had no idea there were so many, though. I'm curious to see what they all are.
On Sun, Mar 22, 2009 at 11:30 PM, Matthew Turk matthewturk@gmail.comwrote:
Stephen, why don't you post the list of parameters here, and I'll add them all as keywords inside the C function? I don't mind doing it. :)
-Matt
On Sun, Mar 22, 2009 at 10:04 PM, Stephen Skory stephenskory@yahoo.com wrote:
I'll try to add these options tomorrow.
I'll post the changes to codereview before committing...
sskory@physics.ucsd.edu o__ Stephen Skory http://physics.ucsd.edu/%7Esskory/ http://physics.ucsd.edu/%7Esskory/_.>/ _Graduate Student ________________________________(_)_(_)_______________
Yt-dev mailing list Yt-dev@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
Yt-dev mailing list Yt-dev@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
In hop.c these are the defaults:
nBucket = 16; nDens = 64; nHop = -1; fDensThresh = -1.0; bSym = 1; bTopHat = 0; nMerge = 4;
bSym and bTopHat are 0/1, nBucket,nSmooth,nDens,nHop, and nMerge are ints, fDensThresh is a float. if fDensThresh is to be >0, it needs to be multipled by the same ratio that the threshold is in SS_HopOutput for it to be correct.
nBucket - Performance tuning in kd-tree search. nDens - N_dens -- # of particles to smooth for density, default 64. nHop - N_hop -- # of particles over which to look for density max, default 16. fDensThresh - Density below which we don't assign to any group, default none. bSym - Use gather-only for spline kernal, else gather-scatter. bTopHat - Use tophat kernal for density, else cubic spline. nMerge - N_merge -- # of particles to catalog group boundaries, default 4.
in regroup.c:
c->peak_thresh = 3.0*dens_outer; c->saddle_thresh = 2.5*dens_outer; c->densthresh = dens_outer; c->mingroupsize = -1;
The defaults are fine here, which puts c->peak_thresh and c->saddle_thresh at a fixed ratio of c->densthresh, but sometimes we want to make them different. Also, the mingroupsize makes a large difference and needs to be adjustible.
c->mingroupsize -- Minimum group size. c->peak_thresh - Density required at group center c->saddle_thresh - Density required at boundary c->densthresh - Density required for a particle to be in a group
Let me know if you want help, Matt!
_______________________________________________________ sskory@physics.ucsd.edu o__ Stephen Skory http://physics.ucsd.edu/%7Esskory/ _.>/ _Graduate Student ________________________________(_)_\(_)_______________
----- Original Message ----
From: Matthew Turk matthewturk@gmail.com To: yt-dev@lists.spacepope.org Sent: Sunday, March 22, 2009 10:30:24 PM Subject: Re: [Yt-dev] yt-hop configuration improvements?
Stephen, why don't you post the list of parameters here, and I'll add them all as keywords inside the C function? I don't mind doing it. :)
-Matt
On Sun, Mar 22, 2009 at 10:04 PM, Stephen Skory wrote:
I'll try to add these options tomorrow.
I'll post the changes to codereview before committing...
sskory@physics.ucsd.edu o__ Stephen Skory http://physics.ucsd.edu/%7Esskory/ _.>/ _Graduate Student ________________________________(_)_(_)_______________
Yt-dev mailing list Yt-dev@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
Yt-dev mailing list Yt-dev@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
Hi Stephen,
I started looking over the code, and the parameters you list below, and I'm not sure most of them are really terribly relevant to the depression in the halo count you see. I'll see if I can change the code in the next little while, but for my own curiosity, what was the density threshold you ran with versus the density threshold of the plot you're trying to reproduce? And have you run with identical density thresholds, to see if that alone fixes it? That parameter is already tunable.
The other one that I think could be of relevance is the saddle joining parameter, which is passed as "-dsaddle" to regroup in the standard Eisenstein distribution. If you look in hop_regroup.c, though, you can see that it's been hard-coded by "GLB." Since I got the code from the enzo/anyl distribution, not from the Eisenstein distribution, I think we can safely say that's Greg. :)
-Matt
On Mon, Mar 23, 2009 at 7:02 AM, Stephen Skory stephenskory@yahoo.com wrote:
In hop.c these are the defaults:
nBucket = 16; nDens = 64; nHop = -1; fDensThresh = -1.0; bSym = 1; bTopHat = 0; nMerge = 4;
bSym and bTopHat are 0/1, nBucket,nSmooth,nDens,nHop, and nMerge are ints, fDensThresh is a float. if fDensThresh is to be >0, it needs to be multipled by the same ratio that the threshold is in SS_HopOutput for it to be correct.
nBucket - Performance tuning in kd-tree search. nDens - N_dens -- # of particles to smooth for density, default 64. nHop - N_hop -- # of particles over which to look for density max, default 16. fDensThresh - Density below which we don't assign to any group, default none. bSym - Use gather-only for spline kernal, else gather-scatter. bTopHat - Use tophat kernal for density, else cubic spline. nMerge - N_merge -- # of particles to catalog group boundaries, default 4.
in regroup.c:
c->peak_thresh = 3.0*dens_outer; c->saddle_thresh = 2.5*dens_outer; c->densthresh = dens_outer; c->mingroupsize = -1;
The defaults are fine here, which puts c->peak_thresh and c->saddle_thresh at a fixed ratio of c->densthresh, but sometimes we want to make them different. Also, the mingroupsize makes a large difference and needs to be adjustible.
c->mingroupsize -- Minimum group size. c->peak_thresh - Density required at group center c->saddle_thresh - Density required at boundary c->densthresh - Density required for a particle to be in a group
Let me know if you want help, Matt!
_______________________________________________________ sskory@physics.ucsd.edu o__ Stephen Skory http://physics.ucsd.edu/%7Esskory/ _.>/ _Graduate Student ________________________________(_)_(_)_______________
----- Original Message ----
From: Matthew Turk matthewturk@gmail.com To: yt-dev@lists.spacepope.org Sent: Sunday, March 22, 2009 10:30:24 PM Subject: Re: [Yt-dev] yt-hop configuration improvements?
Stephen, why don't you post the list of parameters here, and I'll add them all as keywords inside the C function? I don't mind doing it. :)
-Matt
On Sun, Mar 22, 2009 at 10:04 PM, Stephen Skory wrote:
I'll try to add these options tomorrow.
I'll post the changes to codereview before committing...
_______________________________________________________ sskory@physics.ucsd.edu o__ Stephen Skory http://physics.ucsd.edu/%7Esskory/ _.>/ _Graduate Student ________________________________(_)_(_)_______________
Yt-dev mailing list Yt-dev@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
Yt-dev mailing list Yt-dev@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
Yt-dev mailing list Yt-dev@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
Does anyone have any idea how long parallel hop should take on one of the L7 light cone datasets? I've got it running on the last one, RD0036, on 32 cores for the last 90 minutes on kraken? Stephen, you probably know better than anybody. Any ideas?
Thanks,
Britton
On Mon, Mar 23, 2009 at 9:52 AM, Matthew Turk matthewturk@gmail.com wrote:
Hi Stephen,
I started looking over the code, and the parameters you list below, and I'm not sure most of them are really terribly relevant to the depression in the halo count you see. I'll see if I can change the code in the next little while, but for my own curiosity, what was the density threshold you ran with versus the density threshold of the plot you're trying to reproduce? And have you run with identical density thresholds, to see if that alone fixes it? That parameter is already tunable.
The other one that I think could be of relevance is the saddle joining parameter, which is passed as "-dsaddle" to regroup in the standard Eisenstein distribution. If you look in hop_regroup.c, though, you can see that it's been hard-coded by "GLB." Since I got the code from the enzo/anyl distribution, not from the Eisenstein distribution, I think we can safely say that's Greg. :)
-Matt
On Mon, Mar 23, 2009 at 7:02 AM, Stephen Skory stephenskory@yahoo.com wrote:
In hop.c these are the defaults:
nBucket = 16; nDens = 64; nHop = -1; fDensThresh = -1.0; bSym = 1; bTopHat = 0; nMerge = 4;
bSym and bTopHat are 0/1, nBucket,nSmooth,nDens,nHop, and nMerge are
ints, fDensThresh is a float. if fDensThresh is to be >0, it needs to be multipled by the same ratio that the threshold is in SS_HopOutput for it to be correct.
nBucket - Performance tuning in kd-tree search. nDens - N_dens -- # of particles to smooth for density, default 64. nHop - N_hop -- # of particles over which to look for density max,
default 16.
fDensThresh - Density below which we don't assign to any group, default
none.
bSym - Use gather-only for spline kernal, else gather-scatter. bTopHat - Use tophat kernal for density, else cubic spline. nMerge - N_merge -- # of particles to catalog group boundaries, default
in regroup.c:
c->peak_thresh = 3.0*dens_outer; c->saddle_thresh = 2.5*dens_outer; c->densthresh = dens_outer; c->mingroupsize = -1;
The defaults are fine here, which puts c->peak_thresh and
c->saddle_thresh at a fixed ratio of c->densthresh, but sometimes we want to make them different. Also, the mingroupsize makes a large difference and needs to be adjustible.
c->mingroupsize -- Minimum group size. c->peak_thresh - Density required at group center c->saddle_thresh - Density required at boundary c->densthresh - Density required for a particle to be in a group
Let me know if you want help, Matt!
sskory@physics.ucsd.edu o__ Stephen Skory http://physics.ucsd.edu/%7Esskory/ http://physics.ucsd.edu/%7Esskory/_.>/ _Graduate Student ________________________________(_)_(_)_______________
----- Original Message ----
From: Matthew Turk matthewturk@gmail.com To: yt-dev@lists.spacepope.org Sent: Sunday, March 22, 2009 10:30:24 PM Subject: Re: [Yt-dev] yt-hop configuration improvements?
Stephen, why don't you post the list of parameters here, and I'll add them all as keywords inside the C function? I don't mind doing it. :)
-Matt
On Sun, Mar 22, 2009 at 10:04 PM, Stephen Skory wrote:
I'll try to add these options tomorrow.
I'll post the changes to codereview before committing...
sskory@physics.ucsd.edu o__ Stephen Skory http://physics.ucsd.edu/%7Esskory/ http://physics.ucsd.edu/%7Esskory/_.>/ _Graduate Student ________________________________(_)_(_)_______________
Yt-dev mailing list Yt-dev@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
Yt-dev mailing list Yt-dev@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
Yt-dev mailing list Yt-dev@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
Yt-dev mailing list Yt-dev@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
Britton,
Does anyone have any idea how long parallel hop should take on one of the L7 light cone datasets? I've got it running on the last one, RD0036, on 32 cores for the last 90 minutes on kraken? Stephen, you probably know better than anybody. Any ideas?
I typically have been running with 64 cores on Ranger or Kraken, and that typically takes about 5 minutes per core to do the actual HOP. The rest of the time is reading data, and Ranger is noticeably faster than Kraken. I'd say it's about 20-25 minutes on Ranger and more like 30 on Kraken. 90 minutes is too much, unless Kraken's disk is taking a dump, which is entirely possible. I've been seeing lots of sluggish periods on Kraken's scratch disk. Oh, I should add that I am also giving 8GB per MPI task on both of these machines.
_______________________________________________________ sskory@physics.ucsd.edu o__ Stephen Skory http://physics.ucsd.edu/%7Esskory/ _.>/ _Graduate Student ________________________________(_)_\(_)_______________
Matt,
but for my own curiosity, what was the density threshold you ran with versus the density threshold of the plot you're trying to reproduce? And have you run with identical density thresholds, to see if that alone fixes it? That parameter is already tunable.
I actually haven't gotten the plot yet that Brian made due to my HaloProfiler problem (which I'm about to start working on). I have got his list of haloes reproduced. The plot I made earlier was with a threshold=160.0, and Brian ran at 80.0. He also ran with a lower minimum group size in regroup (2 versus 20), which is something that is currently hard coded in yt-hop. So the lower threshold and the lower minimum group size is what I needed.
The other one that I think could be of relevance is the saddle joining parameter, which is passed as "-dsaddle" to regroup in the standard Eisenstein distribution. If you look in hop_regroup.c, though, you can see that it's been hard-coded by "GLB." Since I got the code from the enzo/anyl distribution, not from the Eisenstein distribution, I think we can safely say that's Greg. :)
It would be nice to make that a parameter then.
So in summary, I see what you're saying. I'd then add at minimum, the minimum group size, and -dsaddle.
_______________________________________________________ sskory@physics.ucsd.edu o__ Stephen Skory http://physics.ucsd.edu/%7Esskory/ _.>/ _Graduate Student ________________________________(_)_\(_)_______________
Hi Stephen,
I actually haven't gotten the plot yet that Brian made due to my HaloProfiler problem (which I'm about to start working on). I have got his list of haloes reproduced. The plot I made earlier was with a threshold=160.0, and Brian ran at 80.0. He also ran with a lower minimum group size in regroup (2 versus 20), which is something that is currently hard coded in yt-hop. So the lower threshold and the lower minimum group size is what I needed.
The hop website says two things of note here:
""" -mingroup int Only groups with this many particles or more will be outputted. Default: 10
Remember that HOP is biased against finding groups smaller than N_dens or N_hop; hence, setting -mingroup to be tiny is usually pointless. """
I'll make this a parameter, but I am going to leave it at -1 for default, which sets it to 10. This is what it's been doing... I've also set the dsaddle by default to be what Greg suggested, 2.5*threshold.
So in summary, I see what you're saying. I'd then add at minimum, the minimum group size, and -dsaddle.
Done, changeset is on bitbucket:
http://bitbucket.org/MatthewTurk/yt/changeset/4f9a83beabb2/
Strictly speaking I don't need to set self.mingroupsize and self.dsaddle in the HaloFinder, but that's not a big deal. HopOutput.py doesn't need these changes because they're optional arguments.
-Matt
-
Britton Smith -
Matthew Turk -
Stephen Skory