Hi everyone,
I think this question is mainly for Stephen and possibly John, if he's on this list. I was trying to make a direct comparison between the halos found with yt FOF and with John Wise's FOF for the same simulation. The mass functions definitely look the same, but the particle membership for any given halo is totally different. I'm sure this is just a matter of the particle indexing. Does yt FOF use the particle IDs from enzo, or some different system? Any ideas for how I might correct this issue would be greatly appreciated.
Thanks, Britton
Britton,
I think this question is mainly for Stephen and possibly John, if he's on this list. I was trying to make a direct comparison between the halos found with yt FOF and with John Wise's FOF for the same simulation. The mass functions definitely look the same, but the particle membership for any given halo is totally different. I'm sure this is just a matter of the particle indexing. Does yt FOF use the particle IDs from enzo, or some different system? Any ideas for how I might correct this issue would be greatly appreciated.
Unless there is some bug inside of the FOF code, or in the interface between yt and the FOF code, it should be the IDs from the enzo dataset, just like the IDs from a run of yt HOP. I never did extensive testing on yt FOF so it's possible there is a bug. Have you checked against the raw enzo data to see which is correct?
Thanks for digging into this stuff, by the way!
sskory@physics.ucsd.edu o__ Stephen Skory http://physics.ucsd.edu/~sskory/ _.>/ _Graduate Student ________________________________(_)_\(_)_______________
Stephen,
I'll check the yt FOF IDs against Enzo right now. The halo output of the code seems just fine, so if there's a bug, it's minor. I'll let you know what I find.
Thanks for your input.
Britton
On Thu, Oct 15, 2009 at 9:53 AM, Stephen Skory stephenskory@yahoo.comwrote:
Britton,
I think this question is mainly for Stephen and possibly John, if he's on this list. I was trying to make a direct comparison between the halos found with yt FOF and with John Wise's FOF for the same simulation. The mass functions definitely look the same, but the particle membership for any given halo is totally different. I'm sure this is just a matter of the particle indexing. Does yt FOF use the particle IDs from enzo, or some different system? Any ideas for how I might correct this issue would be greatly appreciated.
Unless there is some bug inside of the FOF code, or in the interface between yt and the FOF code, it should be the IDs from the enzo dataset, just like the IDs from a run of yt HOP. I never did extensive testing on yt FOF so it's possible there is a bug. Have you checked against the raw enzo data to see which is correct?
Thanks for digging into this stuff, by the way!
sskory@physics.ucsd.edu o__ Stephen Skory http://physics.ucsd.edu/~sskory/ http://physics.ucsd.edu/%7Esskory/ _.>/ _Graduate Student ________________________________(_)_(_)_______________
Yt-dev mailing list Yt-dev@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
Britton,
I'll check the yt FOF IDs against Enzo right now. The halo output of the code seems just fine, so if there's a bug, it's minor. I'll let you know what I find.
I remembered that the kD tree yt FOF uses is a bit inaccurate, and that could be a source of difference between the two sets of FOF haloes. But you said the differences were huge, and the kD inaccuracy should give only small differences, especially in FOF. Just a thought.
sskory@physics.ucsd.edu o__ Stephen Skory http://physics.ucsd.edu/~sskory/ _.>/ _Graduate Student ________________________________(_)_\(_)_______________
Stephen,
It looks like if I take any random particle in a halo made with yt FOF, it's in a completely different halo in inline FOF, or not in any halo at all. Michele is checking this out right now, but it basically looks like the particle lists for the same halo found with the two different methods are completely different.
Britton
On Thu, Oct 15, 2009 at 10:22 AM, Stephen Skory stephenskory@yahoo.comwrote:
Britton,
I'll check the yt FOF IDs against Enzo right now. The halo output of the code seems just fine, so if there's a bug, it's minor. I'll let you know what I find.
I remembered that the kD tree yt FOF uses is a bit inaccurate, and that could be a source of difference between the two sets of FOF haloes. But you said the differences were huge, and the kD inaccuracy should give only small differences, especially in FOF. Just a thought.
sskory@physics.ucsd.edu o__ Stephen Skory http://physics.ucsd.edu/~sskory/ http://physics.ucsd.edu/%7Esskory/ _.>/ _Graduate Student ________________________________(_)_(_)_______________
Yt-dev mailing list Yt-dev@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
Britton,
What about the halos themselves? How do they compare to each other? Can you perform a simple N^2 check on the halos for closest neighbor in the other method, compare the masses and radii, and see if this is in fact a halo finding problem?
What about plotting them? Do you see visually similar results?
-Matt
On Thu, Oct 15, 2009 at 9:43 AM, Britton Smith brittonsmith@gmail.com wrote:
Stephen,
It looks like if I take any random particle in a halo made with yt FOF, it's in a completely different halo in inline FOF, or not in any halo at all. Michele is checking this out right now, but it basically looks like the particle lists for the same halo found with the two different methods are completely different.
Britton
On Thu, Oct 15, 2009 at 10:22 AM, Stephen Skory stephenskory@yahoo.com wrote: >
Britton,
I'll check the yt FOF IDs against Enzo right now. The halo output of the code seems just fine, so if there's a bug, it's minor. I'll let you know what I find.
I remembered that the kD tree yt FOF uses is a bit inaccurate, and that could be a source of difference between the two sets of FOF haloes. But you said the differences were huge, and the kD inaccuracy should give only small differences, especially in FOF. Just a thought.
_______________________________________________________ sskory@physics.ucsd.edu o__ Stephen Skory http://physics.ucsd.edu/~sskory/ _.>/ _Graduate Student ________________________________(_)_(_)_______________
Yt-dev mailing list Yt-dev@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
Yt-dev mailing list Yt-dev@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
Hey Britton and Stephen,
Unless it was already obvious, I just wanted to chime in that the
inline FOF uses the same particles IDs in the main calculations of enzo.
John
On 15 Oct 2009, at 12:43, Britton Smith wrote:
Stephen,
It looks like if I take any random particle in a halo made with yt
FOF, it's in a completely different halo in inline FOF, or not in
any halo at all. Michele is checking this out right now, but it
basically looks like the particle lists for the same halo found with
the two different methods are completely different.
Britton
On Thu, Oct 15, 2009 at 10:22 AM, Stephen Skory <stephenskory@yahoo.com
wrote: Britton,
I'll check the yt FOF IDs against Enzo right now. The halo output
of the code seems just fine, so if there's a bug, it's minor. I'll
let you know what I find.
I remembered that the kD tree yt FOF uses is a bit inaccurate, and
that could be a source of difference between the two sets of FOF
haloes. But you said the differences were huge, and the kD
inaccuracy should give only small differences, especially in FOF.
Just a thought.
sskory@physics.ucsd.edu o__ Stephen Skory http://physics.ucsd.edu/~sskory/ _.>/ _Graduate Student ________________________________(_)_(_)_______________
Yt-dev mailing list Yt-dev@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
Yt-dev mailing list Yt-dev@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
Ok, call off the troops. It looks like we made a bit of a mistake over here. After checking two particles lists made on the same data with yt FOF and inline FOF, we've found that the particle lists are actually nearly identical. The only difference is that yt FOF is completely missing 4 small (~600 particles for a 128^3 sim) halos that are found by the inline FOF. We still don't know what is the cause of this. Making a projection of the position where the inline FOF finds one of these missing halos, it looks like there might be something there, but it's pretty marginal given the size of the halo.
Anyway, I would say it's safe to return to Defcon 5. I apologize for causing a fuss and wasting people's time. We should have looked a little more carefully at the issue. We're still working out some issues over here, but I don't think they involve the halo finders.
Sorry again for the inconvenience.
Britton
On Thu, Oct 15, 2009 at 10:46 AM, John Wise jwise@astro.princeton.eduwrote:
Hey Britton and Stephen,
Unless it was already obvious, I just wanted to chime in that the inline FOF uses the same particles IDs in the main calculations of enzo.
John
On 15 Oct 2009, at 12:43, Britton Smith wrote:
Stephen,
It looks like if I take any random particle in a halo made with yt FOF, it's in a completely different halo in inline FOF, or not in any halo at all. Michele is checking this out right now, but it basically looks like the particle lists for the same halo found with the two different methods are completely different.
Britton
On Thu, Oct 15, 2009 at 10:22 AM, Stephen Skory stephenskory@yahoo.comwrote:
Britton,
I'll check the yt FOF IDs against Enzo right now. The halo output of the code seems just fine, so if there's a bug, it's minor. I'll let you know what I find.
I remembered that the kD tree yt FOF uses is a bit inaccurate, and that could be a source of difference between the two sets of FOF haloes. But you said the differences were huge, and the kD inaccuracy should give only small differences, especially in FOF. Just a thought.
sskory@physics.ucsd.edu o__ Stephen Skory http://physics.ucsd.edu/~sskory/ http://physics.ucsd.edu/%7Esskory/_.>/ _Graduate Student ________________________________(_)_(_)_______________
Yt-dev mailing list Yt-dev@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
Yt-dev mailing list Yt-dev@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
Yt-dev mailing list Yt-dev@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
-
Britton Smith -
John Wise -
Matthew Turk -
Stephen Skory