
Dear yt users, Is there a way to run the halo finder on part of the simulation volume? For example, I have a box 256 Mpc on a side but I only want to find halos inside a smaller box, say 20 Mpc on a side. Thanks for your help, Irina

Irina,
Is there a way to run the halo finder on part of the simulation volume? For example, I have a box 256 Mpc on a side but I only want to find halos inside a smaller box, say 20 Mpc on a side.
Unfortunately, this is not currently possible in yt. However, it should be possible if the right modifications are done. Something along these lines is how HOP/FOF are parallelized, so it should be translateable to your problem. I am not making promises that I will do this, but that it is possible. I'll think about it. However, I wonder why you only want halos in a small region? Do you have a nested simulation? Does it seem like the halo finding is taking too long? Are you looking for substructure? Depending on the answers to these questions, the modifications discussed above may not be required. _______________________________________________________ sskory@physics.ucsd.edu o__ Stephen Skory http://physics.ucsd.edu/~sskory/ _.>/ _Graduate Student ________________________________(_)_\(_)_______________

Stephen, This *was* possible. Until Irina emailed, I was under the impression it still was; I don't know why the functionality was removed. I believe that changing the data_source check and construction should make this possible, similarly to how the projection mechanism checks if a source has been provided. The only variables that are required to be initialized in __init__ related to the data_source are padding and _data_source; if _data_source is supplied (i.e., add on data_source to the contructor) and "is not None" then it should simply set padding to 0.0 and set self._data_source equal to the supplied source. -Matt On Mon, Sep 20, 2010 at 8:40 AM, Stephen Skory <stephenskory@yahoo.com> wrote:
Irina,
Is there a way to run the halo finder on part of the simulation volume? For example, I have a box 256 Mpc on a side but I only want to find halos inside a smaller box, say 20 Mpc on a side.
Unfortunately, this is not currently possible in yt. However, it should be possible if the right modifications are done. Something along these lines is how HOP/FOF are parallelized, so it should be translateable to your problem. I am not making promises that I will do this, but that it is possible. I'll think about it.
However, I wonder why you only want halos in a small region? Do you have a nested simulation? Does it seem like the halo finding is taking too long? Are you looking for substructure? Depending on the answers to these questions, the modifications discussed above may not be required.
_______________________________________________________ sskory@physics.ucsd.edu o__ Stephen Skory http://physics.ucsd.edu/~sskory/ _.>/ _Graduate Student ________________________________(_)_\(_)_______________
_______________________________________________ yt-users mailing list yt-users@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org

Matt,
This *was* possible. Until Irina emailed, I was under the impression it still was; I don't know why the functionality was removed.
Color me wrong! If it worked before, unless I broke things, it should work now. So, Irina, give it a shot and let us know if you have problems. _______________________________________________________ sskory@physics.ucsd.edu o__ Stephen Skory http://physics.ucsd.edu/~sskory/ _.>/ _Graduate Student ________________________________(_)_\(_)_______________

Matt and Stephen, Thanks for your help and ignore my previous email, send it before reading your replies. I'll try to do what Matt suggested. Irina Quoting "Stephen Skory" <stephenskory@yahoo.com>:
Matt,
This *was* possible. Until Irina emailed, I was under the impression it still was; I don't know why the functionality was removed.
Color me wrong! If it worked before, unless I broke things, it should work now. So, Irina, give it a shot and let us know if you have problems.
_______________________________________________________ sskory@physics.ucsd.edu o__ Stephen Skory http://physics.ucsd.edu/~sskory/ _.>/ _Graduate Student ________________________________(_)_\(_)_______________
_______________________________________________ yt-users mailing list yt-users@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org

Matt and Stephen, I followed Matt's idea and it worked great in my simple case. Thanks a lot, it saves quite a lot of time! Irina Quoting "Matthew Turk" <matthewturk@gmail.com>:
Stephen,
This *was* possible. Until Irina emailed, I was under the impression it still was; I don't know why the functionality was removed.
I believe that changing the data_source check and construction should make this possible, similarly to how the projection mechanism checks if a source has been provided. The only variables that are required to be initialized in __init__ related to the data_source are padding and _data_source; if _data_source is supplied (i.e., add on data_source to the contructor) and "is not None" then it should simply set padding to 0.0 and set self._data_source equal to the supplied source.
-Matt
On Mon, Sep 20, 2010 at 8:40 AM, Stephen Skory <stephenskory@yahoo.com> wrote:
Irina,
Is there a way to run the halo finder on part of the simulation volume? For example, I have a box 256 Mpc on a side but I only want to find halos inside a smaller box, say 20 Mpc on a side.
Unfortunately, this is not currently possible in yt. However, it should be possible if the right modifications are done. Something along these lines is how HOP/FOF are parallelized, so it should be translateable to your problem. I am not making promises that I will do this, but that it is possible. I'll think about it.
However, I wonder why you only want halos in a small region? Do you have a nested simulation? Does it seem like the halo finding is taking too long? Are you looking for substructure? Depending on the answers to these questions, the modifications discussed above may not be required.
_______________________________________________________ sskory@physics.ucsd.edu o__ Stephen Skory http://physics.ucsd.edu/~sskory/ _.>/ _Graduate Student ________________________________(_)_\(_)_______________
_______________________________________________ yt-users mailing list yt-users@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
_______________________________________________ yt-users mailing list yt-users@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org

Hi all, This is a follow up on this discussion from a month ago. I want to do the same thing Irina wanted to do--run hop on a smaller region in my simulation box. I read this discussion, but I'm a python neophyte and I'm not sure I follow what to do. Could someone post an example script? I'd really appreciate it. Thanks Christine On Tue, 2010-09-21 at 14:57 +0200, Irina Dvorkin wrote:
Matt and Stephen,
I followed Matt's idea and it worked great in my simple case. Thanks a lot, it saves quite a lot of time!
Irina
Quoting "Matthew Turk" <matthewturk@gmail.com>:
Stephen,
This *was* possible. Until Irina emailed, I was under the impression it still was; I don't know why the functionality was removed.
I believe that changing the data_source check and construction should make this possible, similarly to how the projection mechanism checks if a source has been provided. The only variables that are required to be initialized in __init__ related to the data_source are padding and _data_source; if _data_source is supplied (i.e., add on data_source to the contructor) and "is not None" then it should simply set padding to 0.0 and set self._data_source equal to the supplied source.
-Matt
On Mon, Sep 20, 2010 at 8:40 AM, Stephen Skory <stephenskory@yahoo.com> wrote:
Irina,
Is there a way to run the halo finder on part of the simulation volume? For example, I have a box 256 Mpc on a side but I only want to find halos inside a smaller box, say 20 Mpc on a side.
Unfortunately, this is not currently possible in yt. However, it should be possible if the right modifications are done. Something along these lines is how HOP/FOF are parallelized, so it should be translateable to your problem. I am not making promises that I will do this, but that it is possible. I'll think about it.
However, I wonder why you only want halos in a small region? Do you have a nested simulation? Does it seem like the halo finding is taking too long? Are you looking for substructure? Depending on the answers to these questions, the modifications discussed above may not be required.
_______________________________________________________ sskory@physics.ucsd.edu o__ Stephen Skory http://physics.ucsd.edu/~sskory/ _.>/ _Graduate Student ________________________________(_)_\(_)_______________
_______________________________________________ yt-users mailing list yt-users@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
_______________________________________________ yt-users mailing list yt-users@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
_______________________________________________ yt-users mailing list yt-users@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org

Hi Christine, Here's what I did (probably not the smartest way, but I'm also a python beginner). I modified the file HaloFinding.py (in the older yt version). There, you have class HOPHaloFinder(GenericHaloFinder, HOPHaloList) and its function _init_ . There it reads the data source using the function partition_hierarchy_3d. I disabled that (of course a more elegant way exists) as following: # Irina - trying to run HF on a subvolume if box is True: self.padding=0.0 LE, RE = self.pf["DomainLeftEdge"].copy(), self.pf["DomainRightEdge"].copy() LEbox, REbox = self.pf["RefineRegionLeftEdge"].copy(), self.pf["RefineRegionRightEdge"].copy() self._data_source = self.hierarchy.region(self.center, LEbox, REbox) self.bounds = (LEbox, REbox) else: self.padding = padding #* pf["unitary"] # This should be clevererer padded, LE, RE, self._data_source = self._partition_hierarchy_3d(padding=self.padding) self.bounds = (LE, RE) -end-of-code- box is just a flag I use, when box=True I run HOP on a subvolume, so the -else- part is what the code looked like originally. The subvolume itself is given by pf["RefineRegionLeftEdge"] and same for RightEdge, but that's good just for my application since I have these variables anyway, I suppose these can be given as a separate input. I hope this helps, Irina Quoting "Christine Simpson" <csimpson@astro.columbia.edu>:
Hi all,
This is a follow up on this discussion from a month ago. I want to do the same thing Irina wanted to do--run hop on a smaller region in my simulation box. I read this discussion, but I'm a python neophyte and I'm not sure I follow what to do. Could someone post an example script? I'd really appreciate it.
Thanks Christine
On Tue, 2010-09-21 at 14:57 +0200, Irina Dvorkin wrote:
Matt and Stephen,
I followed Matt's idea and it worked great in my simple case. Thanks a lot, it saves quite a lot of time!
Irina
Quoting "Matthew Turk" <matthewturk@gmail.com>:
Stephen,
This *was* possible. Until Irina emailed, I was under the impression it still was; I don't know why the functionality was removed.
I believe that changing the data_source check and construction should make this possible, similarly to how the projection mechanism checks if a source has been provided. The only variables that are required to be initialized in __init__ related to the data_source are padding and _data_source; if _data_source is supplied (i.e., add on data_source to the contructor) and "is not None" then it should simply set padding to 0.0 and set self._data_source equal to the supplied source.
-Matt
Irina,
Is there a way to run the halo finder on part of the simulation volume? For example, I have a box 256 Mpc on a side but I only want to find halos inside a smaller box, say 20 Mpc on a side.
Unfortunately, this is not currently possible in yt. However, it should be possible if the right modifications are done. Something along these lines is how HOP/FOF are parallelized, so it should be translateable to your problem. I am not making promises that I will do this, but that it is
On Mon, Sep 20, 2010 at 8:40 AM, Stephen Skory <stephenskory@yahoo.com> wrote: possible. I'll think
about it.
However, I wonder why you only want halos in a small region? Do you have a nested simulation? Does it seem like the halo finding is taking too long? Are you looking for substructure? Depending on the answers to these questions, the modifications discussed above may not be required.
_______________________________________________________ sskory@physics.ucsd.edu o__ Stephen Skory http://physics.ucsd.edu/~sskory/ _.>/ _Graduate Student ________________________________(_)_\(_)_______________
_______________________________________________ yt-users mailing list yt-users@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
_______________________________________________ yt-users mailing list yt-users@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
_______________________________________________ yt-users mailing list yt-users@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
_______________________________________________ yt-users mailing list yt-users@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org

Hi Stephen, Basically I want to save time. I'm working with the SCA, where each simulation is 256 Mpc/h on a side, but the AMR was done only in the central region, where the cluster is supposed to form (a few tens of Mpc). So I'm interested only in the region with the good resolution. Also, the halos I want to find are not on the boundary of this region, since that would mean they have mediocre resolution. Maybe you could give me some hints and I will try and think about it myself, at least for my particular problem. Thanks a lot, Irina Quoting "Stephen Skory" <stephenskory@yahoo.com>:
Irina,
Is there a way to run the halo finder on part of the simulation volume? For example, I have a box 256 Mpc on a side but I only want to find halos inside a smaller box, say 20 Mpc on a side.
Unfortunately, this is not currently possible in yt. However, it should be possible if the right modifications are done. Something along these lines is how HOP/FOF are parallelized, so it should be translateable to your problem. I am not making promises that I will do this, but that it is possible. I'll think about it.
However, I wonder why you only want halos in a small region? Do you have a nested simulation? Does it seem like the halo finding is taking too long? Are you looking for substructure? Depending on the answers to these questions, the modifications discussed above may not be required.
_______________________________________________________ sskory@physics.ucsd.edu o__ Stephen Skory http://physics.ucsd.edu/~sskory/ _.>/ _Graduate Student ________________________________(_)_\(_)_______________
_______________________________________________ yt-users mailing list yt-users@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
participants (4)
-
Christine Simpson
-
Irina Dvorkin
-
Matthew Turk
-
Stephen Skory