Cameron -- My experience has been that HOP/FOF are slow on datasets this large, and PHOP uses a large amount of memory. Anyways, regardless of that we should figure out what is going on with rockstar given its use in the community.
Hilary -- Just to be clear, you are sending the infiniband-disabling flags to this last run, correct?On Mon, Nov 4, 2013 at 4:36 PM, Cameron Hummels <chummels@gmail.com> wrote:
Are you tied to using rockstar? I know yt works well with HOP, PHOP, and FOF without many problems...--On Mon, Nov 4, 2013 at 3:50 PM, Hilary Egan <hilaryye@gmail.com> wrote:
Hi all,I'm still having trouble getting rockstar to run on my dataset. I've moved my data to a different super computer that uses the mpirun command rather than kraken's aprun so that I could make sure there aren't any infiniband issues, but Im still seeing similar issues. I've also determined I can run the halo finder just fine on a smaller test dataset, which is leading me to believe that its some sort of memory issue, but I can't quite figure out how I would go about fixing it. I've tried playing with the number of readers and the number of nodes I'm running on, to no avail. For reference, the dataset is a 1024^3 unigrid enzo run. If anyone has any suggestions, I'd love to hear them!Thanks!HilaryError message: http://paste.yt-project.org/show/4024/On Wed, Oct 30, 2013 at 8:50 AM, John Wise <jwise@physics.gatech.edu> wrote:
Hi Hilary,It's no problem at all to include all of your questions in a single email. It's probably better this way!
On 10/29/2013 12:30 PM, Hilary Egan wrote:
I'm quite confused on a number of points related to running the rockstar
halo finder, so I hope its alright that I put all these questions into
this one email!
I haven't seen that error before, but I still have to specific to *not* run on infiniband when running Rockstar on a single node. With OpenMPI, you would use "mpirun -n 32 --mca btl ^openib ...", but I haven't done this on kraken with their aprun but hopefully it's easily accompolished!
1. I can't seem to run the rockstar halo finder at all without getting
this error followed by a segmentation fault and crash.
[Warning] Network IO Failure (PID XXXXXX): Connection reset by peer
[Network] Packet receive retry count at: 1
It sort of seems like this issue
(http://lists.spacepope.org/htdig.cgi/yt-dev-spacepope.org/2012-November/002681.html)
but I couldn't really figure out what the resolution was from the
thread. Im attempting to run this on kraken and it doesn't matter if I
use a single compute node or multiple, I get the same error. (I hope
this isn't the infiniband issue the docs warned about, I couldn't figure
out if that is how kraken is connected and I got an error that the
suggested flag doesn't exist so I didn't press the issue.)
I'm not sure whether you can use yt's merger tree code with the Rockstar halos. I haven't tried.
2. Whenever I finally do get the halo finder to work, I need the results
to be in a form that the merger tree can use. It seems as though the
MergerTree needs the results in the same form as the other halo finders
give, so would getting the halo list and then dumping it as usual be the
appropriate strategy? Ie:
rh.run()
halo_list = rh.halo_list()
halo_list.dump('MergerHalos')
2.5. The docs sort of give mixed messages on whether or not I could just
be calling MergerTree with the argument halo_finder_function =
RockstarHaloFinder. At this point I've pretty thoroughly convinced
myself that I can't, but it would be nice if that was clarified. (Just a
thoroughly overwhelmed new user's perspective!)
However, I've used Consistent Trees
https://code.google.com/p/consistent-trees/
with Rockstar's halo lists, which is also written by Peter Behroozi. I've chosen this route because the algorithm seems to be more physically robust in constructing parent/child relationships and boundness. All of the instructions are in the README of the code, and it's pretty straightforward and fast (probably 5-10 minutes for a 512^3 simulation with 60 outputs) to run.
I also have a visualizing script for consistent tree's output.
https://bitbucket.org/jwise77/rockstar-dot
>From consistent tree's output, you can use the provided script, halo_trees_to_catalog.pl, (instructions also in the README) to convert the tree output into halo lists.This actually came up recently. It's best to supply a TimeSeriesData object. Here's the link to the email for more details.
3. I'm a little confused as to whether or not I have to use a
TimeSeriesData object rather than the usual single time output when
instantiating the halo finder. Under "Rockstar Halo Finding" it uses
TimeSeriesData, unlike the rest of the examples, but under the
subheading "Output Analysis" it just uses pf. The "Output Analysis"
example also doesn't call the run() method, which leads me to believe
something else entirely is going on, but its not quite clear.
http://lists.spacepope.org/pipermail/yt-users-spacepope.org/2013-August/003845.html
Cheers,
John
--
John Wise
Assistant Professor of Physics
Center for Relativistic Astrophysics, Georgia Tech
http://cosmo.gatech.edu
_______________________________________________
yt-users mailing list
yt-users@lists.spacepope.org
http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
_______________________________________________
yt-users mailing list
yt-users@lists.spacepope.org
http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
Cameron HummelsPostdoctoral ResearcherSteward ObservatoryUniversity of Arizona
_______________________________________________
yt-users mailing list
yt-users@lists.spacepope.org
http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
_______________________________________________
yt-users mailing list
yt-users@lists.spacepope.org
http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org