On Tue, Mar 13, 2018 at 2:21 PM, Nick Gnedin <ngnedin@gmail.com> wrote:

Britton,

That helps, and Rockstar starts and loads data, but now it gives an internal error:

P001 yt : [INFO ] 2018-03-13 16:03:11,005 Created 2048 chunks for ARTIO
P001 yt : [WARNING ] 2018-03-13 16:05:09,506 Total Particle Count: 1.342e+08
[Error] Couldn't open # dsname index
:strict! (Err: Servname not supported for ai_socktype)

That error comes from the function default_addrinfo(...) in rockstar source code, file inet/socket.c.

I don't know how well you guys know rockstar - let me know if you think this is the dead end.

n

On 03/13/2018 03:18 PM, Britton Smith wrote:

Hi Nick,

You might need to add the "num_readers" and "num_writers" keywords to this line:
rhf = RockstarHaloFinder(d, num_readers=8, num_writers=8)
The readers are the i/o nodes and the writers are the actual halo finding instances. 8 for each is a good place to start. When you run, you'll need to have (num_readers + num_writers + 1) MPI processes, where the extra one is for the server.

Hopefully, that takes care of it.

Britton

On Tue, Mar 13, 2018 at 12:43 PM, Nick Gnedin <ngnedin@gmail.com <mailto:ngnedin@gmail.com>> wrote:

Nathan,

Thank you. I followed the online instructions and now Rockstar
starts, but gives me an error, irrespective of whether I call it
directly or via a HaloCatalog object:

---------- code ---------
   d = yt.load(root+"/rei20_a"+aexp+"/rei20_a"+aexp+".art")
   rhf = RockstarHaloFinder(d)
   rhf.run()
-------------------------

Traceback (most recent call last):
   File "hfc.py", line 15, in <module>
   rhf = RockstarHaloFinder(d)
   File
"/scratch/midway2/gnedin/TMP/yt/yt/analysis_modules/halo_finding/rockstar/rockstar.py",
line 230, in __init__
   self.pool, self.workgroup = self.runner.setup_pool()
   File
"/scratch/midway2/gnedin/TMP/yt/yt/analysis_modules/halo_finding/rockstar/rockstar.py",
line 105, in setup_pool
   (self.num_writers, "writers") ]
   File
"/scratch/midway2/gnedin/TMP/yt/yt/utilities/parallel_tools/parallel_analysis_interface.py",
line 400, in from_sizes
   pool.add_workgroup(size, name = name)
   File
"/scratch/midway2/gnedin/TMP/yt/yt/utilities/parallel_tools/parallel_analysis_interface.py",
line 368, in add_workgroup
   group = self.comm.comm.Get_group().Incl(ranks)
AttributeError: 'NoneType' object has no attribute 'Get_group'

On 03/13/2018 02:01 PM, Nathan Goldbaum wrote:

On Tue, Mar 13, 2018 at 1:52 PM, Nick Gnedin <ngnedin@gmail.com
<mailto:ngnedin@gmail.com> <mailto:ngnedin@gmail.com
<mailto:ngnedin@gmail.com>>> wrote:

   Britton,

   I am trying to run Rockstar, but it does not seem to be
packaged
   with yt by default, import fails:

   Traceback (most recent call last):
      File "hfc.py", line 3, in <module>
      from yt.analysis_modules.halo_finding.rockstar.api import
   RockstarHaloFinder
      File
"/home/gnedin/anaconda3/lib/python3.6/site-packages/yt/analysis_modules/halo_finding/rockstar/api.py",
   line 16, in <module>
      from .rockstar import RockstarHaloFinder
      File
"/home/gnedin/anaconda3/lib/python3.6/site-packages/yt/analysis_modules/halo_finding/rockstar/rockstar.py",
   line 28, in <module>
      from . import rockstar_interface
   ImportError: cannot import name 'rockstar_interface'

   I think I am using the latest versions of python and yt:

   yt module located at:
      /home/gnedin/anaconda3/lib/python3.6/site-packages

   The current version of yt is:

   ---
   Version = 3.4.1
   ---

It is not packaged with yt out of the box. The easiest way to
get a copy of yt built with the rockstar bindings is to install
yt with the install script. You'll need to modify the script so
that INST_ROCKSTAR=1 once you've downloaded it.

Altenatively you can manually build yt and rockstar following
the instructions in the docs here:

http://yt-project.org/docs/dev/installing.html#installing-support-for-the-rockstar-halo-finder
<http://yt-project.org/docs/dev/installing.html#installing-support-for-the-rockstar-halo-finder>

The rockstar halo finder is licensed under GPLv3 so
unfortunately we can't distribute it with the yt binaries on
pypi or conda-forge without changing their license as well.

   n

   On 03/13/2018 12:44 PM, Britton Smith wrote:

   Hi Nick,

   Thanks for your report. Your timing data confirms my
suspicion
   about which part of the code isn't scaling. The
rejoining of
   the halo list after the halo finder is run makes heavy
use of
   MPI broadcast calls. Reworking this shouldn't be too
difficult,
   just a question of someone finding the time. If anyone is
   interested in trying to fix this, I can direct them to the
   places that need the attention.

   Nick, in the mean time, you might try the Rockstar halo
finder
   (either the one built-in to yt or the standalone),
which scales
   quite well. The output from both Rockstar versions is
loadable
   with yt.

   Britton

   On Tue, Mar 13, 2018 at 8:46 AM, Nick Gnedin
<ngnedin@gmail.com <mailto:ngnedin@gmail.com>
   <mailto:ngnedin@gmail.com <mailto:ngnedin@gmail.com>>
<mailto:ngnedin@gmail.com <mailto:ngnedin@gmail.com>

   <mailto:ngnedin@gmail.com <mailto:ngnedin@gmail.com>>>>

wrote:

      This is just a notice to the developer.

      I have run a HOP halo finder for a large ART
simulation (1024^3
      particles) on BlueWaters with the following code:

      import yt
      from yt.analysis_modules.halo_analysis.api import
HaloCatalog
      yt.enable_parallelism()
      path = "/mnt/c/scratch/sciteam/ngnedin/PERM/B40/D"
      aexps = [ "0.1280", "0.1203", "0.1115", "0.1002",
"0.0907" ]
      for aexp in aexps:
         d = yt.load(path+"/rei40_a"+aexp
+"/rei40_a"+aexp+".art")
         hc =
HaloCatalog(data_ds=d,finder_method='hop',output_dir=path+"/a="+aexp+"/hop",finder_kwargs={"dm_only":False,"ptype":"N-BODY"})
         hc.create()

      Because of memory constraints, I have to run it on
at least
   4 MPI
      ranks, and I noticed that yt implementation of HOP
does not
   scale -
      a 4-rank job takes 14.5 hours and an 8-rank one
takes 15.75
   hours.
      Surely, halo finding for billion particles should
scale
   better than
      that.

      Here is some timing info (I can provide a full log
if you care)

      4-rank run:
      P000 yt : [INFO ] 2018-03-11 17:31:25,531
Parameters: ...
      P000 yt : [INFO ] 2018-03-11 18:36:49,781
Initializing
   HOP [1h]
      P002 yt : [INFO ] 2018-03-11 22:14:15,486 Parsing
   outputs [3.5h]
      P000 yt : [INFO ] 2018-03-12 08:06:38,231
Saving halo
   ... [10h]

      8-rank run:
      P000 yt : [INFO ] 2018-03-10 21:03:27,226
Parameters: ...
      P000 yt : [INFO ] 2018-03-10 21:43:52,543
Initializing
   HOP [0.75h]
      P005 yt : [INFO ] 2018-03-10 23:52:10,389 Parsing
   outputs [2h]
      P000 yt : [INFO ] 2018-03-11 12:43:46,645
Saving halo
   ... [12.5h *]

      * - does not scale at all.

      n

      _______________________________________________
      yt-users mailing list -- yt-users@python.org
<mailto:yt-users@python.org>
   <mailto:yt-users@python.org <mailto:yt-users@python.org>>
      <mailto:yt-users@python.org
<mailto:yt-users@python.org> <mailto:yt-users@python.org
<mailto:yt-users@python.org>>>
      To unsubscribe send an email to
yt-users-leave@python.org <mailto:yt-users-leave@python.org>
   <mailto:yt-users-leave@python.org
<mailto:yt-users-leave@python.org>>
      <mailto:yt-users-leave@python.org
<mailto:yt-users-leave@python.org>
   <mailto:yt-users-leave@python.org
<mailto:yt-users-leave@python.org>>>

   _______________________________________________
   yt-users mailing list -- yt-users@python.org
<mailto:yt-users@python.org>
   <mailto:yt-users@python.org <mailto:yt-users@python.org>>
   To unsubscribe send an email to yt-users-leave@python.org
<mailto:yt-users-leave@python.org>
   <mailto:yt-users-leave@python.org
<mailto:yt-users-leave@python.org>>

_______________________________________________
yt-users mailing list -- yt-users@python.org
<mailto:yt-users@python.org>
To unsubscribe send an email to yt-users-leave@python.org
<mailto:yt-users-leave@python.org>