Hi Nick,Sorry, I've not seen that error before.It looks like the very latest Rockstar supports ART natively, but it may have the same issue. That code is here:BrittonOn Tue, Mar 13, 2018 at 2:21 PM, Nick Gnedin <ngnedin@gmail.com> wrote:
Britton,
That helps, and Rockstar starts and loads data, but now it gives an internal error:
P001 yt : [INFO ] 2018-03-13 16:03:11,005 Created 2048 chunks for ARTIO
P001 yt : [WARNING ] 2018-03-13 16:05:09,506 Total Particle Count: 1.342e+08
[Error] Couldn't open # dsname index
:strict! (Err: Servname not supported for ai_socktype)
That error comes from the function default_addrinfo(...) in rockstar source code, file inet/socket.c.
I don't know how well you guys know rockstar - let me know if you think this is the dead end.
n
On 03/13/2018 03:18 PM, Britton Smith wrote:
Hi Nick,
You might need to add the "num_readers" and "num_writers" keywords to this line:
rhf = RockstarHaloFinder(d, num_readers=8, num_writers=8)
The readers are the i/o nodes and the writers are the actual halo finding instances. 8 for each is a good place to start. When you run, you'll need to have (num_readers + num_writers + 1) MPI processes, where the extra one is for the server.
Hopefully, that takes care of it.
Britton<mailto:ngnedin@gmail.com> <mailto:ngnedin@gmail.com
On Tue, Mar 13, 2018 at 12:43 PM, Nick Gnedin <ngnedin@gmail.com <mailto:ngnedin@gmail.com>> wrote:
Nathan,
Thank you. I followed the online instructions and now Rockstar
starts, but gives me an error, irrespective of whether I call it
directly or via a HaloCatalog object:
---------- code ---------
d = yt.load(root+"/rei20_a"+aexp+"/rei20_a"+aexp+".art")
rhf = RockstarHaloFinder(d)
rhf.run()
-------------------------
Traceback (most recent call last):
File "hfc.py", line 15, in <module>
rhf = RockstarHaloFinder(d)
File
"/scratch/midway2/gnedin/TMP/yt/yt/analysis_modules/halo_fin ding/rockstar/rockstar.py",
line 230, in __init__
self.pool, self.workgroup = self.runner.setup_pool()
File
"/scratch/midway2/gnedin/TMP/yt/yt/analysis_modules/halo_fin ding/rockstar/rockstar.py",
line 105, in setup_pool
(self.num_writers, "writers") ]
File
"/scratch/midway2/gnedin/TMP/yt/yt/utilities/parallel_tools/ parallel_analysis_interface.py ",
line 400, in from_sizes
pool.add_workgroup(size, name = name)
File
"/scratch/midway2/gnedin/TMP/yt/yt/utilities/parallel_tools/ parallel_analysis_interface.py ",
line 368, in add_workgroup
group = self.comm.comm.Get_group().Incl(ranks)
AttributeError: 'NoneType' object has no attribute 'Get_group'
On 03/13/2018 02:01 PM, Nathan Goldbaum wrote:
On Tue, Mar 13, 2018 at 1:52 PM, Nick Gnedin <ngnedin@gmail.com
<mailto:ngnedin@gmail.com>>> wrote:
<mailto:ngnedin@gmail.com <mailto:ngnedin@gmail.com>>>>Britton,
I am trying to run Rockstar, but it does not seem to be
packaged
with yt by default, import fails:
Traceback (most recent call last):
File "hfc.py", line 3, in <module>
from yt.analysis_modules.halo_finding.rockstar.api import
RockstarHaloFinder
File
"/home/gnedin/anaconda3/lib/python3.6/site-packages/yt/analy sis_modules/halo_finding/rocks tar/api.py",
line 16, in <module>
from .rockstar import RockstarHaloFinder
File
"/home/gnedin/anaconda3/lib/python3.6/site-packages/yt/analy sis_modules/halo_finding/rocks tar/rockstar.py",
line 28, in <module>
from . import rockstar_interface
ImportError: cannot import name 'rockstar_interface'
I think I am using the latest versions of python and yt:
yt module located at:
/home/gnedin/anaconda3/lib/python3.6/site-packages
The current version of yt is:
---
Version = 3.4.1
---
It is not packaged with yt out of the box. The easiest way to
get a copy of yt built with the rockstar bindings is to install
yt with the install script. You'll need to modify the script so
that INST_ROCKSTAR=1 once you've downloaded it.
Altenatively you can manually build yt and rockstar following
the instructions in the docs here:
http://yt-project.org/docs/dev/installing.html#installing-su pport-for-the-rockstar-halo-fi nder
<http://yt-project.org/docs/dev/installing.html#installing-s >upport-for-the-rockstar-halo-f inder
The rockstar halo finder is licensed under GPLv3 so
unfortunately we can't distribute it with the yt binaries on
pypi or conda-forge without changing their license as well.
n
On 03/13/2018 12:44 PM, Britton Smith wrote:
Hi Nick,
Thanks for your report. Your timing data confirms my
suspicion
about which part of the code isn't scaling. The
rejoining of
the halo list after the halo finder is run makes heavy
use of
MPI broadcast calls. Reworking this shouldn't be too
difficult,
just a question of someone finding the time. If anyone is
interested in trying to fix this, I can direct them to the
places that need the attention.
Nick, in the mean time, you might try the Rockstar halo
finder
(either the one built-in to yt or the standalone),
which scales
quite well. The output from both Rockstar versions is
loadable
with yt.
Britton
On Tue, Mar 13, 2018 at 8:46 AM, Nick Gnedin
<ngnedin@gmail.com <mailto:ngnedin@gmail.com>
<mailto:ngnedin@gmail.com <mailto:ngnedin@gmail.com>>
<mailto:ngnedin@gmail.com <mailto:ngnedin@gmail.com>
wrote:
This is just a notice to the developer.
I have run a HOP halo finder for a large ART
simulation (1024^3
particles) on BlueWaters with the following code:
import yt
from yt.analysis_modules.halo_analysis.api import
HaloCatalog
yt.enable_parallelism()
path = "/mnt/c/scratch/sciteam/ngnedin/PERM/B40/D"
aexps = [ "0.1280", "0.1203", "0.1115", "0.1002",
"0.0907" ]
for aexp in aexps:
d = yt.load(path+"/rei40_a"+aexp
+"/rei40_a"+aexp+".art")
hc =
HaloCatalog(data_ds=d,finder_method='hop',output_dir=path+"/ a="+aexp+"/hop",finder_kwargs= {"dm_only":False,"ptype":"N-BO DY"})
hc.create()
Because of memory constraints, I have to run it on
at least
4 MPI
ranks, and I noticed that yt implementation of HOP
does not
scale -
a 4-rank job takes 14.5 hours and an 8-rank one
takes 15.75
hours.
Surely, halo finding for billion particles should
scale
better than
that.
Here is some timing info (I can provide a full log
if you care)
4-rank run:
P000 yt : [INFO ] 2018-03-11 17:31:25,531
Parameters: ...
P000 yt : [INFO ] 2018-03-11 18:36:49,781
Initializing
HOP [1h]
P002 yt : [INFO ] 2018-03-11 22:14:15,486 Parsing
outputs [3.5h]
P000 yt : [INFO ] 2018-03-12 08:06:38,231
Saving halo
... [10h]
8-rank run:
P000 yt : [INFO ] 2018-03-10 21:03:27,226
Parameters: ...
P000 yt : [INFO ] 2018-03-10 21:43:52,543
Initializing
HOP [0.75h]
P005 yt : [INFO ] 2018-03-10 23:52:10,389 Parsing
outputs [2h]
P000 yt : [INFO ] 2018-03-11 12:43:46,645
Saving halo
... [12.5h *]
* - does not scale at all.
n
_______________________________________________
yt-users mailing list -- yt-users@python.org
<mailto:yt-users@python.org>
<mailto:yt-users@python.org <mailto:yt-users@python.org>>
<mailto:yt-users@python.org
<mailto:yt-users@python.org> <mailto:yt-users@python.org
<mailto:yt-users@python.org>>>
To unsubscribe send an email to
yt-users-leave@python.org <mailto:yt-users-leave@python.org >
<mailto:yt-users-leave@python.org
<mailto:yt-users-leave@python.org >>
<mailto:yt-users-leave@python.org
<mailto:yt-users-leave@python.org >
<mailto:yt-users-leave@python.org
<mailto:yt-users-leave@python.org >>>
_______________________________________________
yt-users mailing list -- yt-users@python.org
<mailto:yt-users@python.org>
<mailto:yt-users@python.org <mailto:yt-users@python.org>>
To unsubscribe send an email to yt-users-leave@python.org
<mailto:yt-users-leave@python.org >
<mailto:yt-users-leave@python.org
<mailto:yt-users-leave@python.org >>
_______________________________________________
yt-users mailing list -- yt-users@python.org
<mailto:yt-users@python.org>
To unsubscribe send an email to yt-users-leave@python.org
<mailto:yt-users-leave@python.org >