yt-users

Download

yt-users@python.org

January 2026

1 participants
1 discussions

Running Rockstar Algorithm using yt astro analysis
by Nixin Saji Abraham Jan. 19, 2026

Jan. 19, 2026

I am trying to run rockstar algorithm using yt astro analysis. What I have with me is gadget 1 data which was converted from big file data format. I have generated 32, 34, 39 chunks of the same data in gadget 1 format. Majority of the time, I get the following error at the start itself. File "/home/user/p1cj_phy/fastpm/yt_astro_analysis/yt_astro_analysis/halo_analysis/halo_catalog/halo_catalog.py", line 346, in create self._run(save_halos, save_output, njobs=njobs, dynamic=dynamic) File "/home/user/p1cj_phy/fastpm/yt_astro_v/lib/python3.12/site-packages/yt/utilities/parallel_tools/parallel_analysis_interface.py", line 322, in barrierize retval = func(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^ File "/home/user/p1cj_phy/fastpm/yt_astro_analysis/yt_astro_analysis/halo_analysis/halo_catalog/halo_catalog.py", line 239, in _run self.finder_method(self) File "/home/user/p1cj_phy/fastpm/yt_astro_analysis/yt_astro_analysis/halo_analysis/halo_catalog/halo_finding_methods.py", line 36, in __call__ return self.function(hc, *self.args, **self.kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/user/p1cj_phy/fastpm/yt_astro_analysis/yt_astro_analysis/halo_analysis/halo_catalog/halo_finding_methods.py", line 88, in _rockstar_method rh = RockstarHaloFinder(ds, **finder_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/user/p1cj_phy/fastpm/yt_astro_analysis/yt_astro_analysis/halo_analysis/halo_finding/rockstar/rockstar.py", line 290, in __init__ p = self._setup_parameters(ts) ^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/user/p1cj_phy/fastpm/yt_astro_analysis/yt_astro_analysis/halo_analysis/halo_finding/rockstar/rockstar.py", line 299, in _setup_parameters tds.index # noqa: B018 ^^^^^^^^^ File "/home/user/p1cj_phy/fastpm/yt_astro_v/lib/python3.12/site-packages/yt/data_objects/static_output.py", line 605, in index self._instantiated_index = self._index_class( ^^^^^^^^^^^^^^^^^^ File "/home/user/p1cj_phy/fastpm/yt_astro_v/lib/python3.12/site-packages/yt/frontends/gadget/data_structures.py", line 195, in __init__ super().__init__(ds, dataset_type) File "/home/user/p1cj_phy/fastpm/yt_astro_v/lib/python3.12/site-packages/yt/geometry/particle_geometry_handler.py", line 73, in __init__ self._initialize_index() File "/home/user/p1cj_phy/fastpm/yt_astro_v/lib/python3.12/site-packages/yt/frontends/gadget/data_structures.py", line 204, in _initialize_index super()._initialize_index() File "/home/user/p1cj_phy/fastpm/yt_astro_v/lib/python3.12/site-packages/yt/frontends/sph/data_structures.py", line 89, in _initialize_index super()._initialize_index() File "/home/user/p1cj_phy/fastpm/yt_astro_v/lib/python3.12/site-packages/yt/geometry/particle_geometry_handler.py", line 237, in _initialize_index max_hsml = self._initialize_coarse_index() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/user/p1cj_phy/fastpm/yt_astro_v/lib/python3.12/site-packages/yt/geometry/particle_geometry_handler.py", line 280, in _initialize_coarse_index self.regions.masks = self.comm.mpi_allreduce(self.regions.masks, op="sum") ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/user/p1cj_phy/fastpm/yt_astro_v/lib/python3.12/site-packages/yt/utilities/parallel_tools/parallel_analysis_interface.py", line 297, in passage return func(self, *args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/user/p1cj_phy/fastpm/yt_astro_v/lib/python3.12/site-packages/yt/utilities/parallel_tools/parallel_analysis_interface.py", line 913, in mpi_allreduce self.comm.Allreduce( File "src/mpi4py/MPI.src/Comm.pyx", line 1115, in mpi4py.MPI.Comm.Allreduce P004 yt : [ERROR ] 2025-12-19 23:49:08,036 Exception: MPI_ERR_ARG: invalid argument of some other kind P004 yt : [ERROR ] 2025-12-19 23:49:08,036 Error occurred on rank 3. -------------------------------------------------------------------------- MPI_ABORT was invoked on rank 4 in communicator MPI_COMM_WORLD with errorcode 1. NOTE: invoking MPI_ABORT causes Open MPI to kill all MPI processes. You may or may not see output from other processes, depending on exactly when Open MPI kills them. -------------------------------------------------------------------------- File "/home/user/p1cj_phy/fastpm/my.example/rockstar.py", line 14, in <module> hc.create() File "/home/user/p1cj_phy/fastpm/yt_astro_analysis/yt_astro_analysis/halo_analysis/halo_catalog/halo_catalog.py", line 346, in create self._run(save_halos, save_output, njobs=njobs, dynamic=dynamic) File "/home/user/p1cj_phy/fastpm/yt_astro_v/lib/python3.12/site-packages/yt/utilities/parallel_tools/parallel_analysis_interface.py", line 322, in barrierize retval = func(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^ File "/home/user/p1cj_phy/fastpm/yt_astro_analysis/yt_astro_analysis/halo_analysis/halo_catalog/halo_catalog.py", line 239, in _run self.finder_method(self) File "/home/user/p1cj_phy/fastpm/yt_astro_analysis/yt_astro_analysis/halo_analysis/halo_catalog/halo_finding_methods.py", line 36, in __call__ return self.function(hc, *self.args, **self.kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/user/p1cj_phy/fastpm/yt_astro_analysis/yt_astro_analysis/halo_analysis/halo_catalog/halo_finding_methods.py", line 88, in _rockstar_method rh = RockstarHaloFinder(ds, **finder_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/user/p1cj_phy/fastpm/yt_astro_analysis/yt_astro_analysis/halo_analysis/halo_finding/rockstar/rockstar.py", line 290, in __init__ p = self._setup_parameters(ts) ^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/user/p1cj_phy/fastpm/yt_astro_analysis/yt_astro_analysis/halo_analysis/halo_finding/rockstar/rockstar.py", line 299, in _setup_parameters tds.index # noqa: B018 ^^^^^^^^^ File "/home/user/p1cj_phy/fastpm/yt_astro_v/lib/python3.12/site-packages/yt/data_objects/static_output.py", line 605, in index self._instantiated_index = self._index_class( ^^^^^^^^^^^^^^^^^^ File "/home/user/p1cj_phy/fastpm/yt_astro_v/lib/python3.12/site-packages/yt/frontends/gadget/data_structures.py", line 195, in __init__ super().__init__(ds, dataset_type) File "/home/user/p1cj_phy/fastpm/yt_astro_v/lib/python3.12/site-packages/yt/geometry/particle_geometry_handler.py", line 73, in __init__ self._initialize_index() File "/home/user/p1cj_phy/fastpm/yt_astro_v/lib/python3.12/site-packages/yt/frontends/gadget/data_structures.py", line 204, in _initialize_index super()._initialize_index() File "/home/user/p1cj_phy/fastpm/yt_astro_v/lib/python3.12/site-packages/yt/frontends/sph/data_structures.py", line 89, in _initialize_index super()._initialize_index() File "/home/user/p1cj_phy/fastpm/yt_astro_v/lib/python3.12/site-packages/yt/geometry/particle_geometry_handler.py", line 237, in _initialize_index max_hsml = self._initialize_coarse_index() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/user/p1cj_phy/fastpm/yt_astro_v/lib/python3.12/site-packages/yt/geometry/particle_geometry_handler.py", line 280, in _initialize_coarse_index self.regions.masks = self.comm.mpi_allreduce(self.regions.masks, op="sum") ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/user/p1cj_phy/fastpm/yt_astro_v/lib/python3.12/site-packages/yt/utilities/parallel_tools/parallel_analysis_interface.py", line 297, in passage return func(self, *args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/user/p1cj_phy/fastpm/yt_astro_v/lib/python3.12/site-packages/yt/utilities/parallel_tools/parallel_analysis_interface.py", line 913, in mpi_allreduce self.comm.Allreduce( File "src/mpi4py/MPI.src/Comm.pyx", line 1115, in mpi4py.MPI.Comm.Allreduce P001 yt : [ERROR ] 2025-12-19 23:49:08,554 Exception: MPI_ERR_ARG: invalid argument of some other kind P001 yt : [ERROR ] 2025-12-19 23:49:08,554 Error occurred on rank 0. File "/home/user/p1cj_phy/fastpm/my.example/rockstar.py", line 14, in <module> hc.create() File "/home/user/p1cj_phy/fastpm/yt_astro_analysis/yt_astro_analysis/halo_analysis/halo_catalog/halo_catalog.py", line 346, in create self._run(save_halos, save_output, njobs=njobs, dynamic=dynamic) File "/home/user/p1cj_phy/fastpm/yt_astro_v/lib/python3.12/site-packages/yt/utilities/parallel_tools/parallel_analysis_interface.py", line 322, in barrierize retval = func(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^ File "/home/user/p1cj_phy/fastpm/yt_astro_analysis/yt_astro_analysis/halo_analysis/halo_catalog/halo_catalog.py", line 239, in _run self.finder_method(self) File "/home/user/p1cj_phy/fastpm/yt_astro_analysis/yt_astro_analysis/halo_analysis/halo_catalog/halo_finding_methods.py", line 36, in __call__ return self.function(hc, *self.args, **self.kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/user/p1cj_phy/fastpm/yt_astro_analysis/yt_astro_analysis/halo_analysis/halo_catalog/halo_finding_methods.py", line 88, in _rockstar_method rh = RockstarHaloFinder(ds, **finder_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/user/p1cj_phy/fastpm/yt_astro_analysis/yt_astro_analysis/halo_analysis/halo_finding/rockstar/rockstar.py", line 290, in __init__ p = self._setup_parameters(ts) ^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/user/p1cj_phy/fastpm/yt_astro_analysis/yt_astro_analysis/halo_analysis/halo_finding/rockstar/rockstar.py", line 299, in _setup_parameters tds.index # noqa: B018 ^^^^^^^^^ File "/home/user/p1cj_phy/fastpm/yt_astro_v/lib/python3.12/site-packages/yt/data_objects/static_output.py", line 605, in index self._instantiated_index = self._index_class( ^^^^^^^^^^^^^^^^^^ File "/home/user/p1cj_phy/fastpm/yt_astro_v/lib/python3.12/site-packages/yt/frontends/gadget/data_structures.py", line 195, in __init__ super().__init__(ds, dataset_type) File "/home/user/p1cj_phy/fastpm/yt_astro_v/lib/python3.12/site-packages/yt/geometry/particle_geometry_handler.py", line 73, in __init__ self._initialize_index() File "/home/user/p1cj_phy/fastpm/yt_astro_v/lib/python3.12/site-packages/yt/frontends/gadget/data_structures.py", line 204, in _initialize_index super()._initialize_index() File "/home/user/p1cj_phy/fastpm/yt_astro_v/lib/python3.12/site-packages/yt/frontends/sph/data_structures.py", line 89, in _initialize_index super()._initialize_index() File "/home/user/p1cj_phy/fastpm/yt_astro_v/lib/python3.12/site-packages/yt/geometry/particle_geometry_handler.py", line 237, in _initialize_index max_hsml = self._initialize_coarse_index() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/user/p1cj_phy/fastpm/yt_astro_v/lib/python3.12/site-packages/yt/geometry/particle_geometry_handler.py", line 280, in _initialize_coarse_index self.regions.masks = self.comm.mpi_allreduce(self.regions.masks, op="sum") ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/user/p1cj_phy/fastpm/yt_astro_v/lib/python3.12/site-packages/yt/utilities/parallel_tools/parallel_analysis_interface.py", line 297, in passage return func(self, *args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/user/p1cj_phy/fastpm/yt_astro_v/lib/python3.12/site-packages/yt/utilities/parallel_tools/parallel_analysis_interface.py", line 913, in mpi_allreduce self.comm.Allreduce( File "src/mpi4py/MPI.src/Comm.pyx", line 1115, in mpi4py.MPI.Comm.Allreduce P002 yt : [ERROR ] 2025-12-19 23:49:08,936 Exception: MPI_ERR_ARG: invalid argument of some other kind P002 yt : [ERROR ] 2025-12-19 23:49:08,936 Error occurred on rank 1. File "/home/user/p1cj_phy/fastpm/my.example/rockstar.py", line 14, in <module> hc.create() File "/home/user/p1cj_phy/fastpm/yt_astro_analysis/yt_astro_analysis/halo_analysis/halo_catalog/halo_catalog.py", line 346, in create self._run(save_halos, save_output, njobs=njobs, dynamic=dynamic) File "/home/user/p1cj_phy/fastpm/yt_astro_v/lib/python3.12/site-packages/yt/utilities/parallel_tools/parallel_analysis_interface.py", line 322, in barrierize retval = func(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^ File "/home/user/p1cj_phy/fastpm/yt_astro_analysis/yt_astro_analysis/halo_analysis/halo_catalog/halo_catalog.py", line 239, in _run self.finder_method(self) File "/home/user/p1cj_phy/fastpm/yt_astro_analysis/yt_astro_analysis/halo_analysis/halo_catalog/halo_finding_methods.py", line 36, in __call__ return self.function(hc, *self.args, **self.kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/user/p1cj_phy/fastpm/yt_astro_analysis/yt_astro_analysis/halo_analysis/halo_catalog/halo_finding_methods.py", line 88, in _rockstar_method rh = RockstarHaloFinder(ds, **finder_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/user/p1cj_phy/fastpm/yt_astro_analysis/yt_astro_analysis/halo_analysis/halo_finding/rockstar/rockstar.py", line 290, in __init__ p = self._setup_parameters(ts) ^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/user/p1cj_phy/fastpm/yt_astro_analysis/yt_astro_analysis/halo_analysis/halo_finding/rockstar/rockstar.py", line 299, in _setup_parameters tds.index # noqa: B018 ^^^^^^^^^ File "/home/user/p1cj_phy/fastpm/yt_astro_v/lib/python3.12/site-packages/yt/data_objects/static_output.py", line 605, in index self._instantiated_index = self._index_class( ^^^^^^^^^^^^^^^^^^ File "/home/user/p1cj_phy/fastpm/yt_astro_v/lib/python3.12/site-packages/yt/frontends/gadget/data_structures.py", line 195, in __init__ super().__init__(ds, dataset_type) File "/home/user/p1cj_phy/fastpm/yt_astro_v/lib/python3.12/site-packages/yt/geometry/particle_geometry_handler.py", line 73, in __init__ self._initialize_index() File "/home/user/p1cj_phy/fastpm/yt_astro_v/lib/python3.12/site-packages/yt/frontends/gadget/data_structures.py", line 204, in _initialize_index super()._initialize_index() File "/home/user/p1cj_phy/fastpm/yt_astro_v/lib/python3.12/site-packages/yt/frontends/sph/data_structures.py", line 89, in _initialize_index super()._initialize_index() File "/home/user/p1cj_phy/fastpm/yt_astro_v/lib/python3.12/site-packages/yt/geometry/particle_geometry_handler.py", line 237, in _initialize_index max_hsml = self._initialize_coarse_index() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/user/p1cj_phy/fastpm/yt_astro_v/lib/python3.12/site-packages/yt/geometry/particle_geometry_handler.py", line 280, in _initialize_coarse_index self.regions.masks = self.comm.mpi_allreduce(self.regions.masks, op="sum") ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/user/p1cj_phy/fastpm/yt_astro_v/lib/python3.12/site-packages/yt/utilities/parallel_tools/parallel_analysis_interface.py", line 297, in passage return func(self, *args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/user/p1cj_phy/fastpm/yt_astro_v/lib/python3.12/site-packages/yt/utilities/parallel_tools/parallel_analysis_interface.py", line 913, in mpi_allreduce self.comm.Allreduce( File "src/mpi4py/MPI.src/Comm.pyx", line 1115, in mpi4py.MPI.Comm.Allreduce P003 yt : [ERROR ] 2025-12-19 23:49:08,961 Exception: MPI_ERR_ARG: invalid argument of some other kind P003 yt : [ERROR ] 2025-12-19 23:49:08,961 Error occurred on rank 2. [Node001:85381] 3 more processes have sent help message help-mpi-api.txt / mpi-abort [Node001:85381] Set MCA parameter "orte_base_help_aggregate" to 0 to see all help / error messages -- I have tried different reader and writer combinations (4&9, 4&11, 4&13, 8&11 etc.). I am not able to understand this error and solve it. Also I didn't succeed in creating a code that converts data from big file to gadget 2 data format. Plese help.

2 1