yt-users

Download

yt-users@python.org

December 2025

2 participants
2 discussions

Running Rockstar Algorithm using yt astro analysis
by Nixin Saji Abraham Jan. 19, 2026

Jan. 19, 2026

I am trying to run rockstar algorithm using yt astro analysis. What I have with me is gadget 1 data which was converted from big file data format. I have generated 32, 34, 39 chunks of the same data in gadget 1 format. Majority of the time, I get the following error at the start itself. File "/home/user/p1cj_phy/fastpm/yt_astro_analysis/yt_astro_analysis/halo_analysis/halo_catalog/halo_catalog.py", line 346, in create self._run(save_halos, save_output, njobs=njobs, dynamic=dynamic) File "/home/user/p1cj_phy/fastpm/yt_astro_v/lib/python3.12/site-packages/yt/utilities/parallel_tools/parallel_analysis_interface.py", line 322, in barrierize retval = func(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^ File "/home/user/p1cj_phy/fastpm/yt_astro_analysis/yt_astro_analysis/halo_analysis/halo_catalog/halo_catalog.py", line 239, in _run self.finder_method(self) File "/home/user/p1cj_phy/fastpm/yt_astro_analysis/yt_astro_analysis/halo_analysis/halo_catalog/halo_finding_methods.py", line 36, in __call__ return self.function(hc, *self.args, **self.kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/user/p1cj_phy/fastpm/yt_astro_analysis/yt_astro_analysis/halo_analysis/halo_catalog/halo_finding_methods.py", line 88, in _rockstar_method rh = RockstarHaloFinder(ds, **finder_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/user/p1cj_phy/fastpm/yt_astro_analysis/yt_astro_analysis/halo_analysis/halo_finding/rockstar/rockstar.py", line 290, in __init__ p = self._setup_parameters(ts) ^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/user/p1cj_phy/fastpm/yt_astro_analysis/yt_astro_analysis/halo_analysis/halo_finding/rockstar/rockstar.py", line 299, in _setup_parameters tds.index # noqa: B018 ^^^^^^^^^ File "/home/user/p1cj_phy/fastpm/yt_astro_v/lib/python3.12/site-packages/yt/data_objects/static_output.py", line 605, in index self._instantiated_index = self._index_class( ^^^^^^^^^^^^^^^^^^ File "/home/user/p1cj_phy/fastpm/yt_astro_v/lib/python3.12/site-packages/yt/frontends/gadget/data_structures.py", line 195, in __init__ super().__init__(ds, dataset_type) File "/home/user/p1cj_phy/fastpm/yt_astro_v/lib/python3.12/site-packages/yt/geometry/particle_geometry_handler.py", line 73, in __init__ self._initialize_index() File "/home/user/p1cj_phy/fastpm/yt_astro_v/lib/python3.12/site-packages/yt/frontends/gadget/data_structures.py", line 204, in _initialize_index super()._initialize_index() File "/home/user/p1cj_phy/fastpm/yt_astro_v/lib/python3.12/site-packages/yt/frontends/sph/data_structures.py", line 89, in _initialize_index super()._initialize_index() File "/home/user/p1cj_phy/fastpm/yt_astro_v/lib/python3.12/site-packages/yt/geometry/particle_geometry_handler.py", line 237, in _initialize_index max_hsml = self._initialize_coarse_index() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/user/p1cj_phy/fastpm/yt_astro_v/lib/python3.12/site-packages/yt/geometry/particle_geometry_handler.py", line 280, in _initialize_coarse_index self.regions.masks = self.comm.mpi_allreduce(self.regions.masks, op="sum") ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/user/p1cj_phy/fastpm/yt_astro_v/lib/python3.12/site-packages/yt/utilities/parallel_tools/parallel_analysis_interface.py", line 297, in passage return func(self, *args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/user/p1cj_phy/fastpm/yt_astro_v/lib/python3.12/site-packages/yt/utilities/parallel_tools/parallel_analysis_interface.py", line 913, in mpi_allreduce self.comm.Allreduce( File "src/mpi4py/MPI.src/Comm.pyx", line 1115, in mpi4py.MPI.Comm.Allreduce P004 yt : [ERROR ] 2025-12-19 23:49:08,036 Exception: MPI_ERR_ARG: invalid argument of some other kind P004 yt : [ERROR ] 2025-12-19 23:49:08,036 Error occurred on rank 3. -------------------------------------------------------------------------- MPI_ABORT was invoked on rank 4 in communicator MPI_COMM_WORLD with errorcode 1. NOTE: invoking MPI_ABORT causes Open MPI to kill all MPI processes. You may or may not see output from other processes, depending on exactly when Open MPI kills them. -------------------------------------------------------------------------- File "/home/user/p1cj_phy/fastpm/my.example/rockstar.py", line 14, in <module> hc.create() File "/home/user/p1cj_phy/fastpm/yt_astro_analysis/yt_astro_analysis/halo_analysis/halo_catalog/halo_catalog.py", line 346, in create self._run(save_halos, save_output, njobs=njobs, dynamic=dynamic) File "/home/user/p1cj_phy/fastpm/yt_astro_v/lib/python3.12/site-packages/yt/utilities/parallel_tools/parallel_analysis_interface.py", line 322, in barrierize retval = func(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^ File "/home/user/p1cj_phy/fastpm/yt_astro_analysis/yt_astro_analysis/halo_analysis/halo_catalog/halo_catalog.py", line 239, in _run self.finder_method(self) File "/home/user/p1cj_phy/fastpm/yt_astro_analysis/yt_astro_analysis/halo_analysis/halo_catalog/halo_finding_methods.py", line 36, in __call__ return self.function(hc, *self.args, **self.kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/user/p1cj_phy/fastpm/yt_astro_analysis/yt_astro_analysis/halo_analysis/halo_catalog/halo_finding_methods.py", line 88, in _rockstar_method rh = RockstarHaloFinder(ds, **finder_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/user/p1cj_phy/fastpm/yt_astro_analysis/yt_astro_analysis/halo_analysis/halo_finding/rockstar/rockstar.py", line 290, in __init__ p = self._setup_parameters(ts) ^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/user/p1cj_phy/fastpm/yt_astro_analysis/yt_astro_analysis/halo_analysis/halo_finding/rockstar/rockstar.py", line 299, in _setup_parameters tds.index # noqa: B018 ^^^^^^^^^ File "/home/user/p1cj_phy/fastpm/yt_astro_v/lib/python3.12/site-packages/yt/data_objects/static_output.py", line 605, in index self._instantiated_index = self._index_class( ^^^^^^^^^^^^^^^^^^ File "/home/user/p1cj_phy/fastpm/yt_astro_v/lib/python3.12/site-packages/yt/frontends/gadget/data_structures.py", line 195, in __init__ super().__init__(ds, dataset_type) File "/home/user/p1cj_phy/fastpm/yt_astro_v/lib/python3.12/site-packages/yt/geometry/particle_geometry_handler.py", line 73, in __init__ self._initialize_index() File "/home/user/p1cj_phy/fastpm/yt_astro_v/lib/python3.12/site-packages/yt/frontends/gadget/data_structures.py", line 204, in _initialize_index super()._initialize_index() File "/home/user/p1cj_phy/fastpm/yt_astro_v/lib/python3.12/site-packages/yt/frontends/sph/data_structures.py", line 89, in _initialize_index super()._initialize_index() File "/home/user/p1cj_phy/fastpm/yt_astro_v/lib/python3.12/site-packages/yt/geometry/particle_geometry_handler.py", line 237, in _initialize_index max_hsml = self._initialize_coarse_index() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/user/p1cj_phy/fastpm/yt_astro_v/lib/python3.12/site-packages/yt/geometry/particle_geometry_handler.py", line 280, in _initialize_coarse_index self.regions.masks = self.comm.mpi_allreduce(self.regions.masks, op="sum") ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/user/p1cj_phy/fastpm/yt_astro_v/lib/python3.12/site-packages/yt/utilities/parallel_tools/parallel_analysis_interface.py", line 297, in passage return func(self, *args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/user/p1cj_phy/fastpm/yt_astro_v/lib/python3.12/site-packages/yt/utilities/parallel_tools/parallel_analysis_interface.py", line 913, in mpi_allreduce self.comm.Allreduce( File "src/mpi4py/MPI.src/Comm.pyx", line 1115, in mpi4py.MPI.Comm.Allreduce P001 yt : [ERROR ] 2025-12-19 23:49:08,554 Exception: MPI_ERR_ARG: invalid argument of some other kind P001 yt : [ERROR ] 2025-12-19 23:49:08,554 Error occurred on rank 0. File "/home/user/p1cj_phy/fastpm/my.example/rockstar.py", line 14, in <module> hc.create() File "/home/user/p1cj_phy/fastpm/yt_astro_analysis/yt_astro_analysis/halo_analysis/halo_catalog/halo_catalog.py", line 346, in create self._run(save_halos, save_output, njobs=njobs, dynamic=dynamic) File "/home/user/p1cj_phy/fastpm/yt_astro_v/lib/python3.12/site-packages/yt/utilities/parallel_tools/parallel_analysis_interface.py", line 322, in barrierize retval = func(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^ File "/home/user/p1cj_phy/fastpm/yt_astro_analysis/yt_astro_analysis/halo_analysis/halo_catalog/halo_catalog.py", line 239, in _run self.finder_method(self) File "/home/user/p1cj_phy/fastpm/yt_astro_analysis/yt_astro_analysis/halo_analysis/halo_catalog/halo_finding_methods.py", line 36, in __call__ return self.function(hc, *self.args, **self.kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/user/p1cj_phy/fastpm/yt_astro_analysis/yt_astro_analysis/halo_analysis/halo_catalog/halo_finding_methods.py", line 88, in _rockstar_method rh = RockstarHaloFinder(ds, **finder_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/user/p1cj_phy/fastpm/yt_astro_analysis/yt_astro_analysis/halo_analysis/halo_finding/rockstar/rockstar.py", line 290, in __init__ p = self._setup_parameters(ts) ^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/user/p1cj_phy/fastpm/yt_astro_analysis/yt_astro_analysis/halo_analysis/halo_finding/rockstar/rockstar.py", line 299, in _setup_parameters tds.index # noqa: B018 ^^^^^^^^^ File "/home/user/p1cj_phy/fastpm/yt_astro_v/lib/python3.12/site-packages/yt/data_objects/static_output.py", line 605, in index self._instantiated_index = self._index_class( ^^^^^^^^^^^^^^^^^^ File "/home/user/p1cj_phy/fastpm/yt_astro_v/lib/python3.12/site-packages/yt/frontends/gadget/data_structures.py", line 195, in __init__ super().__init__(ds, dataset_type) File "/home/user/p1cj_phy/fastpm/yt_astro_v/lib/python3.12/site-packages/yt/geometry/particle_geometry_handler.py", line 73, in __init__ self._initialize_index() File "/home/user/p1cj_phy/fastpm/yt_astro_v/lib/python3.12/site-packages/yt/frontends/gadget/data_structures.py", line 204, in _initialize_index super()._initialize_index() File "/home/user/p1cj_phy/fastpm/yt_astro_v/lib/python3.12/site-packages/yt/frontends/sph/data_structures.py", line 89, in _initialize_index super()._initialize_index() File "/home/user/p1cj_phy/fastpm/yt_astro_v/lib/python3.12/site-packages/yt/geometry/particle_geometry_handler.py", line 237, in _initialize_index max_hsml = self._initialize_coarse_index() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/user/p1cj_phy/fastpm/yt_astro_v/lib/python3.12/site-packages/yt/geometry/particle_geometry_handler.py", line 280, in _initialize_coarse_index self.regions.masks = self.comm.mpi_allreduce(self.regions.masks, op="sum") ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/user/p1cj_phy/fastpm/yt_astro_v/lib/python3.12/site-packages/yt/utilities/parallel_tools/parallel_analysis_interface.py", line 297, in passage return func(self, *args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/user/p1cj_phy/fastpm/yt_astro_v/lib/python3.12/site-packages/yt/utilities/parallel_tools/parallel_analysis_interface.py", line 913, in mpi_allreduce self.comm.Allreduce( File "src/mpi4py/MPI.src/Comm.pyx", line 1115, in mpi4py.MPI.Comm.Allreduce P002 yt : [ERROR ] 2025-12-19 23:49:08,936 Exception: MPI_ERR_ARG: invalid argument of some other kind P002 yt : [ERROR ] 2025-12-19 23:49:08,936 Error occurred on rank 1. File "/home/user/p1cj_phy/fastpm/my.example/rockstar.py", line 14, in <module> hc.create() File "/home/user/p1cj_phy/fastpm/yt_astro_analysis/yt_astro_analysis/halo_analysis/halo_catalog/halo_catalog.py", line 346, in create self._run(save_halos, save_output, njobs=njobs, dynamic=dynamic) File "/home/user/p1cj_phy/fastpm/yt_astro_v/lib/python3.12/site-packages/yt/utilities/parallel_tools/parallel_analysis_interface.py", line 322, in barrierize retval = func(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^ File "/home/user/p1cj_phy/fastpm/yt_astro_analysis/yt_astro_analysis/halo_analysis/halo_catalog/halo_catalog.py", line 239, in _run self.finder_method(self) File "/home/user/p1cj_phy/fastpm/yt_astro_analysis/yt_astro_analysis/halo_analysis/halo_catalog/halo_finding_methods.py", line 36, in __call__ return self.function(hc, *self.args, **self.kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/user/p1cj_phy/fastpm/yt_astro_analysis/yt_astro_analysis/halo_analysis/halo_catalog/halo_finding_methods.py", line 88, in _rockstar_method rh = RockstarHaloFinder(ds, **finder_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/user/p1cj_phy/fastpm/yt_astro_analysis/yt_astro_analysis/halo_analysis/halo_finding/rockstar/rockstar.py", line 290, in __init__ p = self._setup_parameters(ts) ^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/user/p1cj_phy/fastpm/yt_astro_analysis/yt_astro_analysis/halo_analysis/halo_finding/rockstar/rockstar.py", line 299, in _setup_parameters tds.index # noqa: B018 ^^^^^^^^^ File "/home/user/p1cj_phy/fastpm/yt_astro_v/lib/python3.12/site-packages/yt/data_objects/static_output.py", line 605, in index self._instantiated_index = self._index_class( ^^^^^^^^^^^^^^^^^^ File "/home/user/p1cj_phy/fastpm/yt_astro_v/lib/python3.12/site-packages/yt/frontends/gadget/data_structures.py", line 195, in __init__ super().__init__(ds, dataset_type) File "/home/user/p1cj_phy/fastpm/yt_astro_v/lib/python3.12/site-packages/yt/geometry/particle_geometry_handler.py", line 73, in __init__ self._initialize_index() File "/home/user/p1cj_phy/fastpm/yt_astro_v/lib/python3.12/site-packages/yt/frontends/gadget/data_structures.py", line 204, in _initialize_index super()._initialize_index() File "/home/user/p1cj_phy/fastpm/yt_astro_v/lib/python3.12/site-packages/yt/frontends/sph/data_structures.py", line 89, in _initialize_index super()._initialize_index() File "/home/user/p1cj_phy/fastpm/yt_astro_v/lib/python3.12/site-packages/yt/geometry/particle_geometry_handler.py", line 237, in _initialize_index max_hsml = self._initialize_coarse_index() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/user/p1cj_phy/fastpm/yt_astro_v/lib/python3.12/site-packages/yt/geometry/particle_geometry_handler.py", line 280, in _initialize_coarse_index self.regions.masks = self.comm.mpi_allreduce(self.regions.masks, op="sum") ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/user/p1cj_phy/fastpm/yt_astro_v/lib/python3.12/site-packages/yt/utilities/parallel_tools/parallel_analysis_interface.py", line 297, in passage return func(self, *args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/user/p1cj_phy/fastpm/yt_astro_v/lib/python3.12/site-packages/yt/utilities/parallel_tools/parallel_analysis_interface.py", line 913, in mpi_allreduce self.comm.Allreduce( File "src/mpi4py/MPI.src/Comm.pyx", line 1115, in mpi4py.MPI.Comm.Allreduce P003 yt : [ERROR ] 2025-12-19 23:49:08,961 Exception: MPI_ERR_ARG: invalid argument of some other kind P003 yt : [ERROR ] 2025-12-19 23:49:08,961 Error occurred on rank 2. [Node001:85381] 3 more processes have sent help message help-mpi-api.txt / mpi-abort [Node001:85381] Set MCA parameter "orte_base_help_aggregate" to 0 to see all help / error messages -- I have tried different reader and writer combinations (4&9, 4&11, 4&13, 8&11 etc.). I am not able to understand this error and solve it. Also I didn't succeed in creating a code that converts data from big file to gadget 2 data format. Plese help.

2 1

[yt-Survey] understand what the community needs and feedback
by Shin-Rong Tsai Dec. 11, 2025

Dec. 11, 2025

Hi All, We recently received Azure cloud compute resources, and we expect to provide some compute resources to the yt community for testing new functionality! Before this, we would like to understand what the community needs and feedback. *Please help us complete the survey. It takes less than 10 min:* https://forms.gle/NSgLFxv1Z6e6H2jC8 Thanks, Shin-Rong

1 0