[yt-users] Re: Error while running yt with MPI

It works now, thank you. On 2/21/2018 11:40 AM, Nathan Goldbaum wrote:

Hi Nick,

Are you running yt 3.4.1? I'm pretty sure you're running into the bug that was fixed by this pull request:

https://github.com/yt-project/yt/pull/1538

We just released yt 3.4.1 last Friday and it should be available to install via conda-forge:

$ conda update -c conda-forge yt

Hope that helps,

Nathan

On Wed, Feb 21, 2018 at 11:29 AM, Nick Gnedin mailto:ngnedin@gmail.com> wrote:

Folks,

I am trying to run yt halo finder with MPI, and I am encounter the following error. Could you perhaps give me a hint how to debug it further?

Here is a simple script:

---- SCRIPT ------- import yt from yt.analysis_modules.halo_analysis.api import HaloCatalog yt.enable_parallelism() path = "." aexp = "0.0802" d = yt.load(path+"/rei10_a"+aexp +"/rei10_a"+aexp+".art") hc = HaloCatalog(data_ds=d,finder_method='hop',output_dir=path+"/a="+aexp+"/hop2",finder_kwargs={"dm_only":False,"ptype":"N-BODY"}) hc.create() -------------------

It runs just fine in a serial mode (with yt.enable_parallelism() call commented out), but with MPI it gives a type error deep inside yt:

---- SERIAL OUTPUT ------- /data/gnedin/REI/TMP >p hf.py yt : [WARNING  ] 2018-02-21 11:02:26,502 DeltaDC != 0, which implies auni != abox.  Be sure you understand which expansion parameter is appropriate for your use! (Gnedin, Kravtsov, & Rudd 2011) yt : [INFO     ] 2018-02-21 11:02:26,520 Parameters: current_time    = 392162249.38323104 yr yt : [INFO     ] 2018-02-21 11:02:26,520 Parameters: domain_dimensions       = [64 64 64] yt : [INFO     ] 2018-02-21 11:02:26,521 Parameters: domain_left_edge       = [ 0.  0.  0.] yt : [INFO     ] 2018-02-21 11:02:26,521 Parameters: domain_right_edge       = [ 64.  64.  64.] yt : [INFO     ] 2018-02-21 11:02:26,522 Parameters: cosmological_simulation   = True yt : [INFO     ] 2018-02-21 11:02:26,522 Parameters: current_redshift       = 11.470684242933336 yt : [INFO     ] 2018-02-21 11:02:26,522 Parameters: omega_lambda    = 0.6964000165462494 yt : [INFO     ] 2018-02-21 11:02:26,522 Parameters: omega_matter    = 0.3035999834537506 yt : [INFO     ] 2018-02-21 11:02:26,522 Parameters: hubble_constant       = 0.6814000010490417 yt : [INFO     ] 2018-02-21 11:02:34,808 Created 256 chunks for ARTIO yt : [INFO     ] 2018-02-21 11:02:38,656 Initializing HOP yt : [INFO     ] 2018-02-21 11:02:38,658 Created 256 chunks for ARTIO Copying arrays for 16777216 particles Calling hop... 16777216 1.600e+02 nSmooth = 65 kd->nActive = 16777216 Building Tree... Finding Densities... Finding Densest Neighbors... Grouping... Merging Groups... Writing Output... All Done!Calling regroup... No minimum group size specified.  Assuming 10 particles. ngroups = 51045 yt : [INFO     ] 2018-02-21 11:07:11,099 Parsing outputs yt : [INFO     ] 2018-02-21 11:07:38,770 Parameters: current_time    = 0.0 yt : [INFO     ] 2018-02-21 11:07:38,770 Parameters: domain_dimensions       = [2 2 2] yt : [INFO     ] 2018-02-21 11:07:38,770 Parameters: domain_left_edge       = [ 0.  0.  0.] yt : [INFO     ] 2018-02-21 11:07:38,771 Parameters: domain_right_edge       = [  3.56553853e+24   3.56553853e+24  3.56553853e+24] yt : [INFO     ] 2018-02-21 11:07:38,771 Parameters: cosmological_simulation   = 0.0 yt : [INFO     ] 2018-02-21 11:07:38,789 Allocating for 3.240e+02 particles (index particle type 'all') yt : [INFO     ] 2018-02-21 11:07:38,790 Identified 1.700e+01 octs /data/gnedin/soft/anaconda3/lib/python3.6/site-packages/yt/units/yt_array.py:990: RuntimeWarning: divide by zero encountered in true_divide   return super(YTArray, self).__truediv__(ro) yt : [INFO     ] 2018-02-21 11:07:39,223 Saving halo catalog (324 halos) to ./a=0.0802/hop/hop. ---------------------------

------ MPI OUTPUT --------- /data/gnedin/REI/TMP >/data/gnedin/soft/anaconda3/bin/mpirun -n 2 /data/gnedin/soft/anaconda3/bin/python3 hf.py yt : [INFO     ] 2018-02-21 11:20:47,675 Global parallel computation enabled: 0 / 2 yt : [INFO     ] 2018-02-21 11:20:47,676 Global parallel computation enabled: 1 / 2 P000 yt : [WARNING  ] 2018-02-21 11:20:47,681 DeltaDC != 0, which implies auni != abox.  Be sure you understand which expansion parameter is appropriate for your use! (Gnedin, Kravtsov, & Rudd 2011) P001 yt : [WARNING  ] 2018-02-21 11:20:47,687 DeltaDC != 0, which implies auni != abox.  Be sure you understand which expansion parameter is appropriate for your use! (Gnedin, Kravtsov, & Rudd 2011) P000 yt : [INFO     ] 2018-02-21 11:20:47,691 Parameters: current_time            = 392162249.38323104 yr P000 yt : [INFO     ] 2018-02-21 11:20:47,691 Parameters: domain_dimensions         = [64 64 64] P000 yt : [INFO     ] 2018-02-21 11:20:47,692 Parameters: domain_left_edge          = [ 0.  0.  0.] P000 yt : [INFO     ] 2018-02-21 11:20:47,692 Parameters: domain_right_edge         = [ 64.  64.  64.] P000 yt : [INFO     ] 2018-02-21 11:20:47,692 Parameters: cosmological_simulation   = True P000 yt : [INFO     ] 2018-02-21 11:20:47,692 Parameters: current_redshift          = 11.470684242933336 P000 yt : [INFO     ] 2018-02-21 11:20:47,692 Parameters: omega_lambda            = 0.6964000165462494 P000 yt : [INFO     ] 2018-02-21 11:20:47,692 Parameters: omega_matter            = 0.3035999834537506 P000 yt : [INFO     ] 2018-02-21 11:20:47,692 Parameters: hubble_constant           = 0.6814000010490417   File "hf.py", line 15, in <module>     hc.create()   File "/data/gnedin/soft/anaconda3/lib/python3.6/site-packages/yt/analysis_modules/halo_analysis/halo_catalog.py", line 334, in create     self._run(save_halos, save_catalog, njobs=njobs, dynamic=dynamic)   File "/data/gnedin/soft/anaconda3/lib/python3.6/site-packages/yt/utilities/parallel_tools/parallel_analysis_interface.py", line 306, in barrierize     retval = func(*args, **kwargs)   File "/data/gnedin/soft/anaconda3/lib/python3.6/site-packages/yt/analysis_modules/halo_analysis/halo_catalog.py", line 403, in _run     self.halos_ds = self.finder_method(self.data_ds)   File "/data/gnedin/soft/anaconda3/lib/python3.6/site-packages/yt/analysis_modules/halo_analysis/halo_finding_methods.py", line 44, in __call__     return self.function(ds, *self.args, **self.kwargs)   File "/data/gnedin/soft/anaconda3/lib/python3.6/site-packages/yt/analysis_modules/halo_analysis/halo_finding_methods.py", line 51, in _hop_method     halo_list = HOPHaloFinder(ds, **finder_kwargs)   File "/data/gnedin/soft/anaconda3/lib/python3.6/site-packages/yt/analysis_modules/halo_finding/halo_objects.py", line 1483, in __init__     padding=self.padding)   File "/data/gnedin/soft/anaconda3/lib/python3.6/site-packages/yt/utilities/parallel_tools/parallel_analysis_interface.py", line 1191, in partition_index_3d     cx, cy, cz = np.unravel_index(mi, cc)   File "hf.py", line 15, in <module>     hc.create()   File "/data/gnedin/soft/anaconda3/lib/python3.6/site-packages/yt/analysis_modules/halo_analysis/halo_catalog.py", line 334, in create     self._run(save_halos, save_catalog, njobs=njobs, dynamic=dynamic)   File "/data/gnedin/soft/anaconda3/lib/python3.6/site-packages/yt/utilities/parallel_tools/parallel_analysis_interface.py", line 306, in barrierize     retval = func(*args, **kwargs)   File "/data/gnedin/soft/anaconda3/lib/python3.6/site-packages/yt/analysis_modules/halo_analysis/halo_catalog.py", line 403, in _run     self.halos_ds = self.finder_method(self.data_ds)   File "/data/gnedin/soft/anaconda3/lib/python3.6/site-packages/yt/analysis_modules/halo_analysis/halo_finding_methods.py", line 44, in __call__     return self.function(ds, *self.args, **self.kwargs)   File "/data/gnedin/soft/anaconda3/lib/python3.6/site-packages/yt/analysis_modules/halo_analysis/halo_finding_methods.py", line 51, in _hop_method     halo_list = HOPHaloFinder(ds, **finder_kwargs)   File "/data/gnedin/soft/anaconda3/lib/python3.6/site-packages/yt/analysis_modules/halo_finding/halo_objects.py", line 1483, in __init__     padding=self.padding)   File "/data/gnedin/soft/anaconda3/lib/python3.6/site-packages/yt/utilities/parallel_tools/parallel_analysis_interface.py", line 1191, in partition_index_3d     cx, cy, cz = np.unravel_index(mi, cc) P000 yt : [ERROR    ] 2018-02-21 11:20:48,807 TypeError: Iterator operand 0 dtype could not be cast from dtype('float64') to dtype('int64') according to the rule 'same_kind' P000 yt : [ERROR    ] 2018-02-21 11:20:48,807 Error occured on rank 0. application called MPI_Abort(MPI_COMM_WORLD, 1) - process 0 P001 yt : [ERROR    ] 2018-02-21 11:20:48,807 TypeError: Iterator operand 0 dtype could not be cast from dtype('float64') to dtype('int64') according to the rule 'same_kind' P001 yt : [ERROR    ] 2018-02-21 11:20:48,807 Error occured on rank 1. application called MPI_Abort(MPI_COMM_WORLD, 1) - process 1 -------------------------------------

Any suggestion would be highly appreciated.

Nick Gnedin

_______________________________________________ yt-users mailing list -- yt-users@python.org mailto:yt-users@python.org To unsubscribe send an email to yt-users-leave@python.org mailto:yt-users-leave@python.org _______________________________________________ yt-users mailing list -- yt-users@python.org mailto:yt-users@python.org To unsubscribe send an email to yt-users-leave@python.org mailto:yt-users-leave@python.org