Re: [yt-users] Problem with ParallellAnalysisInterface._mpi_allsum

The array shouldn't be too small. The data contain 1024^3 cells (20 million cells within the cut_region) and I am running it on 60 processors (120 doesn't work either). This is my script: from yt.mods import * from yt.analysis_modules.api import EnzoSimulation import numpy as na from krakenPlugins import * from mpi4py import MPI ########################################################### simName = '50Mpc_1024unigrid.par' dataDir = '/lustre/scratch/britton/box_size_study/50Mpc_1024/run_17f_cl_5D' es = EnzoSimulation('%s/%s' %(dataDir,simName),get_redshift_outputs=False) dataCntr = 0 numBins = 1000 allHisty = na.array([na.zeros(numBins+1)]) allHistx = na.array([na.zeros(numBins+1)]) es = es.allOutputs[:85] for output in es: pf = load('%s%s' %(dataDir,output['filename'][1:])) dd = pf.h.all_data() pc = PlotCollection(pf) cut = dd.cut_region(["grid['Metallicity'] <= 1.e-6","grid['Temperature'] <= 10.**5.","grid['Temperature'] >= 300.","grid['Baryon_Overdensity'] >= 1.","grid['Baryon_Overdensity'] <= 100."]) pc.add_profile_object(cut, ['Density','Ones'], weight=None, x_bins=numBins,x_log=True) ones = pc.plots[-1].data["Ones"] bod = pc.plots[-1].data["Density"] allHisty = na.concatenate((allHisty,[ones])) allHistx = na.concatenate((allHistx,[bod])) dataCntr += 1 del pf,dd,pc,cut,ones,bod if MPI.COMM_WORLD.rank == 0: print '***Saving to .npy file. UTC Time: %s***' na.save('%s/histograms_y.npy'%saveDir,allHisty) na.save('%s/histograms_x.npy'%saveDir,allHistx) On Mon, Jun 6, 2011 at 6:24 PM, Matthew Turk wrote:

Hi Anthony,

I tried it on a small dataset and I was unable to reproduce it. Do you think that the array is small enough that some of the processors aren't getting any data? I was able to get the profile command to work all the way down to arrays of size 19, run on 20 processors.

Could you post the entirety of your script?

-Matt

...

Hello,

I am trying to add a profile object to a Plot Collection (via pc.add_profile_object(data,fields) ) while running in parallel on Kraken. I get the following error: "TypeError: message: expecting a list or tuple" which ultimately comes from mpi4py.MPI.Comm.Allreduce which is called by ParallelAnalysisInterface._mpi_allsum(). In ._mpi_allsum() there is the following comment: "# We use old-school pickling here on the assumption

On Mon, Jun 6, 2011 at 5:15 PM, Anthony Harness wrote: the

...

arrays are relatively small ( < 1e7 elements )". The dataset I am working with is larger than 1e7 elements, so is _mpi_allsum not able to pass such a large array to Comm.Allreduce?

Thanks, Anthony

Here is the traceback:

File "/yt-2.1stable-py2.7-linux-x86_64.egg/yt/data_objects/profiles.py", line 146, in add_fields self._lazy_add_fields(fields, weight, accumulation) File "/yt-2.1stable-py2.7-linux-x86_64.egg/yt/data_objects/profiles.py", line 94, in _lazy_add_fields for gi,grid in enumerate(self._get_grids(fields)): File

"/yt-2.1stable-py2.7-linux-x86_64.egg/yt/utilities/parallel_tools/parallel_analysis_interface.py",

...

line 134, in __iter__ if not self.just_list: self.pobj._finalize_parallel() File "/yt-2.1stable-py2.7-linux-x86_64.egg/yt/data_objects/profiles.py", line 122, in _finalize_parallel self.__data[key] = self._mpi_allsum(self.__data[key]) File

"/yt-2.1stable-py2.7-linux-x86_64.egg/yt/utilities/parallel_tools/parallel_analysis_interface.py",

...

line 185, in passage return func(self, data) File

"/yt-2.1stable-py2.7-linux-x86_64.egg/yt/utilities/parallel_tools/parallel_analysis_interface.py",

...

line 1124, in _mpi_allsum MPI.COMM_WORLD.Allreduce(data, tr, op=MPI.SUM) File "Comm.pyx", line 530, in mpi4py.MPI.Comm.Allreduce (src/mpi4py_MPI.c:43646) File "message.pxi", line 426, in mpi4py.MPI._p_msg_cco.for_allreduce (src/mpi4py_MPI.c:14446) File "message.pxi", line 33, in mpi4py.MPI.message_simple (src/mpi4py_MPI.c:11108) TypeError: message: expecting a list or tuple

_______________________________________________ yt-users mailing list yt-users@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org

_______________________________________________ yt-users mailing list yt-users@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org