On Mon, Jun 6, 2011 at 7:40 PM, Matthew Turk <
matthewturk@gmail.com> wrote:
> Hi Anthony,
>
> Using the grid cutting method, which I thought might be causing
> problems, I was again unable to reproduce the issue. If you could,
> would you mind running with --detailed and sending me (off-list) the
> log file, so that I can try to examine the problematic output?
>
> -Matt
>
> On Mon, Jun 6, 2011 at 7:17 PM, Anthony Harness
> <
anthony.harness@colorado.edu> wrote:
>> The array shouldn't be too small. The data contain 1024^3 cells (20 million
>> cells within the cut_region) and I am running it on 60 processors (120
>> doesn't work either). This is my script:
>> from yt.mods import *
>> from yt.analysis_modules.api import EnzoSimulation
>> import numpy as na
>> from krakenPlugins import *
>> from mpi4py import MPI
>> ###########################################################
>> simName = '50Mpc_1024unigrid.par'
>> dataDir = '/lustre/scratch/britton/box_size_study/50Mpc_1024/run_17f_cl_5D'
>> es = EnzoSimulation('%s/%s' %(dataDir,simName),get_redshift_outputs=False)
>> dataCntr = 0
>> numBins = 1000
>> allHisty = na.array([na.zeros(numBins+1)])
>> allHistx = na.array([na.zeros(numBins+1)])
>> es = es.allOutputs[:85]
>> for output in es:
>> pf = load('%s%s' %(dataDir,output['filename'][1:]))
>> dd = pf.h.all_data()
>> pc = PlotCollection(pf)
>> cut = dd.cut_region(["grid['Metallicity'] <= 1.e-6","grid['Temperature'] <=
>> 10.**5.","grid['Temperature'] >= 300.","grid['Baryon_Overdensity'] >=
>> 1.","grid['Baryon_Overdensity'] <= 100."])
>> pc.add_profile_object(cut, ['Density','Ones'], weight=None,
>> x_bins=numBins,x_log=True)
>> ones = pc.plots[-1].data["Ones"]
>> bod = pc.plots[-1].data["Density"]
>> allHisty = na.concatenate((allHisty,[ones]))
>> allHistx = na.concatenate((allHistx,[bod]))
>> dataCntr += 1
>> del pf,dd,pc,cut,ones,bod
>> if MPI.COMM_WORLD.rank == 0:
>> print '***Saving to .npy file. UTC Time: %s***'
>> na.save('%s/histograms_y.npy'%saveDir,allHisty)
>> na.save('%s/histograms_x.npy'%saveDir,allHistx)
>>
>>
>> On Mon, Jun 6, 2011 at 6:24 PM, Matthew Turk <
matthewturk@gmail.com> wrote:
>>>
>>> Hi Anthony,
>>>
>>> I tried it on a small dataset and I was unable to reproduce it. Do
>>> you think that the array is small enough that some of the processors
>>> aren't getting any data? I was able to get the profile command to
>>> work all the way down to arrays of size 19, run on 20 processors.
>>>
>>> Could you post the entirety of your script?
>>>
>>> -Matt
>>>
>>> On Mon, Jun 6, 2011 at 5:15 PM, Anthony Harness
>>> <
anthony.harness@colorado.edu> wrote:
>>> > Hello,
>>> >
>>> > I am trying to add a profile object to a Plot Collection (via
>>> > pc.add_profile_object(data,fields) ) while running in parallel on
>>> > Kraken. I
>>> > get the following error: "TypeError: message: expecting a list or tuple"
>>> > which ultimately comes from mpi4py.MPI.Comm.Allreduce which is called by
>>> > ParallelAnalysisInterface._mpi_allsum(). In ._mpi_allsum() there is the
>>> > following comment: "# We use old-school pickling here on the assumption
>>> > the
>>> > arrays are relatively small ( < 1e7 elements )". The dataset I am
>>> > working
>>> > with is larger than 1e7 elements, so is _mpi_allsum not able to pass
>>> > such a
>>> > large array to Comm.Allreduce?
>>> >
>>> > Thanks,
>>> > Anthony
>>> >
>>> > Here is the traceback:
>>> >
>>> > File "/yt-2.1stable-py2.7-linux-x86_64.egg/yt/data_objects/profiles.py",
>>> > line 146, in add_fields
>>> > self._lazy_add_fields(fields, weight, accumulation)
>>> > File "/yt-2.1stable-py2.7-linux-x86_64.egg/yt/data_objects/profiles.py",
>>> > line 94, in _lazy_add_fields
>>> > for gi,grid in enumerate(self._get_grids(fields)):
>>> > File
>>> >
>>> > "/yt-2.1stable-py2.7-linux-x86_64.egg/yt/utilities/parallel_tools/parallel_analysis_interface.py",
>>> > line 134, in __iter__
>>> > if not self.just_list: self.pobj._finalize_parallel()
>>> > File "/yt-2.1stable-py2.7-linux-x86_64.egg/yt/data_objects/profiles.py",
>>> > line 122, in _finalize_parallel
>>> > self.__data[key] = self._mpi_allsum(self.__data[key])
>>> > File
>>> >
>>> > "/yt-2.1stable-py2.7-linux-x86_64.egg/yt/utilities/parallel_tools/parallel_analysis_interface.py",
>>> > line 185, in passage
>>> > return func(self, data)
>>> > File
>>> >
>>> > "/yt-2.1stable-py2.7-linux-x86_64.egg/yt/utilities/parallel_tools/parallel_analysis_interface.py",
>>> > line 1124, in _mpi_allsum
>>> > MPI.COMM_WORLD.Allreduce(data, tr, op=MPI.SUM)
>>> > File "Comm.pyx", line 530, in mpi4py.MPI.Comm.Allreduce
>>> > (src/mpi4py_MPI.c:43646)
>>> > File "message.pxi", line 426, in mpi4py.MPI._p_msg_cco.for_allreduce
>>> > (src/mpi4py_MPI.c:14446)
>>> > File "message.pxi", line 33, in mpi4py.MPI.message_simple
>>> > (src/mpi4py_MPI.c:11108)
>>> > TypeError: message: expecting a list or tuple
>>> >
>>> >
>>> > _______________________________________________
>>> > yt-users mailing list
>>> >
yt-users@lists.spacepope.org
>>> >
http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
>>> >
>>> >
>>> _______________________________________________
>>> yt-users mailing list
>>>
yt-users@lists.spacepope.org
>>>
http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
>>
>>
>> _______________________________________________
>> yt-users mailing list
>>
yt-users@lists.spacepope.org
>>
http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
>>
>>
>
_______________________________________________
yt-users mailing list
yt-users@lists.spacepope.org
http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org