Hi Suoqing, I ran the code in the interactive mode and it returned me the same error as it was with submitting the job script. What happens is that It checks through all the files and instead of producing the output files or images, it returns me the error. Best On Tue, Oct 18, 2016 at 4:48 PM, Suoqing Ji <suoqing@physics.ucsb.edu> wrote:
Hi Tazkera,
I have the access to Stampede but I can not reproduce your error either (using a slightly different script though).
apparently jobs can only be submitted through the $WORK folder on Stampede
Btw, I believe you could submit job from both work and scratch directory on Stampede.
Could you run your code in interactive mode, by
srun -p normal -t 0:30:00 -n 16 -A _your_project_code_ --pty /bin/bash -l
and then enter
python abundance.py
and see what’s going on?
Best wishes, -- Suoqing JI Ph.D Candidate Department of Physics University of California, Santa Barbara http://web.physics.ucsb.edu/~suoqing
On Oct 18, 2016, at 1:20 PM, tazkera haque <h.tazkera@gmail.com> wrote:
I typed
$ module avail HDF5
------------------------------------------------- /opt/apps/intel15/mvapich2_2_1/modulefiles ------------------------------ -------------------- phdf5/1.8.16
-------------------------------------------------------- /opt/apps/intel15/modulefiles ------------------------------ -------------------------- hdf5/1.8.16 (m,L)
Where: L: Module is loaded m: built for host and native MIC
$ module avail h5py returns blank, if that helps
On Tue, Oct 18, 2016 at 4:08 PM, Nathan Goldbaum <nathan12343@gmail.com> wrote:
Hi Tazkera,
When I tried googling your error message last night, I found that it's associated with more than one MPI process trying to access an HDF5 file at the same time, presumably using a serial version of the HDF5 library.
I just did a quick test and I'm unable to reproduce here on my laptop. Unfortunately I don't have access to stampede so can't reproduce there.
Can you share exactly which h5py and HDF5 library versions you're using?
-Nathan
On Tue, Oct 18, 2016 at 3:05 PM, tazkera haque <h.tazkera@gmail.com> wrote:
HI Nathan,
Sorry to bother you again, but the problem seems to prevail even working from $WORK directory. I got the same error msg this morning again with a different script. Do you see anything wrong with the script I attached ? I have used it for a long time now without any sort of error.
Thanks
On Tue, Oct 18, 2016 at 2:34 AM, tazkera haque <h.tazkera@gmail.com> wrote:
HI Nathan,
I figured out what was going wrong, I submitted my job script from the $SCRATCH folder. apparently jobs can only be submitted through the $WORK folder on Stampede. thanks for your prompt response though
On Tue, Oct 18, 2016 at 1:22 AM, tazkera haque <h.tazkera@gmail.com> wrote:
Hi Nathan,
I tried with one file in my ipython notebook, it seems to work there
On Tue, Oct 18, 2016 at 12:54 AM, tazkera haque <h.tazkera@gmail.com> wrote:
yes it's being run in parallel, I didn't check with one core yet, I will let you know what happens then
On Tue, Oct 18, 2016 at 12:50 AM, Nathan Goldbaum < nathan12343@gmail.com> wrote:
> Is the script being run in parallel? If so, does it crash if you run > it on only one core? > > Nathan > > > On Monday, October 17, 2016, tazkera haque <h.tazkera@gmail.com> > wrote: > >> HI people, >> >> I am using yt 3.3.1 and submitting my SLURM script to stampede. >> I was using this script to find abundances of C, O etc through >> checkpoint files in FLASH. while my script worked fine with old yt (3.1) , >> suddenly it crashed today and returned me the following error: >> >> *yt : [INFO ] 2016-10-17 23:22:24,295 Parameters: current_time >> = 28.1847530806* >> *yt : [INFO ] 2016-10-17 23:22:24,295 Parameters: >> domain_dimensions = [128 128 128]* >> *yt : [INFO ] 2016-10-17 23:22:24,296 Parameters: >> domain_left_edge = [ -2.80000000e+10 -2.80000000e+10 >> -2.80000000e+10]* >> *yt : [INFO ] 2016-10-17 23:22:24,296 Parameters: >> domain_right_edge = [ 2.80000000e+10 2.80000000e+10 >> 2.80000000e+10]* >> *yt : [INFO ] 2016-10-17 23:22:24,296 Parameters: >> cosmological_simulation = 0.0* >> *Executin lessg abundance.py* >> *Traceback (most recent call last):* >> * File "abundance2.py", line 304, in <module>* >> * main(chkFilenames_own)* >> * File "abundance2.py", line 59, in main* >> * pf = yt.load(filenames[n])* >> * File >> "/work/03858/thaque56/sw/yt-new-3.3/yt-conda/lib/python2.7/site-packages/yt/convenience.py", >> line 79, in load* >> * if c._is_valid(*args, **kwargs): candidates.append(n)* >> * File >> "/work/03858/thaque56/sw/yt-new-3.3/yt-conda/lib/python2.7/site-packages/yt/frontends/flash/data_structures.py", >> line 478, in _is_valid* >> * if "bounding box" not in fileh["/"].keys() \* >> * File >> "/work/03858/thaque56/sw/yt-new-3.3/yt-conda/lib/python2.7/site-packages/h5py/_hl/base.py", >> line 368, in keys* >> * return list(self)* >> * File "h5py/_objects.pyx", line 54, in >> h5py._objects.with_phil.wrapper >> (/home/ilan/minonda/conda-bld/work/h5py/_objects.c:2696)* >> * File "h5py/_objects.pyx", line 55, in >> h5py._objects.with_phil.wrapper >> (/home/ilan/minonda/conda-bld/work/h5py/_objects.c:2654)* >> * File >> "/work/03858/thaque56/sw/yt-new-3.3/yt-conda/lib/python2.7/site-packages/h5py/_hl/group.py", >> line 303, in __len__* >> * return self.id.get_num_objs()* >> * File "h5py/_objects.pyx", line 54, in >> h5py._objects.with_phil.wrapper >> (/home/ilan/minonda/conda-bld/work/h5py/_objects.c:2696)* >> * File "h5py/_objects.pyx", line 55, in >> h5py._objects.with_phil.wrapper >> (/home/ilan/minonda/conda-bld/work/h5py/_objects.c:2654)* >> * File "h5py/h5g.pyx", line 321, in h5py.h5g.GroupID.get_num_objs >> (/home/ilan/minonda/conda-bld/work/h5py/h5g.c:4194)* >> *RuntimeError: Can't determine (Bad symbol table node signature)* >> *[c560-102.stampede.tacc.utexas.edu:mpispawn_0][child_handler] MPI >> process (rank: 0, pid: 22137) exited with status 1* >> *TACC: MPI job exited with code: 1* >> >> *TACC: Shutdown complete. Exiting.* >> >> I was wondering if there is something wrong with my code or the new >> yt. I am also attaching my code here to look at. Thanks in advance >> >> Best >> Tazkera >> > > _______________________________________________ > yt-users mailing list > yt-users@lists.spacepope.org > http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org > >
_______________________________________________ yt-users mailing list yt-users@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
_______________________________________________ yt-users mailing list yt-users@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
_______________________________________________ yt-users mailing list yt-users@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
_______________________________________________ yt-users mailing list yt-users@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org