Hi, I am trying to install yt on Steele. When I do "bash install_script.sh", it crashes with H5 warnings and libz.a error.
Any idea?
shankar KU Cosmology
Building dependencies for H5E.c H5E.c: in function `H5Eprint': H5E.c:467: warning: trace info was not inserted Building dependencies for H5F.c H5FDmulti.c: in function `H5Pset_fapl_split': H5FDmulti.c:291: warning: trace info was not inserted H5FDmulti.c: in function `H5Pset_fapl_multi': H5FDmulti.c:430: unable to parse `const char * const *memb_name' H5FDmulti.c: in function `H5Pget_fapl_multi': H5FDmulti.c:537: warning: trace info was not inserted H5FDmulti.c: in function `H5Pset_dxpl_multi': H5FDmulti.c:606: warning: trace info was not inserted H5FDmulti.c: in function `H5Pget_dxpl_multi': H5FDmulti.c:658: warning: trace info was not inserted Building dependencies for H5FDstdio.c H5FDstdio.c: in function `H5Pset_fapl_stdio': H5FDstdio.c:272: warning: trace info was not inserted make[1]: Leaving directory `/autohome/u118/agarwa23/enzo1/src/yt/doc/yt-x86_64/src/hdf5-1.6.8/src' make[1]: Entering directory `/autohome/u118/agarwa23/enzo1/src/yt/doc/yt-x86_64/src/hdf5-1.6.8/src' mkdir .libs mpicc -std=c99 -O3 -Wl,-s -I. -I. -UH5_DEBUG_API -DNDEBUG -D_LARGEFILE_SOURCE -D_LARGEFILE64_SOURCE -D_FILE_OFFSET_BITS=64 -D_POSIX_SOURCE -D_BSD_SOURCE -c H5Dcontig.c -fPIC -DPIC -o .libs/H5Dcontig.lo H5Dcontig.c(497): warning #556: a value of type "const H5D_dxpl_cache_t *" cannot be assigned to an entity of type "H5D_dxpl_cache_t *" H5D_BUILD_IO_INFO(&io_info,dset,dxpl_cache,dxpl_id,store); ^
mpicc -std=c99 -O3 -Wl,-s -I. -I. -UH5_DEBUG_API -DNDEBUG -D_LARGEFILE_SOURCE -D_LARGEFILE64_SOURCE -D_FILE_OFFSET_BITS=64 -D_POSIX_SOURCE - H5Dio.c(3208): warning #556: a value of type "const H5D_dxpl_cache_t *" cannot be assigned to an entity of type "H5D_dxpl_cache_t *" io_info->dxpl_cache=dxpl_cache; ^
mpicc -std=c99 -O3 -Wl,-s -I. -I. -UH5_DEBUG_API -DNDEBUG -D_LARGEFILE_SOURCE -D_LARGEFILE64_SOURCE -D_FILE_OFFSET_BITS=64 -D_POSIX_SOURCE -D_BSD_SOURCE -c H5FD.c -fPIC -DPIC -o .libs/H5FD.lo H5FD.c(3670): warning #810: conversion from "void *" to "herr_t={int}" may lose significant bits HGOTO_ERROR(H5E_VFL, H5E_UNSUPPORTED, NULL, "file driver has no `get_vfd_handle' method"); ^
mpicc -std=c99 -O3 -Wl,-s -I. -I. -UH5_DEBUG_API -DNDEBUG -D_LARGEFILE_SOURCE -D_LARGEFILE64_SOURCE -D_FILE_OFFSET_BITS=64 -D_POSIX_SOURCE -D_BSD_SOURCE -c H5FDmulti.c -fPIC -DPIC -o .libs/H5FDmulti.lo H5FDmulti.c(208): warning #266: function "strlen" declared implicitly if (NULL==(x=malloc(strlen(s)+1))) return NULL; ^
H5FDmulti.c(208): warning #167: argument of type "const char *" is incompatible with parameter of type "void *" if (NULL==(x=malloc(strlen(s)+1))) return NULL; ^
H5FDmulti.c(209): warning #266: function "strcpy" declared implicitly strcpy(x, s); ^
H5FDmulti.c(209): warning #167: argument of type "const char *" is incompatible with parameter of type "void *" strcpy(x, s); ^
H5FDmulti.c(1924): warning #266: function "memset" declared implicitly UNIQUE_MEMBERS(file->fa.memb_map, mt) { ^
mpicc -std=c99 -O3 -Wl,-s -I. -I. -UH5_DEBUG_API -DNDEBUG -D_LARGEFILE_SOURCE -D_LARGEFILE64_SOURCE -D_FILE_OFFSET_BITS=64 -D_POSIX_SOURCE -D_BSD_SOURCE -c H5FDstdio.c -fPIC -DPIC -o .libs/H5FDstdio.lo H5FDstdio.c(774): warning #266: function "memset" declared implicitly memset(buf, 0, size); ^
H5FDstdio.c(800): warning #266: function "memset" declared implicitly memset((unsigned char *)buf + size - nbytes, 0, nbytes); ^
H5FDstdio.c(815): warning #266: function "memset" declared implicitly memset((unsigned char *)buf + n, 0, (size - n)); ^
mpicc -std=c99 -O3 -Wl,-s -I. -I. -UH5_DEBUG_API -DNDEBUG -D_LARGEFILE_SOURCE -D_LARGEFILE64_SOURCE -D_FILE_OFFSET_BITS=64 -D_POSIX_SOURCE -D_BSD_SOURCE -c H5Gnode.c -fPIC -DPIC -o .libs/H5Gnode.lo H5Gnode.c(1671): warning #188: enumerated type mixed with another type udata->type = H5G_get_type(&(sn->entry[loc_idx]), dxpl_id); ^
mpicc -std=c99 -O3 -Wl,-s -I. -I. -UH5_DEBUG_API -DNDEBUG -D_LARGEFILE_SOURCE -D_LARGEFILE64_SOURCE -D_FILE_OFFSET_BITS=64 -D_POSIX_SOURCE -D_BSD_SOURCE -c H5Gstab.c -fPIC -DPIC -o .libs/H5Gstab.lo
mpicc -std=c99 -O3 -Wl,-s -I. -I. -UH5_DEBUG_API -DNDEBUG -D_LARGEFILE_SOURCE -D_LARGEFILE64_SOURCE -D_FILE_OFFSET_BITS=64 -D_POSIX_SOURCE -D_BSD_SOURCE -c H5Obogus.c -fPIC -DPIC -o .libs/H5Obogus.lo
mpicc -std=c99 -O3 -Wl,-s -I. -I. -UH5_DEBUG_API -DNDEBUG -D_LARGEFILE_SOURCE -D_LARGEFILE64_SOURCE -D_FILE_OFFSET_BITS=64 -D_POSIX_SOURCE -D_BSD_SOURCE -c H5Spoint.c -fPIC -DPIC -o .libs/H5Spoint.lo H5Spoint.c(879): warning #167: argument of type "const hsize_t={unsigned long long} **" is incompatible with parameter of type "const hsize_t={unsigned long long} *" if((ret_value=H5S_select_elements(space,op,num_elem,(const hsize_t **)coord))<0) ^
mpicc -std=c99 -O3 -Wl,-s -I. -I. -UH5_DEBUG_API -DNDEBUG -D_LARGEFILE_SOURCE -D_LARGEFILE64_SOURCE -D_FILE_OFFSET_BITS=64 -D_POSIX_SOURCE -D_BSD_SOURCE -c H5Spoint.c -o H5Spoint.o >/dev/null 2>&1 mv -f .libs/H5Spoint.lo H5Spoint.lo rm -f .libs/H5Sselect.lo mpicc -std=c99 -O3 -Wl,-s -I. -I. -UH5_DEBUG_API -DNDEBUG -D_LARGEFILE_SOURCE -D_LARGEFILE64_SOURCE -D_FILE_OFFSET_BITS=64 -D_POSIX_SOURCE -D_BSD_SOURCE -c H5Sselec mv -f .libs/H5Zszip.lo H5Zszip.lo rm -fr .libs/libhdf5.la .libs/libhdf5.* .libs/libhdf5.* mpicc -shared H5.lo H5A.lo H5AC.lo H5B.lo H5Bcache.lo H5C.lo H5D.lo H5Dcontig.lo H5Dcompact.lo H5Defl.lo H5Dio.lo H5Distore.lo H5Dmpio.lo H5Dselect.lo H5Dtest.lo H5E.lo H5F.lo H5Fdbg.lo H5Fmount.lo H5Fsfile.lo H5Fsuper.lo H5FD.lo H5FDcore.lo H5FDfamily.lo H5FDlog.lo H5FDmpi.lo H5FDmpio.lo H5FDmpiposix.lo H5FDmulti.lo H5FDsec2.lo H5FDstdio.lo H5FL.lo H5FO.lo H5FS.lo H5G.lo H5Gent.lo H5Gname.lo H5Gnode.lo H5Gstab.lo H5Gtest.lo H5Gtraverse.lo H5HG.lo H5HGdbg.lo H5HL.lo H5HLdbg.lo H5HP.lo H5I.lo H5MF.lo H5MM.lo H5O.lo H5Oattr.lo H5Obogus.lo H5Ocache.lo H5Ocont.lo H5Odtype.lo H5Oefl.lo H5Ofill.lo H5Olayout.lo H5Omtime.lo H5Oname.lo H5Onull.lo H5Opline.lo H5Osdspace.lo H5Oshared.lo H5Ostab.lo H5P.lo H5Pdcpl.lo H5Pdxpl.lo H5Pfapl.lo H5Pfcpl.lo H5Ptest.lo H5R.lo H5RC.lo H5RS.lo H5S.lo H5Sall.lo H5Shyper.lo H5Smpio.lo H5Snone.lo H5Spoint.lo H5Sselect.lo H5Stest.lo H5SL.lo H5ST.lo H5T.lo H5Tarray.lo H5Tbit.lo H5Tcommit.lo H5Tcompound.lo H5Tconv.lo H5Tcset.lo H5Tenum.lo H5Tfields.lo H5Tfixed.lo H5Tfloat.lo H5Tinit.lo H5Tnative.lo H5Toffset.lo H5Topaque.lo H5Torder.lo H5Tpad.lo H5Tprecis.lo H5Tstrpad.lo H5Tvlen.lo H5TS.lo H5V.lo H5Z.lo H5Zdeflate.lo H5Zfletcher32.lo H5Zshuffle.lo H5Zszip.lo -Wl,--rpath -Wl,/home/ba01/u118/agarwa23/enzo1/src/yt/doc/yt-x86_64/lib64/ -Wl,--rpath -Wl,/home/ba01/u118/agarwa23/enzo1/src/yt/doc/yt-x86_64/lib/ -L/home/ba01/u118/agarwa23/enzo1/src/yt/doc/yt-x86_64/lib/ -L/home/ba01/u118/agarwa23/enzo1/src/yt/doc/yt-x86_64/lib64/ -lz -lm -Wl,-s -Wl,-soname -Wl,libhdf5.so.0 -o .libs/libhdf5.so.0.0.0 ld: /home/ba01/u118/agarwa23/enzo1/src/yt/doc/yt-x86_64/lib//libz.a(compress.o): relocation R_X86_64_32 against `a local symbol' can not be used when making a shared object; recompile with -fPIC /home/ba01/u118/agarwa23/enzo1/src/yt/doc/yt-x86_64/lib//libz.a: could not read symbols: Bad value make[1]: *** [libhdf5.la] Error 1 make[1]: Leaving directory `/autohome/u118/agarwa23/enzo1/src/yt/doc/yt-x86_64/src/hdf5-1.6.8/src' make: *** [install] Error 1
Hi Shankar,
So what's going on here is a couple things.
libz has been installed by the yt installer script, but for some reason has not been compiled as a shared library. The yt installer script does ask libz to compile as shared, but it seems that this request was not honored. Unfortunately, for the very reasons you see, we want it compiled as shared.
The second thing seems to be that it's using mpicc -- according to the Steele operating guide (http://www.rcac.purdue.edu/userinfo/resources/steele/userguide.cfm#compile_c... ) you probably have an mpi module loaded, and it might also be trying to compile with intel or pgi or one of those compiler sets. For now, the installation procedure for non-GNU is tricky and requires some manual intervention.
I think that you might be able to solve the problem by swapping out the pgi or intel modules for the gnu module (as described in that linked user guide), *unloading* any MPI modules that might be loaded (check via module list), wiping the entire yt-x86_64 directory and re-running the install script. You're early enough in that this shouldn't double up on a lot of time in compilation.
If this still causes problems, could you send the part of the log file dealing with the libz installation, as well as the HDF5? Additionally, if this solution works, I will add it as a note to the install_script.sh file as well as the documentation. This is the solution we had to use on Ranger for just this very reason.
Good luck, and let us know!
-Matt
On Sat, Nov 21, 2009 at 9:41 AM, Agarwal, Shankar sagarwal@ku.edu wrote:
Hi, I am trying to install yt on Steele. When I do "bash install_script.sh", it crashes with H5 warnings and libz.a error.
Any idea?
shankar KU Cosmology
Building dependencies for H5E.c H5E.c: in function `H5Eprint': H5E.c:467: warning: trace info was not inserted Building dependencies for H5F.c H5FDmulti.c: in function `H5Pset_fapl_split': H5FDmulti.c:291: warning: trace info was not inserted H5FDmulti.c: in function `H5Pset_fapl_multi': H5FDmulti.c:430: unable to parse `const char * const *memb_name' H5FDmulti.c: in function `H5Pget_fapl_multi': H5FDmulti.c:537: warning: trace info was not inserted H5FDmulti.c: in function `H5Pset_dxpl_multi': H5FDmulti.c:606: warning: trace info was not inserted H5FDmulti.c: in function `H5Pget_dxpl_multi': H5FDmulti.c:658: warning: trace info was not inserted Building dependencies for H5FDstdio.c H5FDstdio.c: in function `H5Pset_fapl_stdio': H5FDstdio.c:272: warning: trace info was not inserted make[1]: Leaving directory `/autohome/u118/agarwa23/enzo1/src/yt/doc/yt-x86_64/src/hdf5-1.6.8/src' make[1]: Entering directory `/autohome/u118/agarwa23/enzo1/src/yt/doc/yt-x86_64/src/hdf5-1.6.8/src' mkdir .libs mpicc -std=c99 -O3 -Wl,-s -I. -I. -UH5_DEBUG_API -DNDEBUG -D_LARGEFILE_SOURCE -D_LARGEFILE64_SOURCE -D_FILE_OFFSET_BITS=64 -D_POSIX_SOURCE -D_BSD_SOURCE -c H5Dcontig.c -fPIC -DPIC -o .libs/H5Dcontig.lo H5Dcontig.c(497): warning #556: a value of type "const H5D_dxpl_cache_t *" cannot be assigned to an entity of type "H5D_dxpl_cache_t *" H5D_BUILD_IO_INFO(&io_info,dset,dxpl_cache,dxpl_id,store); ^
mpicc -std=c99 -O3 -Wl,-s -I. -I. -UH5_DEBUG_API -DNDEBUG -D_LARGEFILE_SOURCE -D_LARGEFILE64_SOURCE -D_FILE_OFFSET_BITS=64 -D_POSIX_SOURCE - H5Dio.c(3208): warning #556: a value of type "const H5D_dxpl_cache_t *" cannot be assigned to an entity of type "H5D_dxpl_cache_t *" io_info->dxpl_cache=dxpl_cache; ^
mpicc -std=c99 -O3 -Wl,-s -I. -I. -UH5_DEBUG_API -DNDEBUG -D_LARGEFILE_SOURCE -D_LARGEFILE64_SOURCE -D_FILE_OFFSET_BITS=64 -D_POSIX_SOURCE -D_BSD_SOURCE -c H5FD.c -fPIC -DPIC -o .libs/H5FD.lo H5FD.c(3670): warning #810: conversion from "void *" to "herr_t={int}" may lose significant bits HGOTO_ERROR(H5E_VFL, H5E_UNSUPPORTED, NULL, "file driver has no `get_vfd_handle' method"); ^
mpicc -std=c99 -O3 -Wl,-s -I. -I. -UH5_DEBUG_API -DNDEBUG -D_LARGEFILE_SOURCE -D_LARGEFILE64_SOURCE -D_FILE_OFFSET_BITS=64 -D_POSIX_SOURCE -D_BSD_SOURCE -c H5FDmulti.c -fPIC -DPIC -o .libs/H5FDmulti.lo H5FDmulti.c(208): warning #266: function "strlen" declared implicitly if (NULL==(x=malloc(strlen(s)+1))) return NULL; ^
H5FDmulti.c(208): warning #167: argument of type "const char *" is incompatible with parameter of type "void *" if (NULL==(x=malloc(strlen(s)+1))) return NULL; ^
H5FDmulti.c(209): warning #266: function "strcpy" declared implicitly strcpy(x, s); ^
H5FDmulti.c(209): warning #167: argument of type "const char *" is incompatible with parameter of type "void *" strcpy(x, s); ^
H5FDmulti.c(1924): warning #266: function "memset" declared implicitly UNIQUE_MEMBERS(file->fa.memb_map, mt) { ^
mpicc -std=c99 -O3 -Wl,-s -I. -I. -UH5_DEBUG_API -DNDEBUG -D_LARGEFILE_SOURCE -D_LARGEFILE64_SOURCE -D_FILE_OFFSET_BITS=64 -D_POSIX_SOURCE -D_BSD_SOURCE -c H5FDstdio.c -fPIC -DPIC -o .libs/H5FDstdio.lo H5FDstdio.c(774): warning #266: function "memset" declared implicitly memset(buf, 0, size); ^
H5FDstdio.c(800): warning #266: function "memset" declared implicitly memset((unsigned char *)buf + size - nbytes, 0, nbytes); ^
H5FDstdio.c(815): warning #266: function "memset" declared implicitly memset((unsigned char *)buf + n, 0, (size - n)); ^
mpicc -std=c99 -O3 -Wl,-s -I. -I. -UH5_DEBUG_API -DNDEBUG -D_LARGEFILE_SOURCE -D_LARGEFILE64_SOURCE -D_FILE_OFFSET_BITS=64 -D_POSIX_SOURCE -D_BSD_SOURCE -c H5Gnode.c -fPIC -DPIC -o .libs/H5Gnode.lo H5Gnode.c(1671): warning #188: enumerated type mixed with another type udata->type = H5G_get_type(&(sn->entry[loc_idx]), dxpl_id); ^
mpicc -std=c99 -O3 -Wl,-s -I. -I. -UH5_DEBUG_API -DNDEBUG -D_LARGEFILE_SOURCE -D_LARGEFILE64_SOURCE -D_FILE_OFFSET_BITS=64 -D_POSIX_SOURCE -D_BSD_SOURCE -c H5Gstab.c -fPIC -DPIC -o .libs/H5Gstab.lo
mpicc -std=c99 -O3 -Wl,-s -I. -I. -UH5_DEBUG_API -DNDEBUG -D_LARGEFILE_SOURCE -D_LARGEFILE64_SOURCE -D_FILE_OFFSET_BITS=64 -D_POSIX_SOURCE -D_BSD_SOURCE -c H5Obogus.c -fPIC -DPIC -o .libs/H5Obogus.lo
mpicc -std=c99 -O3 -Wl,-s -I. -I. -UH5_DEBUG_API -DNDEBUG -D_LARGEFILE_SOURCE -D_LARGEFILE64_SOURCE -D_FILE_OFFSET_BITS=64 -D_POSIX_SOURCE -D_BSD_SOURCE -c H5Spoint.c -fPIC -DPIC -o .libs/H5Spoint.lo H5Spoint.c(879): warning #167: argument of type "const hsize_t={unsigned long long} **" is incompatible with parameter of type "const hsize_t={unsigned long long} *" if((ret_value=H5S_select_elements(space,op,num_elem,(const hsize_t **)coord))<0) ^
mpicc -std=c99 -O3 -Wl,-s -I. -I. -UH5_DEBUG_API -DNDEBUG -D_LARGEFILE_SOURCE -D_LARGEFILE64_SOURCE -D_FILE_OFFSET_BITS=64 -D_POSIX_SOURCE -D_BSD_SOURCE -c H5Spoint.c -o H5Spoint.o >/dev/null 2>&1 mv -f .libs/H5Spoint.lo H5Spoint.lo rm -f .libs/H5Sselect.lo mpicc -std=c99 -O3 -Wl,-s -I. -I. -UH5_DEBUG_API -DNDEBUG -D_LARGEFILE_SOURCE -D_LARGEFILE64_SOURCE -D_FILE_OFFSET_BITS=64 -D_POSIX_SOURCE -D_BSD_SOURCE -c H5Sselec mv -f .libs/H5Zszip.lo H5Zszip.lo rm -fr .libs/libhdf5.la .libs/libhdf5.* .libs/libhdf5.* mpicc -shared H5.lo H5A.lo H5AC.lo H5B.lo H5Bcache.lo H5C.lo H5D.lo H5Dcontig.lo H5Dcompact.lo H5Defl.lo H5Dio.lo H5Distore.lo H5Dmpio.lo H5Dselect.lo H5Dtest.lo H5E.lo H5F.lo H5Fdbg.lo H5Fmount.lo H5Fsfile.lo H5Fsuper.lo H5FD.lo H5FDcore.lo H5FDfamily.lo H5FDlog.lo H5FDmpi.lo H5FDmpio.lo H5FDmpiposix.lo H5FDmulti.lo H5FDsec2.lo H5FDstdio.lo H5FL.lo H5FO.lo H5FS.lo H5G.lo H5Gent.lo H5Gname.lo H5Gnode.lo H5Gstab.lo H5Gtest.lo H5Gtraverse.lo H5HG.lo H5HGdbg.lo H5HL.lo H5HLdbg.lo H5HP.lo H5I.lo H5MF.lo H5MM.lo H5O.lo H5Oattr.lo H5Obogus.lo H5Ocache.lo H5Ocont.lo H5Odtype.lo H5Oefl.lo H5Ofill.lo H5Olayout.lo H5Omtime.lo H5Oname.lo H5Onull.lo H5Opline.lo H5Osdspace.lo H5Oshared.lo H5Ostab.lo H5P.lo H5Pdcpl.lo H5Pdxpl.lo H5Pfapl.lo H5Pfcpl.lo H5Ptest.lo H5R.lo H5RC.lo H5RS.lo H5S.lo H5Sall.lo H5Shyper.lo H5Smpio.lo H5Snone.lo H5Spoint.lo H5Sselect.lo H5Stest.lo H5SL.lo H5ST.lo H5T.lo H5Tarray.lo H5Tbit.lo H5Tcommit.lo H5Tcompound.lo H5Tconv.lo H5Tcset.lo H5Tenum.lo H5Tfields.lo H5 Tfixed.lo H5Tfloat.lo H5Tinit.lo H5Tnative.lo H5Toffset.lo H5Topaque.lo H5Torder.lo H5Tpad.lo H5Tprecis.lo H5Tstrpad.lo H5Tvlen.lo H5TS.lo H5V.lo H5Z.lo H5Zdeflate.lo H5Zfletcher32.lo H5Zshuffle.lo H5Zszip.lo -Wl,--rpath -Wl,/home/ba01/u118/agarwa23/enzo1/src/yt/doc/yt-x86_64/lib64/ -Wl,--rpath -Wl,/home/ba01/u118/agarwa23/enzo1/src/yt/doc/yt-x86_64/lib/ -L/home/ba01/u118/agarwa23/enzo1/src/yt/doc/yt-x86_64/lib/ -L/home/ba01/u118/agarwa23/enzo1/src/yt/doc/yt-x86_64/lib64/ -lz -lm -Wl,-s -Wl,-soname -Wl,libhdf5.so.0 -o .libs/libhdf5.so.0.0.0 ld: /home/ba01/u118/agarwa23/enzo1/src/yt/doc/yt-x86_64/lib//libz.a(compress.o): relocation R_X86_64_32 against `a local symbol' can not be used when making a shared object; recompile with -fPIC /home/ba01/u118/agarwa23/enzo1/src/yt/doc/yt-x86_64/lib//libz.a: could not read symbols: Bad value make[1]: *** [libhdf5.la] Error 1 make[1]: Leaving directory `/autohome/u118/agarwa23/enzo1/src/yt/doc/yt-x86_64/src/hdf5-1.6.8/src' make: *** [install] Error 1
yt-users mailing list yt-users@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
Hi Matt, so i did...
[agarwa23@tg-steele ~]$ module list Currently Loaded Modulefiles: 1) intel64/11.1.038 2) mpich-intel64/11.1.038 3) java/1.6.0_14 4) mpich2-intel64/11.1.038 5) hdf5-1.8.3/intel-parallel
I can load gnu modules as well. But just wanted to know if it is necessary to remove intel and mpi modules ? I am currently running enzo on 24 processors. Will that get affected ? Also, will loading gnu affect my enzo simulation?
shankar
-----Original Message----- From: yt-users-bounces@lists.spacepope.org on behalf of Matthew Turk Sent: Sat 11/21/2009 11:49 AM To: Discussion of the yt analysis package Subject: Re: [yt-users] installing yt on Steele
Hi Shankar,
So what's going on here is a couple things.
libz has been installed by the yt installer script, but for some reason has not been compiled as a shared library. The yt installer script does ask libz to compile as shared, but it seems that this request was not honored. Unfortunately, for the very reasons you see, we want it compiled as shared.
The second thing seems to be that it's using mpicc -- according to the Steele operating guide (http://www.rcac.purdue.edu/userinfo/resources/steele/userguide.cfm#compile_c... ) you probably have an mpi module loaded, and it might also be trying to compile with intel or pgi or one of those compiler sets. For now, the installation procedure for non-GNU is tricky and requires some manual intervention.
I think that you might be able to solve the problem by swapping out the pgi or intel modules for the gnu module (as described in that linked user guide), *unloading* any MPI modules that might be loaded (check via module list), wiping the entire yt-x86_64 directory and re-running the install script. You're early enough in that this shouldn't double up on a lot of time in compilation.
If this still causes problems, could you send the part of the log file dealing with the libz installation, as well as the HDF5? Additionally, if this solution works, I will add it as a note to the install_script.sh file as well as the documentation. This is the solution we had to use on Ranger for just this very reason.
Good luck, and let us know!
-Matt
On Sat, Nov 21, 2009 at 9:41 AM, Agarwal, Shankar sagarwal@ku.edu wrote:
Hi, I am trying to install yt on Steele. When I do "bash install_script.sh", it crashes with H5 warnings and libz.a error.
Any idea?
shankar KU Cosmology
Building dependencies for H5E.c H5E.c: in function `H5Eprint': H5E.c:467: warning: trace info was not inserted Building dependencies for H5F.c H5FDmulti.c: in function `H5Pset_fapl_split': H5FDmulti.c:291: warning: trace info was not inserted H5FDmulti.c: in function `H5Pset_fapl_multi': H5FDmulti.c:430: unable to parse `const char * const *memb_name' H5FDmulti.c: in function `H5Pget_fapl_multi': H5FDmulti.c:537: warning: trace info was not inserted H5FDmulti.c: in function `H5Pset_dxpl_multi': H5FDmulti.c:606: warning: trace info was not inserted H5FDmulti.c: in function `H5Pget_dxpl_multi': H5FDmulti.c:658: warning: trace info was not inserted Building dependencies for H5FDstdio.c H5FDstdio.c: in function `H5Pset_fapl_stdio': H5FDstdio.c:272: warning: trace info was not inserted make[1]: Leaving directory `/autohome/u118/agarwa23/enzo1/src/yt/doc/yt-x86_64/src/hdf5-1.6.8/src' make[1]: Entering directory `/autohome/u118/agarwa23/enzo1/src/yt/doc/yt-x86_64/src/hdf5-1.6.8/src' mkdir .libs mpicc -std=c99 -O3 -Wl,-s -I. -I. -UH5_DEBUG_API -DNDEBUG -D_LARGEFILE_SOURCE -D_LARGEFILE64_SOURCE -D_FILE_OFFSET_BITS=64 -D_POSIX_SOURCE -D_BSD_SOURCE -c H5Dcontig.c -fPIC -DPIC -o .libs/H5Dcontig.lo H5Dcontig.c(497): warning #556: a value of type "const H5D_dxpl_cache_t *" cannot be assigned to an entity of type "H5D_dxpl_cache_t *" H5D_BUILD_IO_INFO(&io_info,dset,dxpl_cache,dxpl_id,store); ^
mpicc -std=c99 -O3 -Wl,-s -I. -I. -UH5_DEBUG_API -DNDEBUG -D_LARGEFILE_SOURCE -D_LARGEFILE64_SOURCE -D_FILE_OFFSET_BITS=64 -D_POSIX_SOURCE - H5Dio.c(3208): warning #556: a value of type "const H5D_dxpl_cache_t *" cannot be assigned to an entity of type "H5D_dxpl_cache_t *" io_info->dxpl_cache=dxpl_cache; ^
mpicc -std=c99 -O3 -Wl,-s -I. -I. -UH5_DEBUG_API -DNDEBUG -D_LARGEFILE_SOURCE -D_LARGEFILE64_SOURCE -D_FILE_OFFSET_BITS=64 -D_POSIX_SOURCE -D_BSD_SOURCE -c H5FD.c -fPIC -DPIC -o .libs/H5FD.lo H5FD.c(3670): warning #810: conversion from "void *" to "herr_t={int}" may lose significant bits HGOTO_ERROR(H5E_VFL, H5E_UNSUPPORTED, NULL, "file driver has no `get_vfd_handle' method"); ^
mpicc -std=c99 -O3 -Wl,-s -I. -I. -UH5_DEBUG_API -DNDEBUG -D_LARGEFILE_SOURCE -D_LARGEFILE64_SOURCE -D_FILE_OFFSET_BITS=64 -D_POSIX_SOURCE -D_BSD_SOURCE -c H5FDmulti.c -fPIC -DPIC -o .libs/H5FDmulti.lo H5FDmulti.c(208): warning #266: function "strlen" declared implicitly if (NULL==(x=malloc(strlen(s)+1))) return NULL; ^
H5FDmulti.c(208): warning #167: argument of type "const char *" is incompatible with parameter of type "void *" if (NULL==(x=malloc(strlen(s)+1))) return NULL; ^
H5FDmulti.c(209): warning #266: function "strcpy" declared implicitly strcpy(x, s); ^
H5FDmulti.c(209): warning #167: argument of type "const char *" is incompatible with parameter of type "void *" strcpy(x, s); ^
H5FDmulti.c(1924): warning #266: function "memset" declared implicitly UNIQUE_MEMBERS(file->fa.memb_map, mt) { ^
mpicc -std=c99 -O3 -Wl,-s -I. -I. -UH5_DEBUG_API -DNDEBUG -D_LARGEFILE_SOURCE -D_LARGEFILE64_SOURCE -D_FILE_OFFSET_BITS=64 -D_POSIX_SOURCE -D_BSD_SOURCE -c H5FDstdio.c -fPIC -DPIC -o .libs/H5FDstdio.lo H5FDstdio.c(774): warning #266: function "memset" declared implicitly memset(buf, 0, size); ^
H5FDstdio.c(800): warning #266: function "memset" declared implicitly memset((unsigned char *)buf + size - nbytes, 0, nbytes); ^
H5FDstdio.c(815): warning #266: function "memset" declared implicitly memset((unsigned char *)buf + n, 0, (size - n)); ^
mpicc -std=c99 -O3 -Wl,-s -I. -I. -UH5_DEBUG_API -DNDEBUG -D_LARGEFILE_SOURCE -D_LARGEFILE64_SOURCE -D_FILE_OFFSET_BITS=64 -D_POSIX_SOURCE -D_BSD_SOURCE -c H5Gnode.c -fPIC -DPIC -o .libs/H5Gnode.lo H5Gnode.c(1671): warning #188: enumerated type mixed with another type udata->type = H5G_get_type(&(sn->entry[loc_idx]), dxpl_id); ^
mpicc -std=c99 -O3 -Wl,-s -I. -I. -UH5_DEBUG_API -DNDEBUG -D_LARGEFILE_SOURCE -D_LARGEFILE64_SOURCE -D_FILE_OFFSET_BITS=64 -D_POSIX_SOURCE -D_BSD_SOURCE -c H5Gstab.c -fPIC -DPIC -o .libs/H5Gstab.lo
mpicc -std=c99 -O3 -Wl,-s -I. -I. -UH5_DEBUG_API -DNDEBUG -D_LARGEFILE_SOURCE -D_LARGEFILE64_SOURCE -D_FILE_OFFSET_BITS=64 -D_POSIX_SOURCE -D_BSD_SOURCE -c H5Obogus.c -fPIC -DPIC -o .libs/H5Obogus.lo
mpicc -std=c99 -O3 -Wl,-s -I. -I. -UH5_DEBUG_API -DNDEBUG -D_LARGEFILE_SOURCE -D_LARGEFILE64_SOURCE -D_FILE_OFFSET_BITS=64 -D_POSIX_SOURCE -D_BSD_SOURCE -c H5Spoint.c -fPIC -DPIC -o .libs/H5Spoint.lo H5Spoint.c(879): warning #167: argument of type "const hsize_t={unsigned long long} **" is incompatible with parameter of type "const hsize_t={unsigned long long} *" if((ret_value=H5S_select_elements(space,op,num_elem,(const hsize_t **)coord))<0) ^
mpicc -std=c99 -O3 -Wl,-s -I. -I. -UH5_DEBUG_API -DNDEBUG -D_LARGEFILE_SOURCE -D_LARGEFILE64_SOURCE -D_FILE_OFFSET_BITS=64 -D_POSIX_SOURCE -D_BSD_SOURCE -c H5Spoint.c -o H5Spoint.o >/dev/null 2>&1 mv -f .libs/H5Spoint.lo H5Spoint.lo rm -f .libs/H5Sselect.lo mpicc -std=c99 -O3 -Wl,-s -I. -I. -UH5_DEBUG_API -DNDEBUG -D_LARGEFILE_SOURCE -D_LARGEFILE64_SOURCE -D_FILE_OFFSET_BITS=64 -D_POSIX_SOURCE -D_BSD_SOURCE -c H5Sselec mv -f .libs/H5Zszip.lo H5Zszip.lo rm -fr .libs/libhdf5.la .libs/libhdf5.* .libs/libhdf5.* mpicc -shared H5.lo H5A.lo H5AC.lo H5B.lo H5Bcache.lo H5C.lo H5D.lo H5Dcontig.lo H5Dcompact.lo H5Defl.lo H5Dio.lo H5Distore.lo H5Dmpio.lo H5Dselect.lo H5Dtest.lo H5E.lo H5F.lo H5Fdbg.lo H5Fmount.lo H5Fsfile.lo H5Fsuper.lo H5FD.lo H5FDcore.lo H5FDfamily.lo H5FDlog.lo H5FDmpi.lo H5FDmpio.lo H5FDmpiposix.lo H5FDmulti.lo H5FDsec2.lo H5FDstdio.lo H5FL.lo H5FO.lo H5FS.lo H5G.lo H5Gent.lo H5Gname.lo H5Gnode.lo H5Gstab.lo H5Gtest.lo H5Gtraverse.lo H5HG.lo H5HGdbg.lo H5HL.lo H5HLdbg.lo H5HP.lo H5I.lo H5MF.lo H5MM.lo H5O.lo H5Oattr.lo H5Obogus.lo H5Ocache.lo H5Ocont.lo H5Odtype.lo H5Oefl.lo H5Ofill.lo H5Olayout.lo H5Omtime.lo H5Oname.lo H5Onull.lo H5Opline.lo H5Osdspace.lo H5Oshared.lo H5Ostab.lo H5P.lo H5Pdcpl.lo H5Pdxpl.lo H5Pfapl.lo H5Pfcpl.lo H5Ptest.lo H5R.lo H5RC.lo H5RS.lo H5S.lo H5Sall.lo H5Shyper.lo H5Smpio.lo H5Snone.lo H5Spoint.lo H5Sselect.lo H5Stest.lo H5SL.lo H5ST.lo H5T.lo H5Tarray.lo H5Tbit.lo H5Tcommit.lo H5Tcompound.lo H5Tconv.lo H5Tcset.lo H5Tenum.lo H5Tfields.lo H5 Tfixed.lo H5Tfloat.lo H5Tinit.lo H5Tnative.lo H5Toffset.lo H5Topaque.lo H5Torder.lo H5Tpad.lo H5Tprecis.lo H5Tstrpad.lo H5Tvlen.lo H5TS.lo H5V.lo H5Z.lo H5Zdeflate.lo H5Zfletcher32.lo H5Zshuffle.lo H5Zszip.lo -Wl,--rpath -Wl,/home/ba01/u118/agarwa23/enzo1/src/yt/doc/yt-x86_64/lib64/ -Wl,--rpath -Wl,/home/ba01/u118/agarwa23/enzo1/src/yt/doc/yt-x86_64/lib/ -L/home/ba01/u118/agarwa23/enzo1/src/yt/doc/yt-x86_64/lib/ -L/home/ba01/u118/agarwa23/enzo1/src/yt/doc/yt-x86_64/lib64/ -lz -lm -Wl,-s -Wl,-soname -Wl,libhdf5.so.0 -o .libs/libhdf5.so.0.0.0 ld: /home/ba01/u118/agarwa23/enzo1/src/yt/doc/yt-x86_64/lib//libz.a(compress.o): relocation R_X86_64_32 against `a local symbol' can not be used when making a shared object; recompile with -fPIC /home/ba01/u118/agarwa23/enzo1/src/yt/doc/yt-x86_64/lib//libz.a: could not read symbols: Bad value make[1]: *** [libhdf5.la] Error 1 make[1]: Leaving directory `/autohome/u118/agarwa23/enzo1/src/yt/doc/yt-x86_64/src/hdf5-1.6.8/src' make: *** [install] Error 1
yt-users mailing list yt-users@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
_______________________________________________ yt-users mailing list yt-users@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
Hi Shankar,
I can load gnu modules as well. But just wanted to know if it is necessary to remove intel and mpi modules ? I am currently running enzo on 24 processors. Will that get affected ? Also, will loading gnu affect my enzo simulation?
This is only for the compilation step -- any modules can be loaded when you run yt, as long as you follow the directions it spits out at the end of the compilation. For the compilation of yt, you will need to *unload* intel64, mpich-intel-64, mpich2-intel64, and possibly hdf5-1.8.3.
This won't affect an enzo simulation that is currently running, but my understanding is that any simulations you submit will reflect your current module state.
When you have finished the installation, you should feel free to re-load whatever modules you like.
Good luck!
-Matt
Hi Matt,
I got yt installed on Steele with wxpython=0 following your instructions. Two questions though. Towards the end, it asks you to...
PATH => /home/ba01/u118/agarwa23/enzo1/src/yt/doc/yt-x86_64/bin/ PYTHONPATH => /home/ba01/u118/agarwa23/enzo1/src/yt/doc/yt-x86_64/lib/python2.6/site-packages/ LD_LIBRARY_PATH => /home/ba01/u118/agarwa23/enzo1/src/yt/doc/yt-x86_64/lib/
I have a .soft file. And I modified it like...
# Common TeraGrid Software Stack Version 3 (CTSSv3) # # Please consult TeraGrid user documentation at # http://www.teragrid.org/userinfo/software/softenv.php # for information on customizing this file. #
PATH += /home/ba01/u118/agarwa23/enzo1/src/yt/doc/yt-x86_64/bin PYTHONPATH += /home/ba01/u118/agarwa23/enzo1/src/yt/doc/yt-x86_64/lib/python2.6/site-packages LD_LIBRARY_PATH += /home/ba01/u118/agarwa23/enzo1/src/yt/doc/yt-x86_64/lib
# TeraGrid wide basic software suite @teragrid-basic
# TeraGrid wide Globus 4 and Grid software suite @globus-4.0
# Platform recommended development software suite @teragrid-dev
Then, I did ...
[agarwa23@tg-steele ~]$ echo $PATH /apps/rhel5/jdk1.6.0_14/bin:/home/ba01/u118/agarwa23/enzo1/src/yt/doc/yt-x86_64/bin:/bin:/usr/bin:/usr/local/bin:/usr/X11R6/bin:/usr/site/rcd-admin/bin:/home/ba01/u118/agarwa23/bin/linux-rhel5-x86_64:/home/ba01/u118/agarwa23/bin:/usr/pbs/bin:/opt/moab/bin:/tg/tg-policy/bin:/opt/softenv/bin:/tg/tgusage-3.0-r0/bin:/apps/teragrid/tginfo-1.0.2-r1/bin:/tg/gx-map-0.5.3.2-r1/bin:/tg/globus-4.0.8-r2/bin:/tg/globus-4.0.8-r2/sbin:/opt/condor/wrappers:/opt/condor/bin:/opt/condor/sbin:/tg/tgcp-1.0.0-r2/bin:/tg/srb-client-3.4.1-r1/bin:/apps/teragrid/uberftp-1.27-r1/bin
[agarwa23@tg-steele ~]$ echo $PYTHONPATH /home/ba01/u118/agarwa23/enzo1/src/yt/doc/yt-x86_64/lib/python2.6/site-packages
[agarwa23@tg-steele ~]$ echo $LD_LIBRARY_PATH /home/ba01/u118/agarwa23/enzo1/src/yt/doc/yt-x86_64/lib:/tg/globus-4.0.8-r2/lib:/opt/condor/lib:/tg/srb-client-3.4.1-r1/lib:/tg/globus-4.0.1-r3.radon/lib
Does it matter whether yt part is appended or prepended or somewhere in the middle ?
Question 2: I tried to install yt again, with wxpython=1 and got this error...
checking for GTK+ - version >= 2.0.0... no *** Could not run GTK+ test program, checking why... *** The test program failed to compile or link. See the file config.log for the *** exact error that occured. This usually means GTK+ is incorrectly installed. configure: error: The development files for GTK+ were not found. For GTK+ 2, please ensure that pkg-config is in the path and that gtk+-2.0.pc is installed. For GTK+ 1.2 please check that gtk-config is in the path, and that the version is 1.2.3 or above. Also check that the libraries returned by 'pkg-config gtk+-2.0 --libs' or 'gtk-config --libs' are in the LD_LIBRARY_PATH or equivalent.
I was told by NCSA that wxpython=1 is needed for X-forwarding. Is that true ? Or I can just go back to wxpython=0 and install.
shankar
Hi Shankar,
Does it matter whether yt part is appended or prepended or somewhere in the middle ?
From what you've sent, it looks like it should be okay. But I'm not
entirely positive, you'll just have to test it out, I suppose. If it's not, it will fail with errors about not finding libraries, or GLIBC errors, or something along those lines. Hopefully it won't! :)
I was told by NCSA that wxpython=1 is needed for X-forwarding. Is that true ? Or I can just go back to wxpython=0 and install.
I'd say just go back to wxpython=0 -- it's only used by the wxAgg backend in Matplotlib and the yt GUI reason, but it looks like Steele is missing the necessary backend libraries, all of which are a huge pain to install. You should be fine without it.
Good luck!
-Matt
Ok. I switched wxpython=0 and installed yt successfully.
Now, ipython is acting up...
[agarwa23@tg-steele RD0002]$ ipython /home/ba01/u118/agarwa23/enzo1/src/yt/doc/yt-x86_64/lib/python2.6/site-packages/IPython/Magic.py:38: DeprecationWarning: the sets module is deprecated from sets import Set Python 2.6.1 (r261:67515, Nov 22 2009, 11:54:03) Type "copyright", "credits" or "license" for more information.
IPython 0.9.1 -- An enhanced Interactive Python. ? -> Introduction and overview of IPython's features. %quickref -> Quick reference. help -> Python's own help system. object? -> Details about 'object'. ?object also works, ?? prints more.
In [1]: pf=load("RedshiftOutput0002.cpu0001") --------------------------------------------------------------------------- NameError Traceback (most recent call last)
/usr/rmt_share/scratch95/a/agarwa23/0eV/512/box200/sim1/RD0002/<ipython console> in <module>()
NameError: name 'load' is not defined
In [2]:
shankar
Hi Shankar,
"load" is defined in the yt.mods namespace. "iyt" will preload the namespace for you, but if you just use ipython, you have to do something like:
from yt.mods import *
to populate the local namespace will all the yt objects.
-Matt
On Sun, Nov 22, 2009 at 9:17 AM, Agarwal, Shankar sagarwal@ku.edu wrote:
Ok. I switched wxpython=0 and installed yt successfully.
Now, ipython is acting up...
[agarwa23@tg-steele RD0002]$ ipython /home/ba01/u118/agarwa23/enzo1/src/yt/doc/yt-x86_64/lib/python2.6/site-packages/IPython/Magic.py:38: DeprecationWarning: the sets module is deprecated from sets import Set Python 2.6.1 (r261:67515, Nov 22 2009, 11:54:03) Type "copyright", "credits" or "license" for more information.
IPython 0.9.1 -- An enhanced Interactive Python. ? -> Introduction and overview of IPython's features. %quickref -> Quick reference. help -> Python's own help system. object? -> Details about 'object'. ?object also works, ?? prints more.
In [1]: pf=load("RedshiftOutput0002.cpu0001")
NameError Traceback (most recent call last)
/usr/rmt_share/scratch95/a/agarwa23/0eV/512/box200/sim1/RD0002/<ipython console> in <module>()
NameError: name 'load' is not defined
In [2]:
shankar
yt-users mailing list yt-users@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
participants (2)
-
Agarwal, Shankar -
Matthew Turk