Cluster Compile Issue
Hello yt-users, I'm currently attempting to build yt on Scinet GPC cluster, where the Python module can only be loaded alongside Intel and gcc combined. Compiling with Intel modules leads to mercurial compile error. I've attempted to build yt with only the gcc module active, and have achieved a successful build (though ), and yt sources without any errors. Attempting to run ipython or yt serve, however, gives me this error: ImportError: libimf.so: cannot open shared object file: No such file or directory Is there a workaround to this issue? Assistance would be greatly appreciated! Best, Charles
On 07/25/2013 02:27 AM, Chenchong Zhu wrote:
Hello yt-users,
Hi,
I'm currently attempting to build yt on Scinet GPC cluster, where the Python module can only be loaded alongside Intel and gcc combined. Compiling with Intel modules leads to mercurial compile error. I've attempted to build yt with only the gcc module active, and have achieved a successful build (though ), and yt sources without any errors. Attempting to run ipython or yt serve, however, gives me this error:
ImportError: libimf.so: cannot open shared object file: No such file or directory
libimf is part of intel compiler. Try adding proper paths to you LD_LIBRARY_PATH by doing one of the following prior to running ipython: 1) load intel module 2) source intel "activation" script by: source <INTEL_DIR>/bin/compilervars.sh intel64 3) append proper path manually: export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:<INTEL_DIR>/lib/intel64 Alternative (preferred one) is to use yt's installation script. It will compile python2.7 for you. As a result you won't need to rely on system one that introduces intel compiler as a direct dependency. Cheers, Kacper
Hi Kacper (and yt-users), Thanks for your response! I should have been more clear and said that I've only been using the yt install script to do everything, and was surprised that libimf.so wasn't compiled. The full text of my error is: (yt-x86_64)gpc-f101n084-$ ipython Traceback (most recent call last): File "/home/m/mhvk/czhu/yt-x86_64/bin/ipython", line 5, in <module> from IPython.frontend.terminal.ipapp import launch_new_instance File "/home/m/mhvk/czhu/yt-x86_64/lib/python2.7/site-packages/IPython/__init__.py", line 43, in <module> from .config.loader import Config File "/home/m/mhvk/czhu/yt-x86_64/lib/python2.7/site-packages/IPython/config/__init__.py", line 16, in <module> from .application import * File "/home/m/mhvk/czhu/yt-x86_64/lib/python2.7/site-packages/IPython/config/application.py", line 22, in <module> import logging File "/home/m/mhvk/czhu/yt-x86_64/lib/python2.7/logging/__init__.py", line 26, in <module> import sys, os, time, cStringIO, traceback, warnings, weakref ImportError: libimf.so: cannot open shared object file: No such file or directory Best, Charles P.S. I wrote a meaningless "(though )" in my last e-mail, which was supposed to report that installing ZeroMQ using install_script.bash gives the error "stream_engine.hpp(97): error #82: storage class is not first const static size_t greeting_size = 12" which I had to manually fix in the source to "static const", and then rerun install_script.bash.
Hi Charles, Did you have intel compilers loaded when you ran the install script? What's the output of 'module list'? You'll want to make sure that you have GNU compilers loaded and intel compilers unloaded before running the install script. If both are loaded strange things will happen. -Nathan On Thu, Jul 25, 2013 at 4:25 PM, Chenchong Zhu <cczhu@astro.utoronto.ca>wrote:
Hi Kacper (and yt-users),
Thanks for your response!
I should have been more clear and said that I've only been using the yt install script to do everything, and was surprised that libimf.so wasn't compiled. The full text of my error is:
(yt-x86_64)gpc-f101n084-$ ipython Traceback (most recent call last): File "/home/m/mhvk/czhu/yt-x86_64/bin/ipython", line 5, in <module> from IPython.frontend.terminal.ipapp import launch_new_instance File "/home/m/mhvk/czhu/yt-x86_64/lib/python2.7/site-packages/IPython/__init__.py", line 43, in <module> from .config.loader import Config File "/home/m/mhvk/czhu/yt-x86_64/lib/python2.7/site-packages/IPython/config/__init__.py", line 16, in <module> from .application import * File "/home/m/mhvk/czhu/yt-x86_64/lib/python2.7/site-packages/IPython/config/application.py", line 22, in <module> import logging File "/home/m/mhvk/czhu/yt-x86_64/lib/python2.7/logging/__init__.py", line 26, in <module> import sys, os, time, cStringIO, traceback, warnings, weakref ImportError: libimf.so: cannot open shared object file: No such file or directory
Best,
Charles
P.S. I wrote a meaningless "(though )" in my last e-mail, which was supposed to report that installing ZeroMQ using install_script.bash gives the error
"stream_engine.hpp(97): error #82: storage class is not first const static size_t greeting_size = 12"
which I had to manually fix in the source to "static const", and then rerun install_script.bash.
_______________________________________________ yt-users mailing list yt-users@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
Hi Nathan, I was running with only the gcc module compiled, and everything else unloaded. Module list's output is: Currently Loaded Modulefiles: 1) gcc/4.6.1 Charles On Thu, Jul 25, 2013 at 7:36 PM, Nathan Goldbaum <nathan12343@gmail.com>wrote:
Hi Charles,
Did you have intel compilers loaded when you ran the install script? What's the output of 'module list'?
You'll want to make sure that you have GNU compilers loaded and intel compilers unloaded before running the install script. If both are loaded strange things will happen.
-Nathan
On Thu, Jul 25, 2013 at 4:25 PM, Chenchong Zhu <cczhu@astro.utoronto.ca>wrote:
Hi Kacper (and yt-users),
Thanks for your response!
I should have been more clear and said that I've only been using the yt install script to do everything, and was surprised that libimf.so wasn't compiled. The full text of my error is:
(yt-x86_64)gpc-f101n084-$ ipython Traceback (most recent call last): File "/home/m/mhvk/czhu/yt-x86_64/bin/ipython", line 5, in <module> from IPython.frontend.terminal.ipapp import launch_new_instance File "/home/m/mhvk/czhu/yt-x86_64/lib/python2.7/site-packages/IPython/__init__.py", line 43, in <module> from .config.loader import Config File "/home/m/mhvk/czhu/yt-x86_64/lib/python2.7/site-packages/IPython/config/__init__.py", line 16, in <module> from .application import * File "/home/m/mhvk/czhu/yt-x86_64/lib/python2.7/site-packages/IPython/config/application.py", line 22, in <module> import logging File "/home/m/mhvk/czhu/yt-x86_64/lib/python2.7/logging/__init__.py", line 26, in <module> import sys, os, time, cStringIO, traceback, warnings, weakref ImportError: libimf.so: cannot open shared object file: No such file or directory
Best,
Charles
P.S. I wrote a meaningless "(though )" in my last e-mail, which was supposed to report that installing ZeroMQ using install_script.bash gives the error
"stream_engine.hpp(97): error #82: storage class is not first const static size_t greeting_size = 12"
which I had to manually fix in the source to "static const", and then rerun install_script.bash.
_______________________________________________ yt-users mailing list yt-users@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
_______________________________________________ yt-users mailing list yt-users@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
On 26.07.2013 01:25, Chenchong Zhu wrote:
Hi Kacper (and yt-users),
Thanks for your response!
I should have been more clear and said that I've only been using the yt install script to do everything, and was surprised that libimf.so wasn't compiled. The full text of my error is:
(yt-x86_64)gpc-f101n084-$ ipython Traceback (most recent call last): File "/home/m/mhvk/czhu/yt-x86_64/bin/ipython", line 5, in <module> from IPython.frontend.terminal.ipapp import launch_new_instance File "/home/m/mhvk/czhu/yt-x86_64/lib/python2.7/site-packages/IPython/__init__.py", line 43, in <module> from .config.loader import Config File "/home/m/mhvk/czhu/yt-x86_64/lib/python2.7/site-packages/IPython/config/__init__.py", line 16, in <module> from .application import * File "/home/m/mhvk/czhu/yt-x86_64/lib/python2.7/site-packages/IPython/config/application.py", line 22, in <module> import logging File "/home/m/mhvk/czhu/yt-x86_64/lib/python2.7/logging/__init__.py", line 26, in <module> import sys, os, time, cStringIO, traceback, warnings, weakref ImportError: libimf.so: cannot open shared object file: No such file or directory
Hi, it looks like the python itself was compiled with intel. You could verify that by running: cd /home/m/mhvk/czhu/yt-x86_64/lib/python2.7/lib-dynload readelf -d cStringIO.so -d | grep NEEDED See if libimf.so pops there. If that's the case I think you should remove the python2.7* in /home/m/mhvk/czhu/yt-x86_64/src and try installing it again via install_script.sh. Make sure you do *not* have any references to intel in LD_LIBRARY_PATH. Alternatively, you could try running ipython with intel module loaded. If you could paste output 'env' on http://paste.yt-project.org/ we could try to find out where the intel compiler lurks ;) Cheers, Kacper
Best,
Charles
P.S. I wrote a meaningless "(though )" in my last e-mail, which was supposed to report that installing ZeroMQ using install_script.bash gives the error
"stream_engine.hpp(97): error #82: storage class is not first const static size_t greeting_size = 12"
which I had to manually fix in the source to "static const", and then rerun install_script.bash.
_______________________________________________ yt-users mailing list yt-users@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
Hi, I've tried a fresh install with install_script.sh with only gcc loaded; my LD_LIBRARY_PATH only contains /scinet/gpc/compilers/gcc-4.6.1/requiredlibs/lib:/scinet/gpc/compilers/gcc-4.6.1/lib:/scinet/gpc/compilers/gcc-4.6.1/lib64 After building, I still get the same missing libimf.so error. An ls of the directory you specified shows that it actually doesn't exist. My suspicion is the associated python module somehow compiled incorrectly, but the only error I encountered during installation was that ZeroMQ didn't build properly, which was solved by fixing a line in the ZeroMQ source, then restarting install_script.bash. This doesn't have anything to do with my current problem, right? Running ipython with intel loaded has its own issues, but if that's actually easier than debugging this I can give it a go. I've posted the output of env to past.yt-project.org. Best, Charles On Fri, Jul 26, 2013 at 2:49 AM, Kacper Kowalik <xarthisius.kk@gmail.com>wrote:
On 26.07.2013 01:25, Chenchong Zhu wrote:
Hi Kacper (and yt-users),
Thanks for your response!
I should have been more clear and said that I've only been using the yt install script to do everything, and was surprised that libimf.so wasn't compiled. The full text of my error is:
(yt-x86_64)gpc-f101n084-$ ipython Traceback (most recent call last): File "/home/m/mhvk/czhu/yt-x86_64/bin/ipython", line 5, in <module> from IPython.frontend.terminal.ipapp import launch_new_instance File
"/home/m/mhvk/czhu/yt-x86_64/lib/python2.7/site-packages/IPython/__init__.py",
line 43, in <module> from .config.loader import Config File
"/home/m/mhvk/czhu/yt-x86_64/lib/python2.7/site-packages/IPython/config/__init__.py",
line 16, in <module> from .application import * File
"/home/m/mhvk/czhu/yt-x86_64/lib/python2.7/site-packages/IPython/config/application.py",
line 22, in <module> import logging File "/home/m/mhvk/czhu/yt-x86_64/lib/python2.7/logging/__init__.py", line 26, in <module> import sys, os, time, cStringIO, traceback, warnings, weakref ImportError: libimf.so: cannot open shared object file: No such file or directory
Hi, it looks like the python itself was compiled with intel. You could verify that by running:
cd /home/m/mhvk/czhu/yt-x86_64/lib/python2.7/lib-dynload readelf -d cStringIO.so -d | grep NEEDED
See if libimf.so pops there. If that's the case I think you should remove the python2.7* in /home/m/mhvk/czhu/yt-x86_64/src and try installing it again via install_script.sh. Make sure you do *not* have any references to intel in LD_LIBRARY_PATH.
Alternatively, you could try running ipython with intel module loaded.
If you could paste output 'env' on http://paste.yt-project.org/ we could try to find out where the intel compiler lurks ;)
Cheers, Kacper
Best,
Charles
P.S. I wrote a meaningless "(though )" in my last e-mail, which was supposed to report that installing ZeroMQ using install_script.bash gives the error
"stream_engine.hpp(97): error #82: storage class is not first const static size_t greeting_size = 12"
which I had to manually fix in the source to "static const", and then rerun install_script.bash.
_______________________________________________ yt-users mailing list yt-users@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
_______________________________________________ yt-users mailing list yt-users@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
participants (3)
-
Chenchong Zhu -
Kacper Kowalik -
Nathan Goldbaum