Hi y'all, I've spent the morning trying to merge the copy of OpenMP HOP I have with the version in trunk. Although HOP is not the bottleneck when running it, OpenMP HOP does have a concrete speedup, at 8 processors it's better than 5x faster. I wrapped everything in #ifdefs OPENMP and it runs identically as before if it's not turned on. With OpenMP turned on, it compiles, but it gives me this error when I run it on Ranger: from yt.mods import * File "/share/home/00649/tg457850/yt/lib/python2.6/site-packages/yt-1.5dev-py2.6-linux-x86_64.egg/yt/mods.py", line 32, in <module> import yt.lagos as lagos File "/share/home/00649/tg457850/yt/lib/python2.6/site-packages/yt-1.5dev-py2.6-linux-x86_64.egg/yt/lagos/__init__.py", line 105, in <module> from HaloFinding import * File "/share/home/00649/tg457850/yt/lib/python2.6/site-packages/yt-1.5dev-py2.6-linux-x86_64.egg/yt/lagos/HaloFinding.py", line 29, in <module> from yt.lagos.hop.EnzoHop import RunHOP File "/share/home/00649/tg457850/yt/lib/python2.6/site-packages/yt-1.5dev-py2.6-linux-x86_64.egg/yt/lagos/hop/__init__.py", line 3, in <module> from EnzoHop import * ImportError: /share/home/00649/tg457850/yt/lib/python2.6/site-packages/yt-1.5dev-py2.6-linux-x86_64.egg/yt/lagos/hop/EnzoHop.so: undefined symbol: omp_unset_lock I'm using gcc, and the relavent flags are "-fopenmp -DOPENMP". There are other omp calls that it isn't complaining about. I'm wondering if any of you have any thoughts. There's the very real chance this won't work, I can find nothing of substance online about doing this kind of thing. Thanks! _______________________________________________________ sskory@physics.ucsd.edu o__ Stephen Skory http://physics.ucsd.edu/~sskory/ _.>/ _Graduate Student ________________________________(_)_\(_)_______________
There has been quite a bit of discussion of using OpenMP applications inside python extensions, mostly on the numpy mailing list. You should be able to modify the yt/lagos/hop/setup.py file to set up the compiler options as necessary. Using a conditional you can modify the Extension options. The options are described in both the distutils and numpy.distutils documentation. -Matt On Fri, May 8, 2009 at 2:01 PM, Stephen Skory <stephenskory@yahoo.com> wrote:
Hi y'all,
I've spent the morning trying to merge the copy of OpenMP HOP I have with the version in trunk. Although HOP is not the bottleneck when running it, OpenMP HOP does have a concrete speedup, at 8 processors it's better than 5x faster. I wrapped everything in #ifdefs OPENMP and it runs identically as before if it's not turned on. With OpenMP turned on, it compiles, but it gives me this error when I run it on Ranger:
from yt.mods import * File "/share/home/00649/tg457850/yt/lib/python2.6/site-packages/yt-1.5dev-py2.6-linux-x86_64.egg/yt/mods.py", line 32, in <module> import yt.lagos as lagos File "/share/home/00649/tg457850/yt/lib/python2.6/site-packages/yt-1.5dev-py2.6-linux-x86_64.egg/yt/lagos/__init__.py", line 105, in <module> from HaloFinding import * File "/share/home/00649/tg457850/yt/lib/python2.6/site-packages/yt-1.5dev-py2.6-linux-x86_64.egg/yt/lagos/HaloFinding.py", line 29, in <module> from yt.lagos.hop.EnzoHop import RunHOP File "/share/home/00649/tg457850/yt/lib/python2.6/site-packages/yt-1.5dev-py2.6-linux-x86_64.egg/yt/lagos/hop/__init__.py", line 3, in <module> from EnzoHop import * ImportError: /share/home/00649/tg457850/yt/lib/python2.6/site-packages/yt-1.5dev-py2.6-linux-x86_64.egg/yt/lagos/hop/EnzoHop.so: undefined symbol: omp_unset_lock
I'm using gcc, and the relavent flags are "-fopenmp -DOPENMP". There are other omp calls that it isn't complaining about. I'm wondering if any of you have any thoughts. There's the very real chance this won't work, I can find nothing of substance online about doing this kind of thing.
Thanks!
_______________________________________________________ sskory@physics.ucsd.edu o__ Stephen Skory http://physics.ucsd.edu/~sskory/ _.>/ _Graduate Student ________________________________(_)_\(_)_______________
_______________________________________________ Yt-dev mailing list Yt-dev@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
Matt,
You should be able to modify the yt/lagos/hop/setup.py file to set up the compiler options as necessary.
That's actually not my problem right now. I have distutils building with the needed flags. I think I'm missing something else. Here's what I have right now: config.add_extension("EnzoHop", sources= ["EnzoHop.c", "hop_hop.c", "hop_kd.c", "hop_regroup.c", "hop_slice.c", "hop_smooth.c", "para_median.c",], libraries=["m"], extra_compile_args=["-fopenmp -DOPENMP"],) If I add "gomp" to libraries, instead of the omp_unset_lock error, I get: libgomp.so.1: cannot open shared object file: No such file or directory That file is in /usr/lib which is in my LD_LIBRARY_PATH (I'm running on a login node). I searched the numpy discussion list and all I can find about OpenMP are messsage threads on whether or not to use OMP, not how to build modules. _______________________________________________________ sskory@physics.ucsd.edu o__ Stephen Skory http://physics.ucsd.edu/~sskory/ _.>/ _Graduate Student ________________________________(_)_\(_)_______________
I searched the numpy discussion list and all I can find about OpenMP are messsage threads on whether or not to use OMP, not how to build modules.
The rest of the internet isn't very useful either. Do you guys have any ideas? _______________________________________________________ sskory@physics.ucsd.edu o__ Stephen Skory http://physics.ucsd.edu/~sskory/ _.>/ _Graduate Student ________________________________(_)_\(_)_______________
Hi Stephen, -DOPENMP should not be in the extra_args. It should be: define_macros=[("OPENMP",True)], Furthermore, are you sure you need to wrap OpenMP stuff in #defines? Pragmas should pass through the preprocessor without any adjustment. As for the linkage issues, I'd say play with the output of ldd. Do you need to explicitly add "gomp" to the libraries, or is GCC supposed to handle that itself? -Matt On Fri, May 8, 2009 at 4:08 PM, Stephen Skory <stephenskory@yahoo.com> wrote:
Matt,
You should be able to modify the yt/lagos/hop/setup.py file to set up the compiler options as necessary.
That's actually not my problem right now. I have distutils building with the needed flags. I think I'm missing something else. Here's what I have right now:
config.add_extension("EnzoHop", sources= ["EnzoHop.c", "hop_hop.c", "hop_kd.c", "hop_regroup.c", "hop_slice.c", "hop_smooth.c", "para_median.c",], libraries=["m"], extra_compile_args=["-fopenmp -DOPENMP"],)
If I add "gomp" to libraries, instead of the omp_unset_lock error, I get:
libgomp.so.1: cannot open shared object file: No such file or directory
That file is in /usr/lib which is in my LD_LIBRARY_PATH (I'm running on a login node).
I searched the numpy discussion list and all I can find about OpenMP are messsage threads on whether or not to use OMP, not how to build modules.
_______________________________________________________ sskory@physics.ucsd.edu o__ Stephen Skory http://physics.ucsd.edu/~sskory/ _.>/ _Graduate Student ________________________________(_)_\(_)_______________ _______________________________________________ Yt-dev mailing list Yt-dev@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
Matt,
-DOPENMP should not be in the extra_args. It should be:
define_macros=[("OPENMP",True)],
Thanks.
Furthermore, are you sure you need to wrap OpenMP stuff in #defines? Pragmas should pass through the preprocessor without any adjustment.
The OpenMP version of HOP changes a fair bit of the code. It doesn't only put pragmas on for loops. In particular, it uses locks to parallelize the regrouping part of the code, so different threads don't write to the same part of the regroup hash table. Because I wanted to be able to switch back and forth, #ifdefs seemed like the way to go.
As for the linkage issues, I'd say play with the output of ldd. Do you need to explicitly add "gomp" to the libraries, or is GCC supposed to handle that itself?
If I compile a test program on the command line, 'gcc -fopenmp -o test test.c' works without -lgomp. Adding -lgomp doesn't break anything in that case. I agree that gcc is supposed to handle it itself, but the fact that the error message changes with and without it is suspicious. _______________________________________________________ sskory@physics.ucsd.edu o__ Stephen Skory http://physics.ucsd.edu/~sskory/ _.>/ _Graduate Student ________________________________(_)_\(_)_______________
participants (2)
-
Matthew Turk -
Stephen Skory