[Neuroimaging] parallel computation of bundle_distances_mam/mdf ?

Wed Dec 14 13:50:01 EST 2016

I would suggest before going to homebrew etc. to send an e-mail to Xcode
asking for some guidelines. Because it seems clang now comes with openmp
supported maybe all we need to do is update the correct headers.

On Wed, Dec 14, 2016 at 1:45 PM Ariel Rokem <arokem at gmail.com> wrote:

> Hi Paolo,
>
> I can partially reproduce your findings (also on a MAC, OS 10.12.1 in my
> case):
>
> On Wed, Dec 14, 2016 at 9:21 AM, Paolo Avesani <avesani at fbk.eu> wrote:
>
> Just for reference I tried two ways on Mac OS 10.11.5:
> 1. compile with clang
> 2.  compile with gcc provided by anaconda
> In both cases compilation failed.
>
> 1. compile with clang
> ----------------------------
> $ python setup.py build_ext --inplace
> Compiling test.pyx because it changed.
> Cythonizing test.pyx
> running build_ext
> building 'test' extension
> creating build
> creating build/temp.macosx-10.6-x86_64-2.7
> gcc -fno-strict-aliasing -I/Users/paolo/Software/miniconda/include -arch
> x86_64 -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes
> -I/Users/paolo/Software/miniconda/include/python2.7 -c test.c -o
> build/temp.macosx-10.6-x86_64-2.7/test.o -fopenmp
> clang: error: unsupported option '-fopenmp'
> error: command 'gcc' failed with exit status 1
>
>
> I get the "-fopenmp" error, but compilation then proceeds without a
> problem. I don't think that openmp is used (does anyone know how I would
> know for sure?), but that's fine for my needs. As Stephan has reported, you
> can get an omp-enabled gcc from homebrew, but I don't use that, because
> when I do any real data analysis, I do it on AWS ubuntu machines anyway.
>
> I am not sure why your clan exits with status 1, but I don't get that on
> my machine. What do you see when you run `gcc --version`?
>
>
> 2.  compile with gcc provided by anaconda
> ---------------------------------------------------------
> $ python setup.py build_ext --inplace
> Compiling test.pyx because it changed.
> Cythonizing test.pyx
> running build_ext
> building 'test' extension
> gcc -fno-strict-aliasing -I/Users/paolo/Software/miniconda/include -arch
> x86_64 -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes
> -I/Users/paolo/Software/miniconda/include/python2.7 -c test.c -o
> build/temp.macosx-10.6-x86_64-2.7/test.o -fopenmp
> In file included from
> /Users/paolo/Software/miniconda/lib/gcc/x86_64-apple-darwin11.4.2/4.8.5/include-fixed/syslimits.h:7:0,
>                  from
> /Users/paolo/Software/miniconda/lib/gcc/x86_64-apple-darwin11.4.2/4.8.5/include-fixed/limits.h:34,
>                  from
> /Users/paolo/Software/miniconda/include/python2.7/Python.h:19,
>                  from test.c:16:
> /Users/paolo/Software/miniconda/lib/gcc/x86_64-apple-darwin11.4.2/4.8.5/include-fixed/limits.h:168:61:
> fatal error: limits.h: No such file or directory
>  #include_next <limits.h>  /* recurse down to the real one */
> compilation terminated.
> error: command 'gcc' failed with exit status 1
>
>
> Yep. I see a similar error with Anaconda gcc as well.
>
> Cheers,
>
> Ariel
>
>
>
>
> On Wed, Dec 14, 2016 at 4:51 PM, Eleftherios Garyfallidis <
> elef at indiana.edu> wrote:
>
>
> Hi Emanuele,
>
> My understanding is that openmp was only temporarily not available when
> clang replaced gcc in osx.
>
> So, I would suggest to go ahead with openmp. Any current installation
> issues are only temporarily for osx.
> Openmp gives us a lot of capability to play with shared memory and it is a
> standard that will be around
> for very long time. Also, the great integration in cython makes the
> algorithms really easy to read.
> So, especially for this project my recommendation is to use openmp rather
> than multiprocessing. All the way! :)
>
> I am CC'ing Stephan who wrote the instructions for osx. I am sure he can
> help you with this. I would also suggest
> to check if xcode provides any new guis for enabling openmp. I remember
> there was something for that.
>
> Laterz!
> Eleftherios
>
>
>
>
> On Wed, Dec 14, 2016 at 6:29 AM Emanuele Olivetti <olivetti at fbk.eu> wrote:
>
> Hi Eleftherios,
>
> Thank you for pointing me to the MDF example. From what I see the Cython
> syntax is not complex, which is good.
>
> My only concern is the availability of OpenMP in the systems where DiPy is
> used. On a reasonably recent GNU/Linux machine it seems straightforward to
> have libgomp and the proper version of gcc. On other systems - say OSX -
> the situation is less clear to me. According to what I read here
>   http://nipy.org/dipy/installation.html#openmp-with-osx
> the OSX installation steps are not meant for standard end users. Are those
> instructions updated?
> As a test of that, we've just tried to skip the steps described above and
> instead to install gcc with conda on OSX ("conda install gcc"). In the
> process, conda installed the recent gcc-4.8 with libgomp, which seems good
> news. Unfortunately, when we tried to compile a simple example of Cython
> code using parallelization (see below), the process failed (fatal error:
> limits.h : no such file or directory)...
>
> For the reasons above, I am wondering whether the very simple solution of
> using the "multiprocessing" module, available from the standard Python
> library, may be an acceptable first step towards the more efficient
> multithreading of Cython/libgomp. With "multiprocessing", there is no extra
> dependency on libgomp, or recent gcc or else. Moreover, multiprocessing
> does not require to have Cython code, because it works on plain Python too.
>
> Best,
>
> Emanuele
>
> ---- test.pyx ----
> from cython import parallel
> from libc.stdio cimport printf
>
> def test_func():
>     cdef int thread_id = -1
>     with nogil, parallel.parallel(num_threads=10):
>         thread_id = parallel.threadid()
>         printf("Thread ID: %d\n", thread_id)
> -----
>
> ----- setup.py -----
> from distutils.core import setup, Extension
> from Cython.Build import cythonize
>
> extensions = [Extension(
>                 "test",
>                 sources=["test.pyx"],
>                 extra_compile_args=["-fopenmp"],
>                 extra_link_args=["-fopenmp"]
>             )]
>
> setup(
>     ext_modules = cythonize(extensions)
> )
> ----
> python setup.py build_ext --inplace
>
> On Tue, Dec 13, 2016 at 11:17 PM, Eleftherios Garyfallidis <
> elef at indiana.edu> wrote:
>
> Hi Emanuele,
>
> Here is an example of how we calculated the distance matrix in parallel
> (for the MDF) using OpenMP
> https://github.com/nipy/dipy/blob/master/dipy/align/bundlemin.pyx
>
> You can just add another function that does the same using mam. It should
> be really easy to implement as we have
> already done it for the MDF for speeding up SLR.
>
> Then we need to update the bundle_distances* functions to use the parallel
> versions.
>
> I'll be happy to help you with this. Let's try to schedule some time to
> look at this together.
>
> Best regards,
> Eleftherios
>
>
> On Mon, Dec 12, 2016 at 11:16 AM Emanuele Olivetti <olivetti at fbk.eu>
> wrote:
>
> Hi,
>
> I usually compute the distance matrix between two lists of streamlines
> using bundle_distances_mam() or bundle_distances_mdf(). When the lists are
> large, it is convenient and easy to exploit the multiple cores of the CPU
> because such computation is intrinsically (embarassingly) parallel. At the
> moment I'm doing it through the multiprocessing or the joblib modules,
> because I cannot find a way directly from DiPy, at least according to what
> I see in dipy/tracking/distances.pyx . But consider that I am not
> proficient in cython.parallel.
>
> Is there a preferable way to perform such parallel computation? I plan to
> prepare a pull request in future and I'd like to be on the right track.
>
> Best,
>
> Emanuele
>
> _______________________________________________
> Neuroimaging mailing list
> Neuroimaging at python.org
> https://mail.python.org/mailman/listinfo/neuroimaging
>
>
> _______________________________________________
> Neuroimaging mailing list
> Neuroimaging at python.org
> https://mail.python.org/mailman/listinfo/neuroimaging
>
>
> _______________________________________________
> Neuroimaging mailing list
> Neuroimaging at python.org
> https://mail.python.org/mailman/listinfo/neuroimaging
>
>
> _______________________________________________
> Neuroimaging mailing list
> Neuroimaging at python.org
> https://mail.python.org/mailman/listinfo/neuroimaging
>
>
>
>
> --
> -------------------------------------------------------
> Paolo Avesani
> Fondazione Bruno Kessler
> via Sommarive 18,
> 38050 Povo (TN) - I
> phone:   +39 0461 314336 <+39%200461%20314336>
> fax:        +39 0461 302040 <+39%200461%20302040>
> email:     avesani at fbk.eu
> web:       avesani.fbk.eu
>
>
> _______________________________________________
> Neuroimaging mailing list
> Neuroimaging at python.org
> https://mail.python.org/mailman/listinfo/neuroimaging
>
> _______________________________________________
> Neuroimaging mailing list
> Neuroimaging at python.org
> https://mail.python.org/mailman/listinfo/neuroimaging
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/neuroimaging/attachments/20161214/28e1ce16/attachment-0001.html>