[Numpy-discussion] Multithreading support

[Chris Colbert]

Perhaps it's my inability to properly use openmp, but when working on
scikits.image on algorithms doing per-pixel manipulation with numpy arrays
(using Cython), i saw better performance using Python threads and releasing
the GIL than I did with openmp. I found the openmp overhead to be quite
large, and for the size the images i was working with (5 MP), the overhead
wasn't worth it.

I made a post to Cython-dev about it.

cheers,

Chris

On Sat, Feb 13, 2010 at 7:25 AM, René Dudfield <renesd at gmail.com> wrote:

> hi,
>
> see: http://numcorepy.blogspot.com/
>
> They see a benefit when working with large arrays.  Otherwise you are
> limited by memory - and the extra cores don't help with memory bandwidth.
>
> cheers,
>
>
>
>
> On Sat, Feb 13, 2010 at 2:20 PM, David Cournapeau <cournape at gmail.com>wrote:
>
>> On Sat, Feb 13, 2010 at 6:20 PM, Wolfgang Kerzendorf
>> <wkerzendorf at googlemail.com> wrote:
>> > Dear all,
>> >
>> > I don't know much about parallel programming so I don't know how easy it
>> is to do that: When doing simple arrray operations like adding two arrays or
>> adding a number to the array, is numpy able to put this on multiple cores? I
>> have tried it but it doesnt seem to do that. Is there a special multithread
>> implementation of numpy.
>>
>> Depending on your definition of simple operations, Numpy supports
>> multithreaded execution or not. For ufuncs (which is used for things
>> like adding two arrays together, etc...), there is no multithread
>> support.
>>
>> >
>> > IDL has this feature where it checks how many cores available and uses
>> them. This feature in numpy would make an already amazing package even
>> better.
>>
>> AFAIK, using multi-thread at the core level of NumPy has been tried
>> only once a few years ago, without much success (no significant
>> performance improvement). Maybe the approach was flawed in some ways.
>> Some people have suggested using OpenMP, but nobody has every produced
>> something significant AFAIK:
>>
>> http://mail.scipy.org/pipermail/numpy-discussion/2008-March/031897.html
>>
>> Note that Linear algebra operations can run in // depending on your
>> libraries. In particular, the dot function runs in // if your
>> blas/lapack does.
>>
>> cheers,
>>
>> David
>> _______________________________________________
>> NumPy-Discussion mailing list
>> NumPy-Discussion at scipy.org
>> http://mail.scipy.org/mailman/listinfo/numpy-discussion
>>
>
>
> _______________________________________________
> NumPy-Discussion mailing list
> NumPy-Discussion at scipy.org
> http://mail.scipy.org/mailman/listinfo/numpy-discussion
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20100214/476c4840/attachment.html>