[Image-SIG] Multi-threading and PIL

[Kevin Cazabon]

A couple years ago I did some work on enabling multi-threading in the  
PIL core library, with the help of Fredrik and others.  We  
successfully implemented it by releasing the GIL prior to starting  
many of the larger routines/functions in the C library, and re- 
acquiring it before handing back to the wrappers.  I can't remember  
exactly which functions we did this to in total, but it definitely  
included the Image.resize code and so forth.  On my dual-processor  
machine, I saw almost double performance on multi-threaded resizing  
operations - proving it worked.  However, we never touched the pixel  
access sections.

You could look at the resize code in the C module and use the same  
method to try it with the functions you use most.  The trick is just  
figuring out the right places to release and recapture the GIL.

Kevin.


On 06 Aug 2008, at 21:01, Sebastian Haase wrote:

> For pixel-by-pixel stuff you would probably be much (!) faster using  
> numpy.
>
> - Sebastian Haase
>
>
> On Wed, Aug 6, 2008 at 7:49 PM, Stani <spe.stani.be at gmail.com> wrote:
>> Op woensdag 06-08-2008 om 09:45 uur [tijdzone -0600], schreef  
>> Kristofor
>> Amundson:
>>
>>> My problem is that I see little to no improvement in my performance
>>> with the threading, even when I process the images on a machine  
>>> with 8
>>> CPU's. In fact, it only seems to utilize one CPU. Is this due to the
>>> GIL?
>> Probably so.
>>
>>> I am wondering if there is a way for me to properly thread this to
>>> improve my performance.
>> I use mostly subprocess pools to increase multi-core performance,  
>> but of
>> course this might be not evident to implement depending on the  
>> specifics
>> of your software. That is the only way to take advantage of multi- 
>> core
>> CPU (or of course stackless python).
>>
>> To speed up your existing code, you might also give psyco a try  
>> (which
>> is again in active development), profile your code and rewrite some
>> parts in cython or wait for PyPy to mature.
>>
>> Stani
>>
>> _______________________________________________
>> Image-SIG maillist  -  Image-SIG at python.org
>> http://mail.python.org/mailman/listinfo/image-sig
>>
> _______________________________________________
> Image-SIG maillist  -  Image-SIG at python.org
> http://mail.python.org/mailman/listinfo/image-sig