Numpy vs PIL in image statistics
Testing the PIL vs numpy in calculating the mean value of each color channel of an image I timed the following. impil = Image.open("10.tif") imnum = asarray(impil) #in PIL for i in range(1,10): stats = ImageStat.Stat(impil) stats.mean # for numpy for i in range(1,10): imnum.reshape(-1,3).mean(axis=0) The image I tested initially is 2000x2000 RGB tif ~11mb in size. I set a timer in each for loop and measured the performance of numpy 7 times slower than PIL. When I did the the same with an 10x10 RGB tif and with 1000 cycles in for, numpy was 25 times faster than PIL. Why is that? Does mean or reshape, make a copy?
On Wed, May 27, 2009 at 10:33, cp <lubensch.proletariat.inc@gmail.com> wrote:
Testing the PIL vs numpy in calculating the mean value of each color channel of an image I timed the following.
impil = Image.open("10.tif") imnum = asarray(impil)
#in PIL for i in range(1,10): stats = ImageStat.Stat(impil) stats.mean
# for numpy for i in range(1,10): imnum.reshape(-1,3).mean(axis=0)
The image I tested initially is 2000x2000 RGB tif ~11mb in size. I set a timer in each for loop and measured the performance of numpy 7 times slower than PIL. When I did the the same with an 10x10 RGB tif and with 1000 cycles in for, numpy was 25 times faster than PIL. Why is that? Does mean or reshape, make a copy?
reshape() might if the array wasn't contiguous. -- Robert Kern "I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth." -- Umberto Eco
cp wrote:
The image I tested initially is 2000x2000 RGB tif ~11mb in size.
I continued testing, with the initial PIL approach and 3 alternative numpy scripts:
#Script 1 - indexing for i in range(10): imarr[:,:,0].mean() imarr[:,:,1].mean() imarr[:,:,2].mean()
#Script 2 - slicing for i in range(10): imarr[:,:,0:1].mean() imarr[:,:,1:2].mean() imarr[:,:,2:3].mean()
#Script 3 - reshape for i in range(10): imarr.reshape(-1,3).mean(axis=0)
#Script 4 - PIL for i in range(10): stats = ImageStat.stat(img) stats.mean
After profiling the four scripts separately I got the following script 1: 5.432sec script 2: 10.234sec script 3: 4.980sec script 4: 0.741sec
when I profiled scripts 1-3 without calculating the mean, I got similar results of about 0.45sec for 1000 cycles, meaning that even if there is a copy involved the time required is only a small fraction of the whole procedure.Getting back to my initial statement I cannot explain why PIL is very fast in calculations for whole images, but very slow in calculations of small sub-images.
I don't know anything about PIL and its implementation, but I would not be surprised if the cost is mostly accessing items which are not contiguous in memory and bounds checking ( to check where you are in the subimage). Conditional inside loops often kills performances, and the actual computation (one addition/item for naive average implementation) is negligeable in this case. cheers, David
The image I tested initially is 2000x2000 RGB tif ~11mb in size. I continued testing, with the initial PIL approach and 3 alternative numpy scripts:
#Script 1 - indexing for i in range(10): imarr[:,:,0].mean() imarr[:,:,1].mean() imarr[:,:,2].mean() #Script 2 - slicing for i in range(10): imarr[:,:,0:1].mean() imarr[:,:,1:2].mean() imarr[:,:,2:3].mean() #Script 3 - reshape for i in range(10): imarr.reshape(-1,3).mean(axis=0) #Script 4 - PIL for i in range(10): stats = ImageStat.stat(img) stats.mean After profiling the four scripts separately I got the following script 1: 5.432sec script 2: 10.234sec script 3: 4.980sec script 4: 0.741sec when I profiled scripts 1-3 without calculating the mean, I got similar results of about 0.45sec for 1000 cycles, meaning that even if there is a copy involved the time required is only a small fraction of the whole procedure.Getting back to my initial statement I cannot explain why PIL is very fast in calculations for whole images, but very slow in calculations of small sub-images.
I don't know anything about PIL and its implementation, but I would not be surprised if the cost is mostly accessing items which are not contiguous in memory and bounds checking ( to check where you are in the subimage). Conditional inside loops often kills performances, and the actual computation (one addition/item for naive average implementation) is negligeable in this case.
This would definitely be the case in sub-images. However, coming back to my original question, how do you explain that although PIL is extremely fast in large images (2000x2000), numpy is much faster when it comes down to very small images (I tested with 10x10 image files)?
cp wrote:
I don't know anything about PIL and its implementation, but I would not be surprised if the cost is mostly accessing items which are not contiguous in memory and bounds checking ( to check where you are in the subimage). Conditional inside loops often kills performances, and the actual computation (one addition/item for naive average implementation) is negligeable in this case.
This would definitely be the case in sub-images. However, coming back to my original question, how do you explain that although PIL is extremely fast in large images (2000x2000), numpy is much faster when it comes down to very small images (I tested with 10x10 image files)?
I've heard rumors that PIL stores it's images as pointers-to-rows (or cols), i.e. to access a new row you need to dereference a pointer. NumPy on the other hand always stores its memory in blocks. When N grows larger, the N pointer lookups needed in PIL doesn't matter, but they do for low N. Dag Sverre
participants (4)
-
cp -
Dag Sverre Seljebotn -
David Cournapeau -
Robert Kern