[Numpy-discussion] performance of nd_image affine_transform

[Peter Verveer]

On Dec 6, 2005, at 18:24, Sebastian Haase wrote:

> (Peter, I hope you meant to reply to the mailing list...)

Yes, I did :-)

> Hi Peter,
> Does that mean that the performance hit happens "at the lowest  
> level", i.e.
> there is lot's of extra stuff in the in the innermost c loop that  
> iterates
> over the output array ?

Yes, I would say so.

> Or is there just so much setup and cleanup code that
> makes the overall function slow ?

Some of that may be a problem too, for instance, the function does  
malloc and free some temporary data, so that would add if you apply  
the function repeatedly.

> How could I profile C extensions anyway - pointers/hints are  
> appreciated !

Not sure how you would go about that. I guess optimizing would  
require a rewrite for the specific cases that you like to optimize.

>
> Thanks for your nd_image (In case that wasn't clear - I like it a  
> lot !)

Glad to hear that!

>
> Sebastian Haase
>
>
>
> On Tuesday 06 December 2005 01:32, you wrote:
>> Hi Sebastian,
>>
>> The interpolation routines are not really optimized very well. The
>> underlying code is quite generic for spline order, data type, and
>> arbritatry array dimension and data layout. Your fortran code is
>> probably much more optimized. It would be nice to have more optimized
>> code in nd_image, but I do not really have the resources to do that.
>>
>>> Hi,
>>> Thanks Peter for checking on the problem I reported in my last
>>> posting...
>>>
>>> Now I'm looking into using nd_image.affine_transform inplace of a
>>> fortran
>>> routine that I have been using to do this.
>>> a) I need to run this on Windows - where I don't have Fortran
>>> b) My Fortran routine does only do linear interpolation and I like
>>> the idea of
>>> experimenting with splines.
>>>
>>> A and B would of course be good reasons to use nd_image,
>>> BUT c)
>>> for a 512x512 float32 image my fortran takes about 14ms
>>> nd.affine_transform with given output array, prefilter=0 and order=1
>>> takes about 132ms !
>>> With  prefilter=1 it takes 138ms; with prefilter=1 and order=3 it
>>> takes
>>> 279ms !!  (order=2,prefilter=1  takes 226ms ; order=3,prefilter=0
>>> 222ms)
>>> All these are averaged over 10 runs on Linux (P4 2.8GHz)
>>>
>>> Why is nd_image 10x slower ? (spline order 1 does the same as linear
>>> (non-spline) interpolation, right ?)  I would call this many (100,
>>> 1000 ?)
>>> times inside a simplex algorithm which takes already many seconds
>>> to complete
>>> using the Fortran routine...
>>>
>>> Thanks,
>>> Sebastian Haase
>>>
>>> On Monday 05 December 2005 14:31, Peter Verveer wrote:
>>>> Works for me with the latest CVS version.
>>>>
>>>> On 5 Dec, 2005, at 21:13, Sebastian Haase wrote:
>>>>> Hi,
>>>>> When I call nd_image.rotate with reshape=False I always get
>>>>> "output shape not correct"
>>>>>
>>>>>>>> U.nd.rotate(d[0], 20, axes=(-1, -2), reshape=0, output=d[1],
>>>>>>>> order=1,
>>>>>
>>>>> mode="constant", cval=0.0, prefilter=0)
>>>>> Traceback (most recent call last):
>>>>>   File "<input>", line 1, in ?
>>>>>   File "/jws30/haase/PrLin/numarray/nd_image/interpolation.py",
>>>>> line 351, in
>>>>> rotate
>>>>>     output, order, mode, cval, prefilter)
>>>>>   File "/jws30/haase/PrLin/numarray/nd_image/interpolation.py",
>>>>> line 205, in
>>>>> affine_transform
>>>>>     output_type)
>>>>>   File "/jws30/haase/PrLin/numarray/nd_image/_ni_support.py", line
>>>>> 73, in
>>>>> _get_output
>>>>>     raise RuntimeError, "output shape not correct"
>>>>> RuntimeError: output shape not correct
>>>>>
>>>>> I tracked the problem down to  "inputShape != outputShape" one
>>>>> being a tuple
>>>>> the output shape being a list.
>>>>> (Pdb) p shape
>>>>> [128, 528]
>>>>> (Pdb) p output.shape
>>>>> (128, 528)
>>>>> (Pdb) p shape != output.shape
>>>>> 1
>>>>> (Pdb) p shape , output.shape
>>>>> ([128, 528], (128, 528))
>>>>> (Pdb)
>>>>>
>>>>> I'm using a CVS version around 1.3 ( /ni_interpolation.c/1.17/Fri
>>>>> Apr 22
>>>>> 20:35:27 2005//THEAD)
>>>>> but I took a look at the current CVS and it seems to still be a
>>>>> problem
>>>>>
>>>>> Looks like I'm the only one who doesn't want the reshape ;-)
>>>>>
>>>>> Thanks,
>>>>> Sebastian Haase
>>>>>
>>>>>
>>>>> -------------------------------------------------------
>>>>> This SF.net email is sponsored by: Splunk Inc. Do you grep through
>>>>> log files
>>>>> for problems?  Stop!  Download the new AJAX search engine that  
>>>>> makes
>>>>> searching your log files as easy as surfing the  web.  DOWNLOAD
>>>>> SPLUNK!
>>>>> http://ads.osdn.com/?ad_id=7637&alloc_id=16865&op=click
>>>>> _______________________________________________
>>>>> Numpy-discussion mailing list
>>>>> Numpy-discussion at lists.sourceforge.net
>>>>> https://lists.sourceforge.net/lists/listinfo/numpy-discussion
>>>>
>>>> -------------------------------------------------------
>>>> This SF.net email is sponsored by: Splunk Inc. Do you grep through
>>>> log
>>>> files for problems?  Stop!  Download the new AJAX search engine
>>>> that makes
>>>> searching your log files as easy as surfing the  web.  DOWNLOAD
>>>> SPLUNK!
>>>> http://ads.osdn.com/?ad_id=7637&alloc_id=16865&op=click
>>>> _______________________________________________
>>>> Numpy-discussion mailing list
>>>> Numpy-discussion at lists.sourceforge.net
>>>> https://lists.sourceforge.net/lists/listinfo/numpy-discussion
>>>
>>> -------------------------------------------------------
>>> This SF.net email is sponsored by: Splunk Inc. Do you grep through
>>> log files
>>> for problems?  Stop!  Download the new AJAX search engine that makes
>>> searching your log files as easy as surfing the  web.  DOWNLOAD
>>> SPLUNK!
>>> http://ads.osdn.com/?ad_id=7637&alloc_id=16865&op=click
>>> _______________________________________________
>>> Numpy-discussion mailing list
>>> Numpy-discussion at lists.sourceforge.net
>>> https://lists.sourceforge.net/lists/listinfo/numpy-discussion
>
>
> -------------------------------------------------------
> This SF.net email is sponsored by: Splunk Inc. Do you grep through  
> log files
> for problems?  Stop!  Download the new AJAX search engine that makes
> searching your log files as easy as surfing the  web.  DOWNLOAD  
> SPLUNK!
> http://ads.osdn.com/?ad_id=7637&alloc_id=16865&op=click
> _______________________________________________
> Numpy-discussion mailing list
> Numpy-discussion at lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/numpy-discussion