[Numpy-discussion] performance of nd_image affine_transform
Peter Verveer
verveer at embl-heidelberg.de
Tue Dec 6 09:37:13 EST 2005
On Dec 6, 2005, at 18:24, Sebastian Haase wrote:
> (Peter, I hope you meant to reply to the mailing list...)
Yes, I did :-)
> Hi Peter,
> Does that mean that the performance hit happens "at the lowest
> level", i.e.
> there is lot's of extra stuff in the in the innermost c loop that
> iterates
> over the output array ?
Yes, I would say so.
> Or is there just so much setup and cleanup code that
> makes the overall function slow ?
Some of that may be a problem too, for instance, the function does
malloc and free some temporary data, so that would add if you apply
the function repeatedly.
> How could I profile C extensions anyway - pointers/hints are
> appreciated !
Not sure how you would go about that. I guess optimizing would
require a rewrite for the specific cases that you like to optimize.
>
> Thanks for your nd_image (In case that wasn't clear - I like it a
> lot !)
Glad to hear that!
>
> Sebastian Haase
>
>
>
> On Tuesday 06 December 2005 01:32, you wrote:
>> Hi Sebastian,
>>
>> The interpolation routines are not really optimized very well. The
>> underlying code is quite generic for spline order, data type, and
>> arbritatry array dimension and data layout. Your fortran code is
>> probably much more optimized. It would be nice to have more optimized
>> code in nd_image, but I do not really have the resources to do that.
>>
>>> Hi,
>>> Thanks Peter for checking on the problem I reported in my last
>>> posting...
>>>
>>> Now I'm looking into using nd_image.affine_transform inplace of a
>>> fortran
>>> routine that I have been using to do this.
>>> a) I need to run this on Windows - where I don't have Fortran
>>> b) My Fortran routine does only do linear interpolation and I like
>>> the idea of
>>> experimenting with splines.
>>>
>>> A and B would of course be good reasons to use nd_image,
>>> BUT c)
>>> for a 512x512 float32 image my fortran takes about 14ms
>>> nd.affine_transform with given output array, prefilter=0 and order=1
>>> takes about 132ms !
>>> With prefilter=1 it takes 138ms; with prefilter=1 and order=3 it
>>> takes
>>> 279ms !! (order=2,prefilter=1 takes 226ms ; order=3,prefilter=0
>>> 222ms)
>>> All these are averaged over 10 runs on Linux (P4 2.8GHz)
>>>
>>> Why is nd_image 10x slower ? (spline order 1 does the same as linear
>>> (non-spline) interpolation, right ?) I would call this many (100,
>>> 1000 ?)
>>> times inside a simplex algorithm which takes already many seconds
>>> to complete
>>> using the Fortran routine...
>>>
>>> Thanks,
>>> Sebastian Haase
>>>
>>> On Monday 05 December 2005 14:31, Peter Verveer wrote:
>>>> Works for me with the latest CVS version.
>>>>
>>>> On 5 Dec, 2005, at 21:13, Sebastian Haase wrote:
>>>>> Hi,
>>>>> When I call nd_image.rotate with reshape=False I always get
>>>>> "output shape not correct"
>>>>>
>>>>>>>> U.nd.rotate(d[0], 20, axes=(-1, -2), reshape=0, output=d[1],
>>>>>>>> order=1,
>>>>>
>>>>> mode="constant", cval=0.0, prefilter=0)
>>>>> Traceback (most recent call last):
>>>>> File "<input>", line 1, in ?
>>>>> File "/jws30/haase/PrLin/numarray/nd_image/interpolation.py",
>>>>> line 351, in
>>>>> rotate
>>>>> output, order, mode, cval, prefilter)
>>>>> File "/jws30/haase/PrLin/numarray/nd_image/interpolation.py",
>>>>> line 205, in
>>>>> affine_transform
>>>>> output_type)
>>>>> File "/jws30/haase/PrLin/numarray/nd_image/_ni_support.py", line
>>>>> 73, in
>>>>> _get_output
>>>>> raise RuntimeError, "output shape not correct"
>>>>> RuntimeError: output shape not correct
>>>>>
>>>>> I tracked the problem down to "inputShape != outputShape" one
>>>>> being a tuple
>>>>> the output shape being a list.
>>>>> (Pdb) p shape
>>>>> [128, 528]
>>>>> (Pdb) p output.shape
>>>>> (128, 528)
>>>>> (Pdb) p shape != output.shape
>>>>> 1
>>>>> (Pdb) p shape , output.shape
>>>>> ([128, 528], (128, 528))
>>>>> (Pdb)
>>>>>
>>>>> I'm using a CVS version around 1.3 ( /ni_interpolation.c/1.17/Fri
>>>>> Apr 22
>>>>> 20:35:27 2005//THEAD)
>>>>> but I took a look at the current CVS and it seems to still be a
>>>>> problem
>>>>>
>>>>> Looks like I'm the only one who doesn't want the reshape ;-)
>>>>>
>>>>> Thanks,
>>>>> Sebastian Haase
>>>>>
>>>>>
>>>>> -------------------------------------------------------
>>>>> This SF.net email is sponsored by: Splunk Inc. Do you grep through
>>>>> log files
>>>>> for problems? Stop! Download the new AJAX search engine that
>>>>> makes
>>>>> searching your log files as easy as surfing the web. DOWNLOAD
>>>>> SPLUNK!
>>>>> http://ads.osdn.com/?ad_id=7637&alloc_id=16865&op=click
>>>>> _______________________________________________
>>>>> Numpy-discussion mailing list
>>>>> Numpy-discussion at lists.sourceforge.net
>>>>> https://lists.sourceforge.net/lists/listinfo/numpy-discussion
>>>>
>>>> -------------------------------------------------------
>>>> This SF.net email is sponsored by: Splunk Inc. Do you grep through
>>>> log
>>>> files for problems? Stop! Download the new AJAX search engine
>>>> that makes
>>>> searching your log files as easy as surfing the web. DOWNLOAD
>>>> SPLUNK!
>>>> http://ads.osdn.com/?ad_id=7637&alloc_id=16865&op=click
>>>> _______________________________________________
>>>> Numpy-discussion mailing list
>>>> Numpy-discussion at lists.sourceforge.net
>>>> https://lists.sourceforge.net/lists/listinfo/numpy-discussion
>>>
>>> -------------------------------------------------------
>>> This SF.net email is sponsored by: Splunk Inc. Do you grep through
>>> log files
>>> for problems? Stop! Download the new AJAX search engine that makes
>>> searching your log files as easy as surfing the web. DOWNLOAD
>>> SPLUNK!
>>> http://ads.osdn.com/?ad_id=7637&alloc_id=16865&op=click
>>> _______________________________________________
>>> Numpy-discussion mailing list
>>> Numpy-discussion at lists.sourceforge.net
>>> https://lists.sourceforge.net/lists/listinfo/numpy-discussion
>
>
> -------------------------------------------------------
> This SF.net email is sponsored by: Splunk Inc. Do you grep through
> log files
> for problems? Stop! Download the new AJAX search engine that makes
> searching your log files as easy as surfing the web. DOWNLOAD
> SPLUNK!
> http://ads.osdn.com/?ad_id=7637&alloc_id=16865&op=click
> _______________________________________________
> Numpy-discussion mailing list
> Numpy-discussion at lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/numpy-discussion
More information about the NumPy-Discussion
mailing list