Re: [Numpy-discussion] performance of nd_image affine_transform
(Peter, I hope you meant to reply to the mailing list...) Hi Peter, Does that mean that the performance hit happens "at the lowest level", i.e. there is lot's of extra stuff in the in the innermost c loop that iterates over the output array ? Or is there just so much setup and cleanup code that makes the overall function slow ? How could I profile C extensions anyway - pointers/hints are appreciated ! Thanks for your nd_image (In case that wasn't clear - I like it a lot !) Sebastian Haase On Tuesday 06 December 2005 01:32, you wrote:
Hi Sebastian,
The interpolation routines are not really optimized very well. The underlying code is quite generic for spline order, data type, and arbritatry array dimension and data layout. Your fortran code is probably much more optimized. It would be nice to have more optimized code in nd_image, but I do not really have the resources to do that.
Hi, Thanks Peter for checking on the problem I reported in my last posting...
Now I'm looking into using nd_image.affine_transform inplace of a fortran routine that I have been using to do this. a) I need to run this on Windows - where I don't have Fortran b) My Fortran routine does only do linear interpolation and I like the idea of experimenting with splines.
A and B would of course be good reasons to use nd_image, BUT c) for a 512x512 float32 image my fortran takes about 14ms nd.affine_transform with given output array, prefilter=0 and order=1 takes about 132ms ! With prefilter=1 it takes 138ms; with prefilter=1 and order=3 it takes 279ms !! (order=2,prefilter=1 takes 226ms ; order=3,prefilter=0 222ms) All these are averaged over 10 runs on Linux (P4 2.8GHz)
Why is nd_image 10x slower ? (spline order 1 does the same as linear (non-spline) interpolation, right ?) I would call this many (100, 1000 ?) times inside a simplex algorithm which takes already many seconds to complete using the Fortran routine...
Thanks, Sebastian Haase
On Monday 05 December 2005 14:31, Peter Verveer wrote:
Works for me with the latest CVS version.
On 5 Dec, 2005, at 21:13, Sebastian Haase wrote:
Hi, When I call nd_image.rotate with reshape=False I always get "output shape not correct"
> U.nd.rotate(d[0], 20, axes=(-1, -2), reshape=0, output=d[1], > order=1,
mode="constant", cval=0.0, prefilter=0) Traceback (most recent call last): File "<input>", line 1, in ? File "/jws30/haase/PrLin/numarray/nd_image/interpolation.py", line 351, in rotate output, order, mode, cval, prefilter) File "/jws30/haase/PrLin/numarray/nd_image/interpolation.py", line 205, in affine_transform output_type) File "/jws30/haase/PrLin/numarray/nd_image/_ni_support.py", line 73, in _get_output raise RuntimeError, "output shape not correct" RuntimeError: output shape not correct
I tracked the problem down to "inputShape != outputShape" one being a tuple the output shape being a list. (Pdb) p shape [128, 528] (Pdb) p output.shape (128, 528) (Pdb) p shape != output.shape 1 (Pdb) p shape , output.shape ([128, 528], (128, 528)) (Pdb)
I'm using a CVS version around 1.3 ( /ni_interpolation.c/1.17/Fri Apr 22 20:35:27 2005//THEAD) but I took a look at the current CVS and it seems to still be a problem
Looks like I'm the only one who doesn't want the reshape ;-)
Thanks, Sebastian Haase
------------------------------------------------------- This SF.net email is sponsored by: Splunk Inc. Do you grep through log files for problems? Stop! Download the new AJAX search engine that makes searching your log files as easy as surfing the web. DOWNLOAD SPLUNK! http://ads.osdn.com/?ad_id=7637&alloc_id=16865&op=click _______________________________________________ Numpy-discussion mailing list Numpy-discussion@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion
------------------------------------------------------- This SF.net email is sponsored by: Splunk Inc. Do you grep through log files for problems? Stop! Download the new AJAX search engine that makes searching your log files as easy as surfing the web. DOWNLOAD SPLUNK! http://ads.osdn.com/?ad_id=7637&alloc_id=16865&op=click _______________________________________________ Numpy-discussion mailing list Numpy-discussion@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion
------------------------------------------------------- This SF.net email is sponsored by: Splunk Inc. Do you grep through log files for problems? Stop! Download the new AJAX search engine that makes searching your log files as easy as surfing the web. DOWNLOAD SPLUNK! http://ads.osdn.com/?ad_id=7637&alloc_id=16865&op=click _______________________________________________ Numpy-discussion mailing list Numpy-discussion@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion
On Dec 6, 2005, at 18:24, Sebastian Haase wrote:
(Peter, I hope you meant to reply to the mailing list...)
Yes, I did :-)
Hi Peter, Does that mean that the performance hit happens "at the lowest level", i.e. there is lot's of extra stuff in the in the innermost c loop that iterates over the output array ?
Yes, I would say so.
Or is there just so much setup and cleanup code that makes the overall function slow ?
Some of that may be a problem too, for instance, the function does malloc and free some temporary data, so that would add if you apply the function repeatedly.
How could I profile C extensions anyway - pointers/hints are appreciated !
Not sure how you would go about that. I guess optimizing would require a rewrite for the specific cases that you like to optimize.
Thanks for your nd_image (In case that wasn't clear - I like it a lot !)
Glad to hear that!
Sebastian Haase
On Tuesday 06 December 2005 01:32, you wrote:
Hi Sebastian,
The interpolation routines are not really optimized very well. The underlying code is quite generic for spline order, data type, and arbritatry array dimension and data layout. Your fortran code is probably much more optimized. It would be nice to have more optimized code in nd_image, but I do not really have the resources to do that.
Hi, Thanks Peter for checking on the problem I reported in my last posting...
Now I'm looking into using nd_image.affine_transform inplace of a fortran routine that I have been using to do this. a) I need to run this on Windows - where I don't have Fortran b) My Fortran routine does only do linear interpolation and I like the idea of experimenting with splines.
A and B would of course be good reasons to use nd_image, BUT c) for a 512x512 float32 image my fortran takes about 14ms nd.affine_transform with given output array, prefilter=0 and order=1 takes about 132ms ! With prefilter=1 it takes 138ms; with prefilter=1 and order=3 it takes 279ms !! (order=2,prefilter=1 takes 226ms ; order=3,prefilter=0 222ms) All these are averaged over 10 runs on Linux (P4 2.8GHz)
Why is nd_image 10x slower ? (spline order 1 does the same as linear (non-spline) interpolation, right ?) I would call this many (100, 1000 ?) times inside a simplex algorithm which takes already many seconds to complete using the Fortran routine...
Thanks, Sebastian Haase
On Monday 05 December 2005 14:31, Peter Verveer wrote:
Works for me with the latest CVS version.
On 5 Dec, 2005, at 21:13, Sebastian Haase wrote:
Hi, When I call nd_image.rotate with reshape=False I always get "output shape not correct"
>> U.nd.rotate(d[0], 20, axes=(-1, -2), reshape=0, output=d[1], >> order=1,
mode="constant", cval=0.0, prefilter=0) Traceback (most recent call last): File "<input>", line 1, in ? File "/jws30/haase/PrLin/numarray/nd_image/interpolation.py", line 351, in rotate output, order, mode, cval, prefilter) File "/jws30/haase/PrLin/numarray/nd_image/interpolation.py", line 205, in affine_transform output_type) File "/jws30/haase/PrLin/numarray/nd_image/_ni_support.py", line 73, in _get_output raise RuntimeError, "output shape not correct" RuntimeError: output shape not correct
I tracked the problem down to "inputShape != outputShape" one being a tuple the output shape being a list. (Pdb) p shape [128, 528] (Pdb) p output.shape (128, 528) (Pdb) p shape != output.shape 1 (Pdb) p shape , output.shape ([128, 528], (128, 528)) (Pdb)
I'm using a CVS version around 1.3 ( /ni_interpolation.c/1.17/Fri Apr 22 20:35:27 2005//THEAD) but I took a look at the current CVS and it seems to still be a problem
Looks like I'm the only one who doesn't want the reshape ;-)
Thanks, Sebastian Haase
------------------------------------------------------- This SF.net email is sponsored by: Splunk Inc. Do you grep through log files for problems? Stop! Download the new AJAX search engine that makes searching your log files as easy as surfing the web. DOWNLOAD SPLUNK! http://ads.osdn.com/?ad_id=7637&alloc_id=16865&op=click _______________________________________________ Numpy-discussion mailing list Numpy-discussion@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion
------------------------------------------------------- This SF.net email is sponsored by: Splunk Inc. Do you grep through log files for problems? Stop! Download the new AJAX search engine that makes searching your log files as easy as surfing the web. DOWNLOAD SPLUNK! http://ads.osdn.com/?ad_id=7637&alloc_id=16865&op=click _______________________________________________ Numpy-discussion mailing list Numpy-discussion@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion
------------------------------------------------------- This SF.net email is sponsored by: Splunk Inc. Do you grep through log files for problems? Stop! Download the new AJAX search engine that makes searching your log files as easy as surfing the web. DOWNLOAD SPLUNK! http://ads.osdn.com/?ad_id=7637&alloc_id=16865&op=click _______________________________________________ Numpy-discussion mailing list Numpy-discussion@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion
------------------------------------------------------- This SF.net email is sponsored by: Splunk Inc. Do you grep through log files for problems? Stop! Download the new AJAX search engine that makes searching your log files as easy as surfing the web. DOWNLOAD SPLUNK! http://ads.osdn.com/?ad_id=7637&alloc_id=16865&op=click _______________________________________________ Numpy-discussion mailing list Numpy-discussion@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion
participants (2)
-
Peter Verveer -
Sebastian Haase