Re: generic non-linear filtering that support selem and boundary handling
Basically OSX uses Clang, and Clang doesn't (yet) support OpenMP. Some people are working toward this, but it's not in there yet. OSX is the only architecture for which this is a problem. There are a lot of places in skimage where relatively simple optimizations (release the GIL, use prange) would reap very large performance benefits. I'd asked about this before, and it was viewed as not feasible due to Clang's limitations. However, I looked at this again just now and it *might *be that Cython simply ignores OpenMP features under Clang, compiling as if they didn't exist (i.e. basically substitution range for prange). If that is true, I could submit these without much issue... and we could expose pretty dramatic improvements to some existing Cython loops. Sadly, I don't have a Clang platform to test this behavior. Josh On Monday, May 26, 2014 10:48:15 PM UTC-5, Juan Nunez-Iglesias wrote:
Josh, why the parallel stripping out? Is it a Cython version requirement? We can always have this as a "dormant" PR for 1.0, for which we might bump up the version required. It might also be possible to wrap the import around a try-catch clause so that it fails gracefully on an earlier Cython version.
Juan.
On Tue, May 27, 2014 at 1:07 PM, Josh Warner <silvertrumpet999@gmail.com>wrote:
Funny you should request this. I built a Cython framework do exactly what you want (arbitrary selem, boundary handling [padding], and even arbitrary weighting of the selem in 2D and 3D) for local *weighted *variance.
The framework is about as useful as the actual calculation. Replace the innermost loop, change the name, and you get an entirely different non-linear weighted local feature. This requires a somewhat skilled user, but the framework is already done so it isn't as hard as it sounds.
Trouble is, I used some features from cython.parallel so those would have to be stripped out, losing over an order of magnitude in speed. I've been kicking around submitting them, but going back and ripping out the parallel features is painful.
If interested I can clean up and put them in a Gist or two.
Josh
On Monday, May 26, 2014 1:50:10 AM UTC-5, Chencan QIAN wrote:
Essentially, this is something similar to scipy.ndimage.filters.generic_filter (which is too slow because of its internal python for-loop), but implemented in cython as with skimage.filter.rank family.
Given the diversity within skimage.filter.rank, a generic function would be very useful. For example, I'd like to calculate local variance, which is conceptually similar to skimage.filter.rank.mean, but not supported currently.
I guess the absence of such a function must be due to some unresolved technical obstacle?
Thanks~
Chencan
-- You received this message because you are subscribed to the Google Groups "scikit-image" group. To unsubscribe from this group and stop receiving emails from it, send an email to scikit-image+unsubscribe@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Stéfan, I thought I saw some message recently about access to some OSX machines for scikit-image testing? I have one (my laptop) but zero bandwidth for anything but the simplest tests for now... Juan. On Tue, May 27, 2014 at 2:49 PM, Josh Warner <silvertrumpet999@gmail.com>wrote:
Basically OSX uses Clang, and Clang doesn't (yet) support OpenMP. Some people are working toward this, but it's not in there yet. OSX is the only architecture for which this is a problem.
There are a lot of places in skimage where relatively simple optimizations (release the GIL, use prange) would reap very large performance benefits. I'd asked about this before, and it was viewed as not feasible due to Clang's limitations.
However, I looked at this again just now and it *might *be that Cython simply ignores OpenMP features under Clang, compiling as if they didn't exist (i.e. basically substitution range for prange). If that is true, I could submit these without much issue... and we could expose pretty dramatic improvements to some existing Cython loops. Sadly, I don't have a Clang platform to test this behavior.
Josh
On Monday, May 26, 2014 10:48:15 PM UTC-5, Juan Nunez-Iglesias wrote:
Josh, why the parallel stripping out? Is it a Cython version requirement? We can always have this as a "dormant" PR for 1.0, for which we might bump up the version required. It might also be possible to wrap the import around a try-catch clause so that it fails gracefully on an earlier Cython version.
Juan.
On Tue, May 27, 2014 at 1:07 PM, Josh Warner <silvertrumpet999@gmail.com>wrote:
Funny you should request this. I built a Cython framework do exactly what you want (arbitrary selem, boundary handling [padding], and even arbitrary weighting of the selem in 2D and 3D) for local *weighted * variance.
The framework is about as useful as the actual calculation. Replace the innermost loop, change the name, and you get an entirely different non-linear weighted local feature. This requires a somewhat skilled user, but the framework is already done so it isn't as hard as it sounds.
Trouble is, I used some features from cython.parallel so those would have to be stripped out, losing over an order of magnitude in speed. I've been kicking around submitting them, but going back and ripping out the parallel features is painful.
If interested I can clean up and put them in a Gist or two.
Josh
On Monday, May 26, 2014 1:50:10 AM UTC-5, Chencan QIAN wrote:
Essentially, this is something similar to scipy.ndimage.filters.generic_filter (which is too slow because of its internal python for-loop), but implemented in cython as with skimage.filter.rank family.
Given the diversity within skimage.filter.rank, a generic function would be very useful. For example, I'd like to calculate local variance, which is conceptually similar to skimage.filter.rank.mean, but not supported currently.
I guess the absence of such a function must be due to some unresolved technical obstacle?
Thanks~
Chencan
-- You received this message because you are subscribed to the Google Groups "scikit-image" group. To unsubscribe from this group and stop receiving emails from it, send an email to scikit-image+unsubscribe@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
--
You received this message because you are subscribed to the Google Groups "scikit-image" group. To unsubscribe from this group and stop receiving emails from it, send an email to scikit-image+unsubscribe@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
participants (2)
-
Josh Warner -
Juan Nunez-Iglesias