Dividing a large image into smaller overlapping blocks for parallel processing

[Colin Lea]

Thanks Riaan, I've already made a PR for this. See 
here: https://github.com/scikit-image/scikit-image/pull/723/

On Tuesday, September 3, 2013 3:27:51 PM UTC-4, Riaan van den Dool wrote:
>
> I have created a gist with my thoughts of what such a function could look 
> like.
>
> https://gist.github.com/riaanvddool/6428299
>
> The examples shown are a bit contrived, with border_size=(0,0) and 
> non-overlapping 'rolling' windows, but the idea should be clear I think.
>
> The proc_func function can either process and return a value 
> synchronously, or create a separate job/process for each window, depending 
> on implementation. By keeping this logic separate from the windowing 
> function any multiprocessing-type solution can be used according to 
> preference. In the case of asynchronous processing the results tuple will 
> either be filled with None, or any other value returned by proc_func, ie it 
> will not be the asynchronous result itself and the proc_func will have to 
> implement a way for the asynchronous results to be returned when available 
> (callback function as an example).
>
> Your thoughts?
>
> Riaan
>
>
>
>
>
>
>
>  
>
> On Sunday, September 1, 2013 7:25:46 PM UTC+2, Johannes Schönberger wrote:
>>
>> Great! 
>>
>> Am 01.09.2013 um 17:53 schrieb Colin Lea <coli... at gmail.com>: 
>>
>> > I just made a function blocproc to do this. I'll add a pull request 
>> later today. It works in a similar way as the matlab feature. 
>> > 
>> > 
>> > 
>> > On Sunday, September 1, 2013 5:37:22 AM UTC-4, Johannes Schönberger 
>> wrote: 
>> > Yes that should be very useful. Nevertheless, I think a function like 
>> Matlab's blockproc would be a really good addition. 
>> > 
>> > Am 01.09.2013 um 11:34 schrieb Stéfan van der Walt <ste... at sun.ac.za>: 
>> > 
>> > > On Sat, Aug 31, 2013 at 8:17 PM, Johannes Schönberger <js... at demuc.de> 
>> wrote: 
>> > >> - pad image with skimage.util.pad, which allows a large number of 
>> padding methods 
>> > >> - spawn a pool of processes using Python's multiprocessing package 
>> in the standard library 
>> > >> - use shared memory to provide read access to complete image 
>> > >> - define slices of image blocks and add them to a processing queue 
>> > > 
>> > > How about we add an `overlap` parameter to 
>> > > `skimage.utill.view_as_windows`?  That should solve this problem. 
>> > > 
>> > > Stéfan 
>> > > 
>> > > -- 
>> > > You received this message because you are subscribed to the Google 
>> Groups "scikit-image" group. 
>> > > To unsubscribe from this group and stop receiving emails from it, 
>> send an email to scikit-image... at googlegroups.com. 
>> > > For more options, visit https://groups.google.com/groups/opt_out. 
>> > > 
>> > 
>> > 
>> > -- 
>> > You received this message because you are subscribed to the Google 
>> Groups "scikit-image" group. 
>> > To unsubscribe from this group and stop receiving emails from it, send 
>> an email to scikit-image... at googlegroups.com. 
>> > For more options, visit https://groups.google.com/groups/opt_out. 
>>
>>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/scikit-image/attachments/20130903/b05b8339/attachment.html>