Mailman 3 May 2013 - scikit-image

Re: Weird imsave inverting behaviour
by Johannes Schönberger 27 May '13

27 May '13

IMO the convention is to use double ([0, 1]) everywhere, except in rare cases…? Johannes Schönberger Am 27.05.2013 um 08:56 schrieb Ronnie Ghose <ronnie.ghose(a)gmail.com>: > can we just standardize it so that _everything_ uses one of the two representations? If an algorithm needs it differently, it can do the 256*array or array/256 and then again output this standard format > > > On Mon, May 27, 2013 at 2:55 AM, Ronnie Ghose <ronnie.ghose(a)gmail.com> wrote: > while we're on the subject, it seems algorithms / arrays are mixing using / wanting as input, [0,1] and [0,256] for grayscale, which leads to unexpected results > > > On Mon, May 27, 2013 at 2:51 AM, Johannes Schönberger <jschoenberger(a)demuc.de> wrote: > To be honest I agree, right now it is a mess, which is also the reason why I have not touched this package yet. Things I noticed: > > - depending on the plugin, the code and comment / doc string quality should be improved > - what's the purpose of _colormixer.pyx, _histogram.pyx, q_histogram.py, q_color_mixer.py etc. and why are they in the io plugin directory? > - the way plugins are managed should be overhauled and we should discuss it together before implementation > > My ideas: > > - rather than having .ini files I would opt for `register(plugin, desc, funcs, setup_func, *args, **kwargs)` and `unregister(plugin)` functions. This enables also users to add new plugins at runtime. > > - `setup_func` should check whether the plugin is available on this system. > > - what is indicated as `args` and `kwargs` above should definitely contain two more things: an explicit whitelist of supported file formats, extensions and dtypes or an explicit blacklist of file formats, extensions and dtypes. This could be represented as a dictionary: > > {['png']: ['uint8', 'float', '…'], ['tif', 'tiff']: … } > > It allows us to automatically select an available plugin for the given data or raise an error. > > - Explicitly selecting a plugin should be possible through an extra argument to `imread`, `imsave`. E.g. `imsave(file, data, plugin='freeimage')`. > > > > > Johannes Schönberger > > Am 27.05.2013 um 07:27 schrieb Juan Nunez-Iglesias <jni.soma(a)gmail.com>: > > > OH. Right. That's what's happening, it's not inverted, it's looping around with the overflow! (I was wondering why the correlation was not closer to -1.) > > > > Guh. imho: > > > > - This really shouldn't happen. The plugin selection should be aware of the data type, and use only a plugin that can handle it (if available) > > - PIL can definitely do 16bit PNG, so either it is being used incorrectly, or the imsave docstring stating that PIL is the first selection is wrong. I just tested this on my PIL-based Gala library, which is 3D so it required a bit of modification: > > > > In [66]: imio.write_png_image_stack(s[np.newaxis, ...], '~/Desktop/im%02i.png', axis=0 > > ) > > > > In [ > > 67]: srg = imio.read_image_stack('/Users/nuneziglesiasj/Desktop/im0*.png' > > ) > > > > In [ > > 68 > > ]: srg.shape > > Out[ > > 68]: (1040, 1392 > > ) > > > > In [ > > 69 > > ]: (s == srg).all() > > Out[ > > 69]: True > > > > - The IO module is a mess! I know Stefan mentioned overhauling it briefly, but I didn't quite realise how urgent that was. In particular: > > - is `from blah import *` used as a matter of policy or is it a remnant of an earlier age? Took me ages to find the `call` function. > > - `plugin_store`? I have no idea where this gets updated. > > > > So, to the core skimage devs: what is the scope for updating this? Am I allowed to break some functionality in a PR? Or are we sticking with this API? > > > > > > On Mon, May 27, 2013 at 2:32 PM, Christoph Gohlke <cjgohlke(a)gmail.com> wrote: > > On 5/26/2013 8:31 PM, Juan Nunez-Iglesias wrote: > > Hey all, > > > > I noticed that some of my microscopy images appeared inverted after > > saving them with skimage.io.imsave. After some testing, here's some of > > the weirdness. My image is called s: > > > > ```python > > In [37]: np.min(s) > > Out[37]: 119 > > > > In [38]: np.max(s) > > Out[38]: 4095 > > > > In [39]: s.shape > > Out[39]: (1040, 1392) > > > > In [40]: s.dtype > > Out[40]: dtype('uint16') > > > > In [41]: io.imsave('/Users/nuneziglesiasj/Desktop/im.tif', s) > > > > In [42]: sr = io.imread('/Users/nuneziglesiasj/Desktop/im.tif') > > > > In [43]: (s == sr).all() > > Out[43]: Image(False, dtype=bool) > > > > In [44]: sr.min() > > Out[44]: Image(0, dtype=uint8) > > > > In [45]: sr.max() > > Out[45]: Image(255, dtype=uint8) > > ``` > > > > Ok, already this is not desirable... One would hope than an imsave > > followed by an imread would be a no-op, but whatever, let's accept for > > argument's sake that we want to limit ourselves to uint8. Here's the > > real downer: > > > > ```python > > In [46]: np.corrcoef(s.ravel(), sr.ravel()) > > Out[46]: > > array([[ 1. , -0.29849602], > > [-0.29849602, 1. ]]) > > ``` > > > > This is simply unacceptable. I haven't explored any more than this yet > > but maybe someone has an idea about what is going on here? If it helps, > > the problem is with imsave — opening the images in Apple Preview shows > > the inversion. I just wanted to know if it was some weird > > incompatibility between skimage and Preview, rather than skimage acting > > stupid. Sadly, it appears to be the latter. > > > > PS: I just tested with png and get the exact same result. > > > > > > Not many skimage.io plugins are capable of handling 16 bit data. Use gdal (read only), tifffile (uncompressed tiff only), or freeimage, e.g. `io.use_plugin('freeimage')` > > > > Christoph > > > > -- > > You received this message because you are subscribed to the Google Groups "scikit-image" group. > > To unsubscribe from this group and stop receiving emails from it, send an email to scikit-image+unsubscribe(a)googlegroups.com. > > For more options, visit https://groups.google.com/groups/opt_out. > > > > > > > > > > -- > > You received this message because you are subscribed to the Google Groups "scikit-image" group. > > To unsubscribe from this group and stop receiving emails from it, send an email to scikit-image+unsubscribe(a)googlegroups.com. > > For more options, visit https://groups.google.com/groups/opt_out. > > > > > > -- > You received this message because you are subscribed to the Google Groups "scikit-image" group. > To unsubscribe from this group and stop receiving emails from it, send an email to scikit-image+unsubscribe(a)googlegroups.com. > For more options, visit https://groups.google.com/groups/opt_out. > > > > > > -- > You received this message because you are subscribed to the Google Groups "scikit-image" group. > To unsubscribe from this group and stop receiving emails from it, send an email to scikit-image+unsubscribe(a)googlegroups.com. > For more options, visit https://groups.google.com/groups/opt_out. > >

2 1

Re: Weird imsave inverting behaviour
by Ronnie Ghose 27 May '13

27 May '13

can we just standardize it so that _everything_ uses one of the two representations? If an algorithm needs it differently, it can do the 256*array or array/256 and then again output this standard format On Mon, May 27, 2013 at 2:55 AM, Ronnie Ghose <ronnie.ghose(a)gmail.com>wrote: > while we're on the subject, it seems algorithms / arrays are mixing using > / wanting as input, [0,1] and [0,256] for grayscale, which leads to > unexpected results > > > On Mon, May 27, 2013 at 2:51 AM, Johannes Schönberger < > jschoenberger(a)demuc.de> wrote: > >> To be honest I agree, right now it is a mess, which is also the reason >> why I have not touched this package yet. Things I noticed: >> >> - depending on the plugin, the code and comment / doc string quality >> should be improved >> - what's the purpose of _colormixer.pyx, _histogram.pyx, q_histogram.py, >> q_color_mixer.py etc. and why are they in the io plugin directory? >> - the way plugins are managed should be overhauled and we should discuss >> it together before implementation >> >> My ideas: >> >> - rather than having .ini files I would opt for `register(plugin, desc, >> funcs, setup_func, *args, **kwargs)` and `unregister(plugin)` functions. >> This enables also users to add new plugins at runtime. >> >> - `setup_func` should check whether the plugin is available on this >> system. >> >> - what is indicated as `args` and `kwargs` above should definitely >> contain two more things: an explicit whitelist of supported file formats, >> extensions and dtypes or an explicit blacklist of file formats, extensions >> and dtypes. This could be represented as a dictionary: >> >> {['png']: ['uint8', 'float', '…'], ['tif', 'tiff']: … } >> >> It allows us to automatically select an available plugin for the >> given data or raise an error. >> >> - Explicitly selecting a plugin should be possible through an extra >> argument to `imread`, `imsave`. E.g. `imsave(file, data, >> plugin='freeimage')`. >> >> >> >> >> Johannes Schönberger >> >> Am 27.05.2013 um 07:27 schrieb Juan Nunez-Iglesias <jni.soma(a)gmail.com>: >> >> > OH. Right. That's what's happening, it's not inverted, it's looping >> around with the overflow! (I was wondering why the correlation was not >> closer to -1.) >> > >> > Guh. imho: >> > >> > - This really shouldn't happen. The plugin selection should be aware of >> the data type, and use only a plugin that can handle it (if available) >> > - PIL can definitely do 16bit PNG, so either it is being used >> incorrectly, or the imsave docstring stating that PIL is the first >> selection is wrong. I just tested this on my PIL-based Gala library, which >> is 3D so it required a bit of modification: >> > >> > In [66]: imio.write_png_image_stack(s[np.newaxis, ...], >> '~/Desktop/im%02i.png', axis=0 >> > ) >> > >> > In [ >> > 67]: srg = >> imio.read_image_stack('/Users/nuneziglesiasj/Desktop/im0*.png' >> > ) >> > >> > In [ >> > 68 >> > ]: srg.shape >> > Out[ >> > 68]: (1040, 1392 >> > ) >> > >> > In [ >> > 69 >> > ]: (s == srg).all() >> > Out[ >> > 69]: True >> > >> > - The IO module is a mess! I know Stefan mentioned overhauling it >> briefly, but I didn't quite realise how urgent that was. In particular: >> > - is `from blah import *` used as a matter of policy or is it a >> remnant of an earlier age? Took me ages to find the `call` function. >> > - `plugin_store`? I have no idea where this gets updated. >> > >> > So, to the core skimage devs: what is the scope for updating this? Am I >> allowed to break some functionality in a PR? Or are we sticking with this >> API? >> > >> > >> > On Mon, May 27, 2013 at 2:32 PM, Christoph Gohlke <cjgohlke(a)gmail.com> >> wrote: >> > On 5/26/2013 8:31 PM, Juan Nunez-Iglesias wrote: >> > Hey all, >> > >> > I noticed that some of my microscopy images appeared inverted after >> > saving them with skimage.io.imsave. After some testing, here's some of >> > the weirdness. My image is called s: >> > >> > ```python >> > In [37]: np.min(s) >> > Out[37]: 119 >> > >> > In [38]: np.max(s) >> > Out[38]: 4095 >> > >> > In [39]: s.shape >> > Out[39]: (1040, 1392) >> > >> > In [40]: s.dtype >> > Out[40]: dtype('uint16') >> > >> > In [41]: io.imsave('/Users/nuneziglesiasj/Desktop/im.tif', s) >> > >> > In [42]: sr = io.imread('/Users/nuneziglesiasj/Desktop/im.tif') >> > >> > In [43]: (s == sr).all() >> > Out[43]: Image(False, dtype=bool) >> > >> > In [44]: sr.min() >> > Out[44]: Image(0, dtype=uint8) >> > >> > In [45]: sr.max() >> > Out[45]: Image(255, dtype=uint8) >> > ``` >> > >> > Ok, already this is not desirable... One would hope than an imsave >> > followed by an imread would be a no-op, but whatever, let's accept for >> > argument's sake that we want to limit ourselves to uint8. Here's the >> > real downer: >> > >> > ```python >> > In [46]: np.corrcoef(s.ravel(), sr.ravel()) >> > Out[46]: >> > array([[ 1. , -0.29849602], >> > [-0.29849602, 1. ]]) >> > ``` >> > >> > This is simply unacceptable. I haven't explored any more than this yet >> > but maybe someone has an idea about what is going on here? If it helps, >> > the problem is with imsave — opening the images in Apple Preview shows >> > the inversion. I just wanted to know if it was some weird >> > incompatibility between skimage and Preview, rather than skimage acting >> > stupid. Sadly, it appears to be the latter. >> > >> > PS: I just tested with png and get the exact same result. >> > >> > >> > Not many skimage.io plugins are capable of handling 16 bit data. Use >> gdal (read only), tifffile (uncompressed tiff only), or freeimage, e.g. >> `io.use_plugin('freeimage')` >> > >> > Christoph >> > >> > -- >> > You received this message because you are subscribed to the Google >> Groups "scikit-image" group. >> > To unsubscribe from this group and stop receiving emails from it, send >> an email to scikit-image+unsubscribe(a)googlegroups.com. >> > For more options, visit https://groups.google.com/groups/opt_out. >> > >> > >> > >> > >> > -- >> > You received this message because you are subscribed to the Google >> Groups "scikit-image" group. >> > To unsubscribe from this group and stop receiving emails from it, send >> an email to scikit-image+unsubscribe(a)googlegroups.com. >> > For more options, visit https://groups.google.com/groups/opt_out. >> > >> > >> >> -- >> You received this message because you are subscribed to the Google Groups >> "scikit-image" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to scikit-image+unsubscribe(a)googlegroups.com. >> For more options, visit https://groups.google.com/groups/opt_out. >> >> >> >

1 0

Re: Weird imsave inverting behaviour
by Christoph Gohlke 27 May '13

27 May '13

On 5/26/2013 8:31 PM, Juan Nunez-Iglesias wrote: > Hey all, > > I noticed that some of my microscopy images appeared inverted after > saving them with skimage.io.imsave. After some testing, here's some of > the weirdness. My image is called s: > > ```python > In [37]: np.min(s) > Out[37]: 119 > > In [38]: np.max(s) > Out[38]: 4095 > > In [39]: s.shape > Out[39]: (1040, 1392) > > In [40]: s.dtype > Out[40]: dtype('uint16') > > In [41]: io.imsave('/Users/nuneziglesiasj/Desktop/im.tif', s) > > In [42]: sr = io.imread('/Users/nuneziglesiasj/Desktop/im.tif') > > In [43]: (s == sr).all() > Out[43]: Image(False, dtype=bool) > > In [44]: sr.min() > Out[44]: Image(0, dtype=uint8) > > In [45]: sr.max() > Out[45]: Image(255, dtype=uint8) > ``` > > Ok, already this is not desirable... One would hope than an imsave > followed by an imread would be a no-op, but whatever, let's accept for > argument's sake that we want to limit ourselves to uint8. Here's the > real downer: > > ```python > In [46]: np.corrcoef(s.ravel(), sr.ravel()) > Out[46]: > array([[ 1. , -0.29849602], > [-0.29849602, 1. ]]) > ``` > > This is simply unacceptable. I haven't explored any more than this yet > but maybe someone has an idea about what is going on here? If it helps, > the problem is with imsave Ã¯Â¿Â½ opening the images in Apple Preview shows > the inversion. I just wanted to know if it was some weird > incompatibility between skimage and Preview, rather than skimage acting > stupid. Sadly, it appears to be the latter. > > PS: I just tested with png and get the exact same result. > Not many skimage.io plugins are capable of handling 16 bit data. Use gdal (read only), tifffile (uncompressed tiff only), or freeimage, e.g. `io.use_plugin('freeimage')` Christoph

2 1

Weird imsave inverting behaviour
by Juan Nunez-Iglesias 27 May '13

27 May '13

Hey all, I noticed that some of my microscopy images appeared inverted after saving them with skimage.io.imsave. After some testing, here's some of the weirdness. My image is called s: ```python In [37]: np.min(s) Out[37]: 119 In [38]: np.max(s) Out[38]: 4095 In [39]: s.shape Out[39]: (1040, 1392) In [40]: s.dtype Out[40]: dtype('uint16') In [41]: io.imsave('/Users/nuneziglesiasj/Desktop/im.tif', s) In [42]: sr = io.imread('/Users/nuneziglesiasj/Desktop/im.tif') In [43]: (s == sr).all() Out[43]: Image(False, dtype=bool) In [44]: sr.min() Out[44]: Image(0, dtype=uint8) In [45]: sr.max() Out[45]: Image(255, dtype=uint8) ``` Ok, already this is not desirable... One would hope than an imsave followed by an imread would be a no-op, but whatever, let's accept for argument's sake that we want to limit ourselves to uint8. Here's the real downer: ```python In [46]: np.corrcoef(s.ravel(), sr.ravel()) Out[46]: array([[ 1. , -0.29849602], [-0.29849602, 1. ]]) ``` This is simply unacceptable. I haven't explored any more than this yet but maybe someone has an idea about what is going on here? If it helps, the problem is with imsave — opening the images in Apple Preview shows the inversion. I just wanted to know if it was some weird incompatibility between skimage and Preview, rather than skimage acting stupid. Sadly, it appears to be the latter. PS: I just tested with png and get the exact same result.

1 0

Re: numexpr
by Francesc Alted 24 May '13

24 May '13

On 5/20/13 1:58 PM, StÃ¯Â¿Â½fan van der Walt wrote: > On Mon, May 20, 2013 at 1:47 PM, Johannes SchÃ¯Â¿Â½nberger > <jschoenberger(a)demuc.de> wrote: >> Yes, exposure, edge detection, namely all functions that rely on basic array multiplication / ufuncs. > I'm hesitant. The added dependency and code modification adds > complexity, and I don't know how much improvement we will get. Could > you do a benchmark of a typical function? Yes, benchmarking is always a must before taking decisions like this. At any rate, you don't need to make a hard dependency of numexpr. This is what I typically do: try: import numexpr numexpr_ispresent = True except ImportError: numexpr_ispresent = False and then, in your code: if numexpr_ispresent: # you numexpr code here else: # your other code path here That means two code paths, which is a bit more complex, but still acceptable in many cases. -- Francesc Alted

2 1

Re: Some ideas for skimage
by Ankit Agrawal 23 May '13

23 May '13

Does 'make coverage' test the examples in docstrings? If not, is there any separate command to test them?

2 1

Re: ENH: improved, faster algorithm for array padding
by Josh Warner 21 May '13

21 May '13

Thanks guys! I'll be bringing a version of this over to scikit-image.utilin the very near future, to be used if NumPy < 1.8.x. On Monday, May 20, 2013 5:06:31 AM UTC-5, Chintak Sheth wrote: Cheers Josh! > On May 20, 2013 3:27 PM, "Stéfan van der Walt" <ste...(a)sun.ac.za<javascript:>> > wrote: > >> Kudos to Josh Warner for getting this patch into NumPy so quickly! >> >> ---------- Forwarded message ---------- >> From: GitHub >> Date: Sun, May 19, 2013 at 9:50 PM >> Subject: [Numpy-svn] [numpy/numpy] 246c06: ENH: improved, faster >> algorithm for array padding >> >> Branch: refs/heads/master >> Home: https://github.com/numpy/numpy >> Commit: 246c06d2475718ec36ba193494444464e497c69a >> >> https://github.com/numpy/numpy/commit/246c06d2475718ec36ba193494444464e497c… >> Author: Josh Warner (Mac) <warner...(a)mayo.edu <javascript:>> >> Date: 2013-05-19 (Sun, 19 May 2013) >> >> Changed paths: >> M doc/release/1.8.0-notes.rst >> M numpy/lib/arraypad.py >> M numpy/lib/tests/test_arraypad.py >> >> Log Message: >> ----------- >> ENH: improved, faster algorithm for array padding >> >> New padding method which scales much better with dimensionality. >> This new implementation is fully vectorized, builds each abstracted >> n-dimensional padding block in a single step, and takes advantage >> of separability. The API is completely preserved, and the old >> algorithm is used if a vector function is input for `mode`. >> >> The new algorithm is faster for all tested combinations of inputs, >> and scales much better with dimensionality. Execution time reductions >> from ~25% for small rank 1 arrays to >99% for rank 4+ arrays observed. >> >> >> Commit: 7d188bf1c9ac5bf7ee41e0794d59771802b345bf >> >> https://github.com/numpy/numpy/commit/7d188bf1c9ac5bf7ee41e0794d59771802b34… >> Author: Charles Harris <charles...(a)gmail.com <javascript:>> >> Date: 2013-05-19 (Sun, 19 May 2013) >> >> Changed paths: >> M doc/release/1.8.0-notes.rst >> M numpy/lib/arraypad.py >> M numpy/lib/tests/test_arraypad.py >> >> Log Message: >> ----------- >> Merge pull request #3329 from JDWarner/faster_arraypad >> >> ENH: improved, faster algorithm for array padding >> >> >> Compare: >> https://github.com/numpy/numpy/compare/5b28d185a494...7d188bf1c9ac >> >> -- >> You received this message because you are subscribed to the Google Groups >> "scikit-image" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to scikit-image...(a)googlegroups.com <javascript:>. >> For more options, visit https://groups.google.com/groups/opt_out. >> >> >>

1 0

Re: numexpr
by Johannes Schönberger 20 May '13

20 May '13

Yes, exposure, edge detection, namely all functions that rely on basic array multiplication / ufuncs. Johannes Schönberger Am 20.05.2013 um 13:20 schrieb Stéfan van der Walt <stefan(a)sun.ac.za>: > On Mon, May 20, 2013 at 1:17 PM, Johannes Schönberger > <jschoenberger(a)demuc.de> wrote: >> Just have a look at https://code.google.com/p/numexpr/ - the site pretty much covers the advantages of numexpr. > > I'm familiar with the package, but I'm not sure how it benefits us. > It's mainly for speeding up single-line expressions done on all the > data, i.e. the kind of thing that numpy is pretty good for it does a > little bit faster. Are you thinking of something like the recent > exposure adjustment functions? > > Stéfan > > -- > You received this message because you are subscribed to the Google Groups "scikit-image" group. > To unsubscribe from this group and stop receiving emails from it, send an email to scikit-image+unsubscribe(a)googlegroups.com. > For more options, visit https://groups.google.com/groups/opt_out. > >

2 1

Re: numexpr
by Johannes Schönberger 20 May '13

20 May '13

With numexpr you can pass plain Python code, in Cython you have to re-implement the loop by hand. Also, it supports multi-threading by default without any further coding. Johannes Schönberger Am 20.05.2013 um 12:58 schrieb Stéfan van der Walt <stefan(a)sun.ac.za>: > Hi Johannes > > On Fri, May 17, 2013 at 3:56 PM, Johannes Schönberger > <jschoenberger(a)demuc.de> wrote: >> I think, this could also improve performance of some functions in skimage. What's your opinion on adding this as a new dependency? > > Do you think there's any advantage of using numexpr over Cython? > > Stéfan > > -- > You received this message because you are subscribed to the Google Groups "scikit-image" group. > To unsubscribe from this group and stop receiving emails from it, send an email to scikit-image+unsubscribe(a)googlegroups.com. > For more options, visit https://groups.google.com/groups/opt_out. > >

2 2

Re: numexpr
by Ronnie Ghose 20 May '13

20 May '13

i would think it's faster than Cython since it parallel-izes On Mon, May 20, 2013 at 6:58 AM, Stéfan van der Walt <stefan(a)sun.ac.za>wrote: > Hi Johannes > > On Fri, May 17, 2013 at 3:56 PM, Johannes Schönberger > <jschoenberger(a)demuc.de> wrote: > > I think, this could also improve performance of some functions in > skimage. What's your opinion on adding this as a new dependency? > > Do you think there's any advantage of using numexpr over Cython? > > Stéfan > > -- > You received this message because you are subscribed to the Google Groups > "scikit-image" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to scikit-image+unsubscribe(a)googlegroups.com. > For more options, visit https://groups.google.com/groups/opt_out. > > >

1 0