Re: Weird imsave inverting behaviour
IMO the convention is to use double ([0, 1]) everywhere, except in rare cases…? Johannes Schönberger Am 27.05.2013 um 08:56 schrieb Ronnie Ghose <ronnie.ghose@gmail.com>:
can we just standardize it so that _everything_ uses one of the two representations? If an algorithm needs it differently, it can do the 256*array or array/256 and then again output this standard format
On Mon, May 27, 2013 at 2:55 AM, Ronnie Ghose <ronnie.ghose@gmail.com> wrote: while we're on the subject, it seems algorithms / arrays are mixing using / wanting as input, [0,1] and [0,256] for grayscale, which leads to unexpected results
On Mon, May 27, 2013 at 2:51 AM, Johannes Schönberger <jschoenberger@demuc.de> wrote: To be honest I agree, right now it is a mess, which is also the reason why I have not touched this package yet. Things I noticed:
- depending on the plugin, the code and comment / doc string quality should be improved - what's the purpose of _colormixer.pyx, _histogram.pyx, q_histogram.py, q_color_mixer.py etc. and why are they in the io plugin directory? - the way plugins are managed should be overhauled and we should discuss it together before implementation
My ideas:
- rather than having .ini files I would opt for `register(plugin, desc, funcs, setup_func, *args, **kwargs)` and `unregister(plugin)` functions. This enables also users to add new plugins at runtime.
- `setup_func` should check whether the plugin is available on this system.
- what is indicated as `args` and `kwargs` above should definitely contain two more things: an explicit whitelist of supported file formats, extensions and dtypes or an explicit blacklist of file formats, extensions and dtypes. This could be represented as a dictionary:
{['png']: ['uint8', 'float', '…'], ['tif', 'tiff']: … }
It allows us to automatically select an available plugin for the given data or raise an error.
- Explicitly selecting a plugin should be possible through an extra argument to `imread`, `imsave`. E.g. `imsave(file, data, plugin='freeimage')`.
Johannes Schönberger
Am 27.05.2013 um 07:27 schrieb Juan Nunez-Iglesias <jni.soma@gmail.com>:
OH. Right. That's what's happening, it's not inverted, it's looping around with the overflow! (I was wondering why the correlation was not closer to -1.)
Guh. imho:
- This really shouldn't happen. The plugin selection should be aware of the data type, and use only a plugin that can handle it (if available) - PIL can definitely do 16bit PNG, so either it is being used incorrectly, or the imsave docstring stating that PIL is the first selection is wrong. I just tested this on my PIL-based Gala library, which is 3D so it required a bit of modification:
In [66]: imio.write_png_image_stack(s[np.newaxis, ...], '~/Desktop/im%02i.png', axis=0 )
In [ 67]: srg = imio.read_image_stack('/Users/nuneziglesiasj/Desktop/im0*.png' )
In [ 68 ]: srg.shape Out[ 68]: (1040, 1392 )
In [ 69 ]: (s == srg).all() Out[ 69]: True
- The IO module is a mess! I know Stefan mentioned overhauling it briefly, but I didn't quite realise how urgent that was. In particular: - is `from blah import *` used as a matter of policy or is it a remnant of an earlier age? Took me ages to find the `call` function. - `plugin_store`? I have no idea where this gets updated.
So, to the core skimage devs: what is the scope for updating this? Am I allowed to break some functionality in a PR? Or are we sticking with this API?
On Mon, May 27, 2013 at 2:32 PM, Christoph Gohlke <cjgohlke@gmail.com> wrote: On 5/26/2013 8:31 PM, Juan Nunez-Iglesias wrote: Hey all,
I noticed that some of my microscopy images appeared inverted after saving them with skimage.io.imsave. After some testing, here's some of the weirdness. My image is called s:
```python In [37]: np.min(s) Out[37]: 119
In [38]: np.max(s) Out[38]: 4095
In [39]: s.shape Out[39]: (1040, 1392)
In [40]: s.dtype Out[40]: dtype('uint16')
In [41]: io.imsave('/Users/nuneziglesiasj/Desktop/im.tif', s)
In [42]: sr = io.imread('/Users/nuneziglesiasj/Desktop/im.tif')
In [43]: (s == sr).all() Out[43]: Image(False, dtype=bool)
In [44]: sr.min() Out[44]: Image(0, dtype=uint8)
In [45]: sr.max() Out[45]: Image(255, dtype=uint8) ```
Ok, already this is not desirable... One would hope than an imsave followed by an imread would be a no-op, but whatever, let's accept for argument's sake that we want to limit ourselves to uint8. Here's the real downer:
```python In [46]: np.corrcoef(s.ravel(), sr.ravel()) Out[46]: array([[ 1. , -0.29849602], [-0.29849602, 1. ]]) ```
This is simply unacceptable. I haven't explored any more than this yet but maybe someone has an idea about what is going on here? If it helps, the problem is with imsave — opening the images in Apple Preview shows the inversion. I just wanted to know if it was some weird incompatibility between skimage and Preview, rather than skimage acting stupid. Sadly, it appears to be the latter.
PS: I just tested with png and get the exact same result.
Not many skimage.io plugins are capable of handling 16 bit data. Use gdal (read only), tifffile (uncompressed tiff only), or freeimage, e.g. `io.use_plugin('freeimage')`
Christoph
-- You received this message because you are subscribed to the Google Groups "scikit-image" group. To unsubscribe from this group and stop receiving emails from it, send an email to scikit-image+unsubscribe@googlegroups.com. For more options, visit https://groups.google.com/groups/opt_out.
-- You received this message because you are subscribed to the Google Groups "scikit-image" group. To unsubscribe from this group and stop receiving emails from it, send an email to scikit-image+unsubscribe@googlegroups.com. For more options, visit https://groups.google.com/groups/opt_out.
-- You received this message because you are subscribed to the Google Groups "scikit-image" group. To unsubscribe from this group and stop receiving emails from it, send an email to scikit-image+unsubscribe@googlegroups.com. For more options, visit https://groups.google.com/groups/opt_out.
-- You received this message because you are subscribed to the Google Groups "scikit-image" group. To unsubscribe from this group and stop receiving emails from it, send an email to scikit-image+unsubscribe@googlegroups.com. For more options, visit https://groups.google.com/groups/opt_out.
Can I add that in the docs somewhere? hmm... the question is where should general guidelines go.... On Mon, May 27, 2013 at 3:02 AM, Johannes Schönberger < jschoenberger@demuc.de> wrote:
IMO the convention is to use double ([0, 1]) everywhere, except in rare cases…?
Johannes Schönberger
Am 27.05.2013 um 08:56 schrieb Ronnie Ghose <ronnie.ghose@gmail.com>:
can we just standardize it so that _everything_ uses one of the two representations? If an algorithm needs it differently, it can do the 256*array or array/256 and then again output this standard format
On Mon, May 27, 2013 at 2:55 AM, Ronnie Ghose <ronnie.ghose@gmail.com> wrote: while we're on the subject, it seems algorithms / arrays are mixing using / wanting as input, [0,1] and [0,256] for grayscale, which leads to unexpected results
On Mon, May 27, 2013 at 2:51 AM, Johannes Schönberger < jschoenberger@demuc.de> wrote: To be honest I agree, right now it is a mess, which is also the reason why I have not touched this package yet. Things I noticed:
- depending on the plugin, the code and comment / doc string quality should be improved - what's the purpose of _colormixer.pyx, _histogram.pyx, q_histogram.py, q_color_mixer.py etc. and why are they in the io plugin directory? - the way plugins are managed should be overhauled and we should discuss it together before implementation
My ideas:
- rather than having .ini files I would opt for `register(plugin, desc, funcs, setup_func, *args, **kwargs)` and `unregister(plugin)` functions. This enables also users to add new plugins at runtime.
- `setup_func` should check whether the plugin is available on this system.
- what is indicated as `args` and `kwargs` above should definitely contain two more things: an explicit whitelist of supported file formats, extensions and dtypes or an explicit blacklist of file formats, extensions and dtypes. This could be represented as a dictionary:
{['png']: ['uint8', 'float', '…'], ['tif', 'tiff']: … }
It allows us to automatically select an available plugin for the given data or raise an error.
- Explicitly selecting a plugin should be possible through an extra argument to `imread`, `imsave`. E.g. `imsave(file, data, plugin='freeimage')`.
Johannes Schönberger
Am 27.05.2013 um 07:27 schrieb Juan Nunez-Iglesias <jni.soma@gmail.com>:
OH. Right. That's what's happening, it's not inverted, it's looping around with the overflow! (I was wondering why the correlation was not closer to -1.)
Guh. imho:
- This really shouldn't happen. The plugin selection should be aware of the data type, and use only a plugin that can handle it (if available) - PIL can definitely do 16bit PNG, so either it is being used incorrectly, or the imsave docstring stating that PIL is the first selection is wrong. I just tested this on my PIL-based Gala library, which is 3D so it required a bit of modification:
In [66]: imio.write_png_image_stack(s[np.newaxis, ...], '~/Desktop/im%02i.png', axis=0 )
In [ 67]: srg = imio.read_image_stack('/Users/nuneziglesiasj/Desktop/im0*.png' )
In [ 68 ]: srg.shape Out[ 68]: (1040, 1392 )
In [ 69 ]: (s == srg).all() Out[ 69]: True
- The IO module is a mess! I know Stefan mentioned overhauling it briefly, but I didn't quite realise how urgent that was. In particular: - is `from blah import *` used as a matter of policy or is it a remnant of an earlier age? Took me ages to find the `call` function. - `plugin_store`? I have no idea where this gets updated.
So, to the core skimage devs: what is the scope for updating this? Am I allowed to break some functionality in a PR? Or are we sticking with this API?
On Mon, May 27, 2013 at 2:32 PM, Christoph Gohlke <cjgohlke@gmail.com> wrote: On 5/26/2013 8:31 PM, Juan Nunez-Iglesias wrote: Hey all,
I noticed that some of my microscopy images appeared inverted after saving them with skimage.io.imsave. After some testing, here's some of the weirdness. My image is called s:
```python In [37]: np.min(s) Out[37]: 119
In [38]: np.max(s) Out[38]: 4095
In [39]: s.shape Out[39]: (1040, 1392)
In [40]: s.dtype Out[40]: dtype('uint16')
In [41]: io.imsave('/Users/nuneziglesiasj/Desktop/im.tif', s)
In [42]: sr = io.imread('/Users/nuneziglesiasj/Desktop/im.tif')
In [43]: (s == sr).all() Out[43]: Image(False, dtype=bool)
In [44]: sr.min() Out[44]: Image(0, dtype=uint8)
In [45]: sr.max() Out[45]: Image(255, dtype=uint8) ```
Ok, already this is not desirable... One would hope than an imsave followed by an imread would be a no-op, but whatever, let's accept for argument's sake that we want to limit ourselves to uint8. Here's the real downer:
```python In [46]: np.corrcoef(s.ravel(), sr.ravel()) Out[46]: array([[ 1. , -0.29849602], [-0.29849602, 1. ]]) ```
This is simply unacceptable. I haven't explored any more than this yet but maybe someone has an idea about what is going on here? If it helps, the problem is with imsave — opening the images in Apple Preview shows the inversion. I just wanted to know if it was some weird incompatibility between skimage and Preview, rather than skimage acting stupid. Sadly, it appears to be the latter.
PS: I just tested with png and get the exact same result.
Not many skimage.io plugins are capable of handling 16 bit data. Use gdal (read only), tifffile (uncompressed tiff only), or freeimage, e.g. `io.use_plugin('freeimage')`
Christoph
-- You received this message because you are subscribed to the Google Groups "scikit-image" group. To unsubscribe from this group and stop receiving emails from it, send an email to scikit-image+unsubscribe@googlegroups.com. For more options, visit https://groups.google.com/groups/opt_out.
-- You received this message because you are subscribed to the Google Groups "scikit-image" group. To unsubscribe from this group and stop receiving emails from it, send an email to scikit-image+unsubscribe@googlegroups.com. For more options, visit https://groups.google.com/groups/opt_out.
-- You received this message because you are subscribed to the Google Groups "scikit-image" group. To unsubscribe from this group and stop receiving emails from it, send an email to scikit-image+unsubscribe@googlegroups.com. For more options, visit https://groups.google.com/groups/opt_out.
-- You received this message because you are subscribed to the Google Groups "scikit-image" group. To unsubscribe from this group and stop receiving emails from it, send an email to scikit-image+unsubscribe@googlegroups.com. For more options, visit https://groups.google.com/groups/opt_out.
-- You received this message because you are subscribed to the Google Groups "scikit-image" group. To unsubscribe from this group and stop receiving emails from it, send an email to scikit-image+unsubscribe@googlegroups.com. For more options, visit https://groups.google.com/groups/opt_out.
participants (2)
-
Johannes Schönberger -
Ronnie Ghose