Re: Weird imsave inverting behaviour
To be honest I agree, right now it is a mess, which is also the reason why I have not touched this package yet. Things I noticed: - depending on the plugin, the code and comment / doc string quality should be improved - what's the purpose of _colormixer.pyx, _histogram.pyx, q_histogram.py, q_color_mixer.py etc. and why are they in the io plugin directory? - the way plugins are managed should be overhauled and we should discuss it together before implementation My ideas: - rather than having .ini files I would opt for `register(plugin, desc, funcs, setup_func, *args, **kwargs)` and `unregister(plugin)` functions. This enables also users to add new plugins at runtime. - `setup_func` should check whether the plugin is available on this system. - what is indicated as `args` and `kwargs` above should definitely contain two more things: an explicit whitelist of supported file formats, extensions and dtypes or an explicit blacklist of file formats, extensions and dtypes. This could be represented as a dictionary: {['png']: ['uint8', 'float', '…'], ['tif', 'tiff']: … } It allows us to automatically select an available plugin for the given data or raise an error. - Explicitly selecting a plugin should be possible through an extra argument to `imread`, `imsave`. E.g. `imsave(file, data, plugin='freeimage')`. Johannes Schönberger Am 27.05.2013 um 07:27 schrieb Juan Nunez-Iglesias <jni.soma@gmail.com>:
OH. Right. That's what's happening, it's not inverted, it's looping around with the overflow! (I was wondering why the correlation was not closer to -1.)
Guh. imho:
- This really shouldn't happen. The plugin selection should be aware of the data type, and use only a plugin that can handle it (if available) - PIL can definitely do 16bit PNG, so either it is being used incorrectly, or the imsave docstring stating that PIL is the first selection is wrong. I just tested this on my PIL-based Gala library, which is 3D so it required a bit of modification:
In [66]: imio.write_png_image_stack(s[np.newaxis, ...], '~/Desktop/im%02i.png', axis=0 )
In [ 67]: srg = imio.read_image_stack('/Users/nuneziglesiasj/Desktop/im0*.png' )
In [ 68 ]: srg.shape Out[ 68]: (1040, 1392 )
In [ 69 ]: (s == srg).all() Out[ 69]: True
- The IO module is a mess! I know Stefan mentioned overhauling it briefly, but I didn't quite realise how urgent that was. In particular: - is `from blah import *` used as a matter of policy or is it a remnant of an earlier age? Took me ages to find the `call` function. - `plugin_store`? I have no idea where this gets updated.
So, to the core skimage devs: what is the scope for updating this? Am I allowed to break some functionality in a PR? Or are we sticking with this API?
On Mon, May 27, 2013 at 2:32 PM, Christoph Gohlke <cjgohlke@gmail.com> wrote: On 5/26/2013 8:31 PM, Juan Nunez-Iglesias wrote: Hey all,
I noticed that some of my microscopy images appeared inverted after saving them with skimage.io.imsave. After some testing, here's some of the weirdness. My image is called s:
```python In [37]: np.min(s) Out[37]: 119
In [38]: np.max(s) Out[38]: 4095
In [39]: s.shape Out[39]: (1040, 1392)
In [40]: s.dtype Out[40]: dtype('uint16')
In [41]: io.imsave('/Users/nuneziglesiasj/Desktop/im.tif', s)
In [42]: sr = io.imread('/Users/nuneziglesiasj/Desktop/im.tif')
In [43]: (s == sr).all() Out[43]: Image(False, dtype=bool)
In [44]: sr.min() Out[44]: Image(0, dtype=uint8)
In [45]: sr.max() Out[45]: Image(255, dtype=uint8) ```
Ok, already this is not desirable... One would hope than an imsave followed by an imread would be a no-op, but whatever, let's accept for argument's sake that we want to limit ourselves to uint8. Here's the real downer:
```python In [46]: np.corrcoef(s.ravel(), sr.ravel()) Out[46]: array([[ 1. , -0.29849602], [-0.29849602, 1. ]]) ```
This is simply unacceptable. I haven't explored any more than this yet but maybe someone has an idea about what is going on here? If it helps, the problem is with imsave — opening the images in Apple Preview shows the inversion. I just wanted to know if it was some weird incompatibility between skimage and Preview, rather than skimage acting stupid. Sadly, it appears to be the latter.
PS: I just tested with png and get the exact same result.
Not many skimage.io plugins are capable of handling 16 bit data. Use gdal (read only), tifffile (uncompressed tiff only), or freeimage, e.g. `io.use_plugin('freeimage')`
Christoph
-- You received this message because you are subscribed to the Google Groups "scikit-image" group. To unsubscribe from this group and stop receiving emails from it, send an email to scikit-image+unsubscribe@googlegroups.com. For more options, visit https://groups.google.com/groups/opt_out.
-- You received this message because you are subscribed to the Google Groups "scikit-image" group. To unsubscribe from this group and stop receiving emails from it, send an email to scikit-image+unsubscribe@googlegroups.com. For more options, visit https://groups.google.com/groups/opt_out.
while we're on the subject, it seems algorithms / arrays are mixing using / wanting as input, [0,1] and [0,256] for grayscale, which leads to unexpected results On Mon, May 27, 2013 at 2:51 AM, Johannes Schönberger < jschoenberger@demuc.de> wrote:
To be honest I agree, right now it is a mess, which is also the reason why I have not touched this package yet. Things I noticed:
- depending on the plugin, the code and comment / doc string quality should be improved - what's the purpose of _colormixer.pyx, _histogram.pyx, q_histogram.py, q_color_mixer.py etc. and why are they in the io plugin directory? - the way plugins are managed should be overhauled and we should discuss it together before implementation
My ideas:
- rather than having .ini files I would opt for `register(plugin, desc, funcs, setup_func, *args, **kwargs)` and `unregister(plugin)` functions. This enables also users to add new plugins at runtime.
- `setup_func` should check whether the plugin is available on this system.
- what is indicated as `args` and `kwargs` above should definitely contain two more things: an explicit whitelist of supported file formats, extensions and dtypes or an explicit blacklist of file formats, extensions and dtypes. This could be represented as a dictionary:
{['png']: ['uint8', 'float', '…'], ['tif', 'tiff']: … }
It allows us to automatically select an available plugin for the given data or raise an error.
- Explicitly selecting a plugin should be possible through an extra argument to `imread`, `imsave`. E.g. `imsave(file, data, plugin='freeimage')`.
Johannes Schönberger
Am 27.05.2013 um 07:27 schrieb Juan Nunez-Iglesias <jni.soma@gmail.com>:
OH. Right. That's what's happening, it's not inverted, it's looping around with the overflow! (I was wondering why the correlation was not closer to -1.)
Guh. imho:
- This really shouldn't happen. The plugin selection should be aware of the data type, and use only a plugin that can handle it (if available) - PIL can definitely do 16bit PNG, so either it is being used incorrectly, or the imsave docstring stating that PIL is the first selection is wrong. I just tested this on my PIL-based Gala library, which is 3D so it required a bit of modification:
In [66]: imio.write_png_image_stack(s[np.newaxis, ...], '~/Desktop/im%02i.png', axis=0 )
In [ 67]: srg = imio.read_image_stack('/Users/nuneziglesiasj/Desktop/im0*.png' )
In [ 68 ]: srg.shape Out[ 68]: (1040, 1392 )
In [ 69 ]: (s == srg).all() Out[ 69]: True
- The IO module is a mess! I know Stefan mentioned overhauling it briefly, but I didn't quite realise how urgent that was. In particular: - is `from blah import *` used as a matter of policy or is it a remnant of an earlier age? Took me ages to find the `call` function. - `plugin_store`? I have no idea where this gets updated.
So, to the core skimage devs: what is the scope for updating this? Am I allowed to break some functionality in a PR? Or are we sticking with this API?
On Mon, May 27, 2013 at 2:32 PM, Christoph Gohlke <cjgohlke@gmail.com> wrote: On 5/26/2013 8:31 PM, Juan Nunez-Iglesias wrote: Hey all,
I noticed that some of my microscopy images appeared inverted after saving them with skimage.io.imsave. After some testing, here's some of the weirdness. My image is called s:
```python In [37]: np.min(s) Out[37]: 119
In [38]: np.max(s) Out[38]: 4095
In [39]: s.shape Out[39]: (1040, 1392)
In [40]: s.dtype Out[40]: dtype('uint16')
In [41]: io.imsave('/Users/nuneziglesiasj/Desktop/im.tif', s)
In [42]: sr = io.imread('/Users/nuneziglesiasj/Desktop/im.tif')
In [43]: (s == sr).all() Out[43]: Image(False, dtype=bool)
In [44]: sr.min() Out[44]: Image(0, dtype=uint8)
In [45]: sr.max() Out[45]: Image(255, dtype=uint8) ```
Ok, already this is not desirable... One would hope than an imsave followed by an imread would be a no-op, but whatever, let's accept for argument's sake that we want to limit ourselves to uint8. Here's the real downer:
```python In [46]: np.corrcoef(s.ravel(), sr.ravel()) Out[46]: array([[ 1. , -0.29849602], [-0.29849602, 1. ]]) ```
This is simply unacceptable. I haven't explored any more than this yet but maybe someone has an idea about what is going on here? If it helps, the problem is with imsave — opening the images in Apple Preview shows the inversion. I just wanted to know if it was some weird incompatibility between skimage and Preview, rather than skimage acting stupid. Sadly, it appears to be the latter.
PS: I just tested with png and get the exact same result.
Not many skimage.io plugins are capable of handling 16 bit data. Use gdal (read only), tifffile (uncompressed tiff only), or freeimage, e.g. `io.use_plugin('freeimage')`
Christoph
-- You received this message because you are subscribed to the Google Groups "scikit-image" group. To unsubscribe from this group and stop receiving emails from it, send an email to scikit-image+unsubscribe@googlegroups.com. For more options, visit https://groups.google.com/groups/opt_out.
-- You received this message because you are subscribed to the Google Groups "scikit-image" group. To unsubscribe from this group and stop receiving emails from it, send an email to scikit-image+unsubscribe@googlegroups.com. For more options, visit https://groups.google.com/groups/opt_out.
-- You received this message because you are subscribed to the Google Groups "scikit-image" group. To unsubscribe from this group and stop receiving emails from it, send an email to scikit-image+unsubscribe@googlegroups.com. For more options, visit https://groups.google.com/groups/opt_out.
On Mon, May 27, 2013 at 8:51 AM, Johannes Schönberger <jschoenberger@demuc.de> wrote:
- depending on the plugin, the code and comment / doc string quality should be improved - what's the purpose of _colormixer.pyx, _histogram.pyx, q_histogram.py, q_color_mixer.py etc. and why are they in the io plugin directory?
These are used in the fancy QT image viewer, i.e. "imshow(image, fancy=True)".
- the way plugins are managed should be overhauled and we should discuss it together before implementation
My ideas:
- rather than having .ini files I would opt for `register(plugin, desc, funcs, setup_func, *args, **kwargs)` and `unregister(plugin)` functions. This enables also users to add new plugins at runtime.
This sounds very similar to the ideas mentioned by Almar Klein earlier this month: https://groups.google.com/d/msg/imageio/jNwYY6B8n7Y/svdUiYWA4mMJ Unfortunately, there's a trade-off between having things "just work" and the amount of magic that happens behind the scene. When starting scikit-image, we set it as a priority that a person should be able to do ``imread(...)`` on almost any system and have it work. I think this was an important requirement for having demos Just Work.
- what is indicated as `args` and `kwargs` above should definitely contain two more things: an explicit whitelist of supported file formats, extensions and dtypes or an explicit blacklist of file formats, extensions and dtypes. This could be represented as a dictionary:
{['png']: ['uint8', 'float', '…'], ['tif', 'tiff']: … }
It allows us to automatically select an available plugin for the given data or raise an error.
Having formats and data-types specified is a good idea. It can also be done using the ini-file format, but I'd prefer the approach you suggest. That said, let me motivate those configuration files: the idea was that one could add plugins by simply dropping the appropriate files in-place. However, the right place for those plugins to live would probably be inside the skimage repository, so it doesn't matters that much.
- Explicitly selecting a plugin should be possible through an extra argument to `imread`, `imsave`. E.g. `imsave(file, data, plugin='freeimage')`.
Just verifying that this is the way it currently functions. Stéfan
participants (3)
-
Johannes Schönberger -
Ronnie Ghose -
Stéfan van der Walt