Image class

[Juan Nunez-Iglesias]

Well, the nays came out! =)

Josh, you were as eloquent as ever, and have convinced me. =) I think the
right model is indeed having scikit-image close(ish) to the metal, while
higher-level libraries use scikit-image to handle metadata correctly
*within their domain*. We have seen Guillaume suggest OME, while Michael
suggested GDAL. These are two metadata implementations that are *probably
incompatible* and *both* close de-facto standards within their domain. (The
DICOM standard discussed in ITK's docs, linked by Thomas, is yet another
implementation/domain.) Our library is extremely multidisciplinary in
nature (for example, Stéfan just tweeted about someone using skimage to
remove camera motion from NFL broadcasts
<http://phdfootball.blogspot.com.au/2014/09/isolating-player-movement-by.html>),
so we would somehow have to span all of these domains, which each have
their own metadata requirements. Indeed, this would be crazy.

I'll address a few of the other comments that I think are still relevant:

Johannes, slic(image, **metadata) won't work if the metadata dictionary
contains keys that are not found in the function signature, which would be
in basically all real-world scenarios. In order to support this syntax, we
would need to append a **kwargs to the end of every function that supports
metadata.

Additionally, what I was suggesting would also require writing up a
metadata specification and ensuring that all our kwargs are consistently
named with that spec. We've made some efforts in the past to do so
(spacing), but nothing comprehensive.

I'm not sure this is a good thing to do. (**kwargs, unifying function
signatures seems like a no-brainer.)

>From my point of view, the nays have it, and we can close this discussion,
but others feel free to chime in.

Thanks everyone!

Juan.


On Wed, Sep 3, 2014 at 7:51 PM, Christoph Deil <Deil.Christoph at gmail.com>
wrote:

> Hi,
>
> I agree with Josh and Johannes, I think an `Image` class that contains
> metadata and mask is a bad idea for `scikit-image`.
>
> For what it's worth, in the Astropy project [0] we are currently
> struggling with the same question.
> We currently do have an `NDData` class [1] in the Astropy core that has an
> optional `meta` dict and a `mask` (plus some other properties).
> But in the context of implementing an image utility function sub-package
> for Astropy [2] we noticed (see [3] and [4]) that it's very difficult to
> implement image utility functions for `NDData` objects (at least that's my
> interpretation ... others will probably disagree).
>
> The problem is that writing image processing functions becomes very
> complicated, because if images can (probably optionally) have metadata
> parameters or a mask, then *every function in scikit-image* should handle
> them correctly.
> But what "correctly" means depends on the application ...  quickly run into
> the problem that the semantics of how simple operations like e.g.
> `block_reduce` should handle the `mask` will be unclear ... it's better to
> leave this up to the user.
> Plus there's the problem that an API change like this would cause a lot of
> work and pain for current scikit-image users and developers.
>
> So my suggestion would be to leave the Image class (i.e. metadata and mask
> handling) to packages that use scikit-image and not try to implement a
> generic solution in scikit-image.
> If you don't I think the grim future of scikit-image that Josh described
> is realistic.
>
> Cheers,
> Christoph
>
> [0] http://www.astropy.org/
> [1] http://astropy.readthedocs.org/en/latest/nddata/index.html
> [2] http://imageutils.readthedocs.org/en/latest/imageutils/index.html
> [3] https://github.com/astropy/imageutils/issues/13
> [4] https://github.com/astropy/astropy/pull/2855
>
>
>
> On 03 Sep 2014, at 04:56, Johannes Sch önberger <jsch at demuc.de> wrote:
>
> I have not chimed in yet, but I can recall a discussion about the same
> topic a while ago. As back then, I agree with Josh, who has given some very
> good reasons. In addition, I would like to question the usefulness of the
> tightly coupled metadata in general.
>
> Alternatively, we might want to have a unified "metadata" dictionary
> format that would unwrap nicely into our preferred kwarg format, e.g. if we
> did `slic(image, **metadata)` it would just work.
>
>
> Honestly, I do not understand. This is already possible now?
>
> Otherwise, I agree, that it is more convenient to write `slic(image)` than
> `slic(image, spacing_x=metadata["spacing_x"],...)`, where `image.metadata`
> is automatically mapped to kwargs. However, this requires the metadata to
> be standardized and homogeneous. And, at least to my experience - metadata
> is the one thing that is guaranteed to cause trouble, because it is not
> consistent. So, what happens when `spacing_x` is not called `spacing_x` but
> `spacingx`... ?! Either the user actually realizes that, or the "magic",
> that maps the metadata to the `slic` parameters: warns the user, silently
> ignores, fails, etc. All this sounds complicated and intransparent to me.
>
> I think of other use cases, where attaching the metadata directly with the
> image instead of having a separate dict, brings any benefit. In contrast,
> skimage is typically used in conjunction with other libraries, which do not
> know about our wrapper class. So, whenever you call a function from another
> library, you loose the metadata.
>
> To make it short, I am a strong proponent of making the specification of
> function parameters explicit, i.e.
>
> `slic(image, spacing_x=metadata["spacing_x"],
> spacing_y=metadata["spacingy"])`
>
> or
>
> `slic(image, **metadata)`
>
> which is already possible with all skimage functions.
>
> Johannes Sch önberger
>
> On Sep 2, 2014, at 10:16 PM, Josh Warner <silvertrumpet999 at gmail.com>
> wrote:
>
> I see this a little differently, looking at it from the viewpoint of a
> scientific user/new contributor.
>
> One of the biggest reasons scikit-image has been able to grow so well, so
> rapidly, is because the bar for entry is low for new contributors. We
> operate as close to the data as possible. People using Python understand
> NumPy arrays. This is inherently familiar to those with backgrounds like
> Matlab, which is likely one of, if not the, largest population we serve. We
> want these people to be able to contribute without massive bars to entry.
>
> If we used a custom wrapper class, even a light one, I for one would
> likely never have contributed to this package in the first place. Because,
> in order to understand what was even being fed in, I would have to dig up
> that wrapper, figure out what it actually contains, where my data is in it,
> think about how to react given different inputs, and so on. This is a
> tremendously daunting mental task.
>
> In short, moving to a new scheme would be, in my humble opinion, a
> disaster. Imagine having to explain the metadata structure of the
> hypothetical scikit-image construct to everyone at every SciPy Sprint,
> every time, before they could work with the codebase. Imagine users who
> want to know what the code is doing inspecting our source, seeing methods
> called which aren't standard NumPy methods, wondering what is going on, and
> having to dig through several other unclearly linked files in our repo to
> figure that out - if they even get that far. Many have tried to solve the
> image metadata/tagging problem with few real successes; either it ends up
> too extensible (no 1:1 mapping of tag to meaning, too much info) or too
> restrictive (whoops, no 3D or new modality). A perfect system gets devised
> at first, then edge cases become apparent, followed by workarounds, then
> someone comes up with a new system that includes everything to date, the
> API breaks to support it, repeat with new cases... if nothing else, this is
> a massive support headache.
>
> From the big-picture view, we need to decide what we are, and what we want
> to become. To date, scikit-image is a powerful library of tools. We don't
> purport to hold users' hands through the process of implementation, and we
> never have. Sometimes the tools are a bit rough around the edges, and they
> aren't always as easy to use as they could be. But, in return, they end up
> exposing the maximum amount of functionality to end users. At the risk of a
> very stretched metaphor, the current state is a bit like Linux.
>
> Moving to towing around loads of metadata just so we can have complex,
> custom, branching paths behind every algorithm depending on what happens to
> be fed in - that's hand holding time. That's the Apple/OSX method. Yes, our
> package may gain a broader userbase and certain features in examples, etc.
> might be more elegant. But real world uses which don't follow the examples
> will be impeded, and more importantly we will alienate future potential
> contributors.
>
> I, for one, do not fancy implementing or maintaining the latter model. I
> fear for the future of the package even if we did implement it cleanly.
> There are aspects of the package where limited use of a custom wrapper
> would likely be reasonable, such as certain more public facing areas e.g.
> the Viewer, but I see these as the exception rather than the rule.
>
> My opinions on this issue are strong, but I recognize I'm only one voice.
> The most important thing we need to reach consensus on is the path forward,
> as noted above in bold. Up until now it's been rather clear, but wrapping
> NumPy ndarrays would start us down a very different road, one on which I
> fear we could easily lose our way.
>
> Josh
>
> On Tuesday, September 2, 2014 3:52:34 PM UTC+1, Juan Nunez-Iglesias wrote:
> Cool, looks like there's lots of discussion to be had about this, but all
> positive -- I had expected more pushback, for some reason! =)
>
> @stefanv:
>
> Do we have some use cases for this?
>
>
> Spacing and masks, as you mentioned.
>
> Spatial resolutions, so we can deal with images acquired at different
> resolutions, e.g. registering light and electron microscopy images of the
> same sample. Also, as in Tim's talk, to allow drawing of scale bars on
> displayed images.
>
> Image orientation and absolute coordinates, which help with registration
> and montaging.
>
> Image acquisition date, which could be important e.g. to generate time
> series.
>
> That's just off the top of my head, I'm sure there's more!
>
> @stefanv @tacaswell @tonysyu
>
> Please find out how, then we can fix up the io.Image class!
>
>
> Tom's link to the solution in PIMS reminds me that it was indeed Dan who
> pointed out that they'd solved the scalar issue.
>
> @glyg
>
> I would suggest relying on OME.XML
>
>
> I think that might be a little too heavy for our uses, but I might be
> wrong. Either way, it's probably possible to have a compatibility layer
> between XML and whatever use cases we want to follow. (e.g. Automatically
> reading out "PhysicalSizeX/Y/Z" tags and converting those to a "spacing"
> tag.)
>
> Juan.
>
> --
> You received this message because you are subscribed to the Google Groups
> "scikit-image" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to scikit-image+unsubscribe at googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.
>
>
> --
> You received this message because you are subscribed to the Google Groups
> "scikit-image" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to scikit-image+unsubscribe at googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.
>
>
>  --
> You received this message because you are subscribed to the Google Groups
> "scikit-image" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to scikit-image+unsubscribe at googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/scikit-image/attachments/20140904/a6b10a74/attachment.html>