[scikit-image] Memory consumption of measure.label (compared to matlab)

[Gregory Lee]

Hi Martin,

My problem my analysis uses much more memory than I expect.
> I attached output from the memory_profiler package, with which I tried
> to keep track of the memory consumption of my analysis.
> You can see that for an ~8MiB file that I used for testing,
> skimage.measure.label needs to use 56MiB of memory, which surprised me.
>

I haven't looked at it in much detail, but I did find what appear to be
some unnecessary copies in the top-level Cython routine called by
skimage.morphology.label.  I opened a PR to try and avoid this here:
https://github.com/scikit-image/scikit-image/pull/2701

However, I think that PR is going to give a minor performance improvement,
but not help with memory use much if at all.  I think the main reason for
the increased memory usage is that the output type of the label function is
int64 while your input is most likely uint8.  This means that the labels
array requires 8 times the memory usage of the uint8 input.  I don't think
there is much way around that without making a version of the routines that
allows specifying a smaller integer dtype.

- Greg
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/scikit-image/attachments/20170712/32f9fd02/attachment.html>