Mailman 3 December 2011 - scikit-image

Rescale intensity function [was: Asymmetric behavior in dtype conversion]
by Tony Yu Dec. 29, 2011

Dec. 29, 2011

On Wed, Dec 21, 2011 at 11:16 AM, Tony Yu <tsyu80(a)gmail.com> wrote: > > > On Wed, Dec 21, 2011 at 9:15 AM, Christoph Gohlke <cjgohlke(a)gmail.com>wrote: > >> On 12/21/2011 8:41 AM, Stéfan van der Walt wrote: >> >>> On Wed, Dec 21, 2011 at 1:30 AM, Tony Yu<tsyu80(a)gmail.com> wrote: >>> >>>> I think one of two things should happen: >>>> >>>> 1) >>>> [uint -> int]: expand uint to *full* range of int. >>>> [int -> uint]: unchanged (squeeze into range of uint) >>>> >>>> 2) >>>> [uint -> int]: unchanged (fill *positive* range of int) >>>> [int -> uint]: clip negative values from int >>>> >>> People very often represent images in signed integers, even though >>> they only manipulate positive values, e.g. storing 0-255 in int8. So >>> the conversion to signed should not spread those values over the >>> entire range. However, I think you're right that we should add the >>> clipping in 2 to avoid the conversion bias, along with a nice big >>> warning. >>> >>> Regards >>> Stéfan >>> >>> >> At this point it might be worth considering other cases that are >> currently not well handled by skimage dtype conversions. For example: >> >> 1) float images in the range 0.0 - 255.0 >> 2) 1 bit images stored as (u)int >> 3) 10 and 12 bit images stored as uint16 >> >> Christoph >> > > I've definitely been bit by example 1 many times. > > I think we should check that float images are between 0 and 1 in > ``util.dtype._convert`` (and either warn or raise an error), but we > shouldn't have to worry about it elsewhere. That doesn't really solve the > issue, so... > > Presumably, the user would know that their data is in a slightly weird > format (especially for examples 2 and 3), correct? If so, we could just add > a function (maybe "rescale_intensity" or "stretch_intensity" or > "normalize_intensity" or something similar) that takes the image and the > input intensity range as inputs. For example: ``norm_intensity(image_u12, > (0, 4096))``. Then, we could rescale the intensity to fit into the > intensity range of the input dtype. We could also add special values [e.g. > 1) 'uint8', 2) 'bool', 3) 'uint10'/'uint12'] for the second parameter, and > these values could pull the associated range from a table. > > Then, it would be the responsibility of the user to call this function > before converting dtypes. > > -Tony > I forget that not everyone gets notified when I post a pull request. In PR 107 <https://github.com/scikits-image/scikits-image/pull/107> I add a `rescale_intensity` function, which implements my earlier suggestion. Christoph: Does this solve (or at least mitigate) the problems with weird data types? I have yet to implement "special" input ranges---i.e. input range can be a dtype string and map that string to a range. The implementation is easy, but I'm not sure what data types should be supported: 1bit (or bool?), uint10, uint12,... Comments/suggestions? -Tony

1 0

Re: Need an advice
by Nadav Horesh Dec. 28, 2011

Dec. 28, 2011

I compiled after setting the flags for a core2 linux machine. A quick and dirty benchmark showed a x2 improvement over scipy.signal.sepfir2d. I think that are some declaration errors in the python code (for instance I think that the convolution __init__.py should be from ext import pyconvolve instead of from ext import convolve I'll take a closer look later. Thank you, Nadav. 2011/12/28 Tony Yu <tsyu80(a)gmail.com> > > > On Tue, Dec 27, 2011 at 8:24 PM, Tony Yu <tsyu80(a)gmail.com> wrote: > >> >> >> On Tue, Dec 27, 2011 at 8:29 AM, Nadav Horesh <nadavh.horesh(a)gmail.com>wrote: >> >>> How can I get it? >>> >>> Nadav >>> >> >> The code Stefan mentioned is in an old pull request (PR #16<https://github.com/scikits-image/scikits-image/pull/16>). >> That PR is from before the switch from scikits.image to skimage. I've made >> a new branch<https://github.com/tonysyu/scikits-image/commits/skimage-convolution>in my git repo with the convolution code in the new namespace. >> >> If you're already running skimage from git, then you can use git to clone >> this branch: >> >> git remote -f add tonysyu git@github.com:tonysyu/scikits-image.git >> git checkout -b convolution tonysyu/skimage-convolution >> >> >> In case this isn't familiar: The first line adds my git repo to your list >> of remotes, and the "-f" flag fetches the tags from my repo (this ensures >> that you have the info about my branches). My repo is now added to your >> repo as a remote with the name "tonysyu" (you can change this). The second >> line creates a new branch "convolution", clones my "skimage-convolution" >> branch into it, and checks it out. >> >> Hope that helps, >> -Tony >> > > Oops, I forgot to mention: I added a compile flag that is specific to my > system (In convolution/setup.py there's an include flag '-I/usr/...') that > should probably be removed. Also, I should mention that I haven't gotten > this branch to compile on my system (OSX). I believe it should compile OK > on Linux (after removing the '-I/usr/...' compile flag). > > Good luck, > -Tony > >> >> 2011/12/23 Stéfan van der Walt <stefan(a)sun.ac.za> >> >>> On Thu, Dec 22, 2011 at 10:44 PM, Nadav Horesh <nadavh.horesh(a)gmail.com> >>> wrote: >>> > The application is not too sophisticated, and I do use numpy, scipy and >>> > skimage for all I need. The only problem is speed, I need the >>> processing to >>> > be at l5-10 time faster. Convolutions take most of the processing >>> time. I >>> > prefer a flexible solution that would speed up also common nonlinear >>> filters >>> > (i.e. a median filter). >>> >>> We have a PR in the pipeline for doing really, really fast >>> convolutions... but it needs a bit of work. Would you like to have a >>> look at it? >>> >>> Stéfan >>> >> > >

1 0

Re: Need an advice
by Tony Yu Dec. 28, 2011

Dec. 28, 2011

On Tue, Dec 27, 2011 at 8:24 PM, Tony Yu <tsyu80(a)gmail.com> wrote: > > > On Tue, Dec 27, 2011 at 8:29 AM, Nadav Horesh <nadavh.horesh(a)gmail.com>wrote: > >> How can I get it? >> >> Nadav >> > > The code Stefan mentioned is in an old pull request (PR #16<https://github.com/scikits-image/scikits-image/pull/16>). > That PR is from before the switch from scikits.image to skimage. I've made > a new branch<https://github.com/tonysyu/scikits-image/commits/skimage-convolution>in my git repo with the convolution code in the new namespace. > > If you're already running skimage from git, then you can use git to clone > this branch: > > git remote -f add tonysyu git@github.com:tonysyu/scikits-image.git > git checkout -b convolution tonysyu/skimage-convolution > > > In case this isn't familiar: The first line adds my git repo to your list > of remotes, and the "-f" flag fetches the tags from my repo (this ensures > that you have the info about my branches). My repo is now added to your > repo as a remote with the name "tonysyu" (you can change this). The second > line creates a new branch "convolution", clones my "skimage-convolution" > branch into it, and checks it out. > > Hope that helps, > -Tony > Oops, I forgot to mention: I added a compile flag that is specific to my system (In convolution/setup.py there's an include flag '-I/usr/...') that should probably be removed. Also, I should mention that I haven't gotten this branch to compile on my system (OSX). I believe it should compile OK on Linux (after removing the '-I/usr/...' compile flag). Good luck, -Tony > > 2011/12/23 Stéfan van der Walt <stefan(a)sun.ac.za> > >> On Thu, Dec 22, 2011 at 10:44 PM, Nadav Horesh <nadavh.horesh(a)gmail.com> >> wrote: >> > The application is not too sophisticated, and I do use numpy, scipy and >> > skimage for all I need. The only problem is speed, I need the >> processing to >> > be at l5-10 time faster. Convolutions take most of the processing time. >> I >> > prefer a flexible solution that would speed up also common nonlinear >> filters >> > (i.e. a median filter). >> >> We have a PR in the pipeline for doing really, really fast >> convolutions... but it needs a bit of work. Would you like to have a >> look at it? >> >> Stéfan >> >

1 0

Re: Need an advice
by Tony Yu Dec. 28, 2011

Dec. 28, 2011

On Tue, Dec 27, 2011 at 8:29 AM, Nadav Horesh <nadavh.horesh(a)gmail.com>wrote: > How can I get it? > > Nadav > The code Stefan mentioned is in an old pull request (PR #16<https://github.com/scikits-image/scikits-image/pull/16>). That PR is from before the switch from scikits.image to skimage. I've made a new branch<https://github.com/tonysyu/scikits-image/commits/skimage-convolution>in my git repo with the convolution code in the new namespace. If you're already running skimage from git, then you can use git to clone this branch: git remote -f add tonysyu git@github.com:tonysyu/scikits-image.git git checkout -b convolution tonysyu/skimage-convolution In case this isn't familiar: The first line adds my git repo to your list of remotes, and the "-f" flag fetches the tags from my repo (this ensures that you have the info about my branches). My repo is now added to your repo as a remote with the name "tonysyu" (you can change this). The second line creates a new branch "convolution", clones my "skimage-convolution" branch into it, and checks it out. Hope that helps, -Tony > 2011/12/23 Stéfan van der Walt <stefan(a)sun.ac.za> > >> On Thu, Dec 22, 2011 at 10:44 PM, Nadav Horesh <nadavh.horesh(a)gmail.com> >> wrote: >> > The application is not too sophisticated, and I do use numpy, scipy and >> > skimage for all I need. The only problem is speed, I need the >> processing to >> > be at l5-10 time faster. Convolutions take most of the processing time. >> I >> > prefer a flexible solution that would speed up also common nonlinear >> filters >> > (i.e. a median filter). >> >> We have a PR in the pipeline for doing really, really fast >> convolutions... but it needs a bit of work. Would you like to have a >> look at it? >> >> Stéfan >> >

1 0

Re: Need an advice
by Nadav Horesh Dec. 27, 2011

Dec. 27, 2011

How can I get it? Nadav 2011/12/23 Stéfan van der Walt <stefan(a)sun.ac.za> > On Thu, Dec 22, 2011 at 10:44 PM, Nadav Horesh <nadavh.horesh(a)gmail.com> > wrote: > > The application is not too sophisticated, and I do use numpy, scipy and > > skimage for all I need. The only problem is speed, I need the processing > to > > be at l5-10 time faster. Convolutions take most of the processing time. I > > prefer a flexible solution that would speed up also common nonlinear > filters > > (i.e. a median filter). > > We have a PR in the pipeline for doing really, really fast > convolutions... but it needs a bit of work. Would you like to have a > look at it? > > Stéfan >

1 0

Re: plugin freeimage vs gdal on 0.5dev
by jip Dec. 27, 2011

Dec. 27, 2011

Hi, The script works after updating numpy to 1.6.1 (sudo easy_install -U numpy) Thank you again. Jean-Patrick On 22 déc, 22:00, jip <jeanpatrick.pomm...(a)gmail.com> wrote: > thanks, > JP > > On 22 déc, 20:18, Nelle Varoquaux <nelle.varoqu...(a)gmail.com> wrote: > > > > > > > > > Hi JP, > > > You need to upgrade numpy to at least 1.4 > > > Thanks, > > N > > > On 22 December 2011 20:10, jip <jeanpatrick.pomm...(a)gmail.com> wrote: > > > > Hi Stéfan, > > > my numpy install seems to be broken: > > > > In [1]: import numpy as np > > > > In [2]: print np.__version__ > > > 1.3.0 > > > > In [3]: import numpy.compat > > > --------------------------------------------------------------------------- > > > ImportError Traceback (most recent call > > > last) > > > /home/claire/Applications/ProjetPython/<ipython-input-3-2f9167a96fc7> > > > in <module>() > > > ----> 1 import numpy.compat > > > > ImportError: No module named compat > > > > Thanks > > > JP > > > > On 22 déc, 18:33, Stéfan van der Walt <ste...(a)sun.ac.za> wrote: > > > > Hi JP > > > > > It seems like there is something wrong with your numpy. Could you > > > > please execute the following: > > > > > import numpy as np > > > > print np.__version__ > > > > print np > > > > > import numpy.compat > > > > print numpy.compat > > > > > Thanks > > > > Stéfan > > > > > On Thu, Dec 22, 2011 at 5:51 AM, jip <jeanpatrick.pomm...(a)gmail.com> > > > wrote: > > > > > Dear all, > > > > > If I try to load a 12bits TIF image: > > > > > sk.io.use_plugin('freeimage', 'imread') > > > > > sk.io.use_plugin('matplotlib', 'imshow') > > > > > dapi=io.imread(path+CC+im) > > > > > > I have the following message: > > > > > ** Message: pygobject_register_sinkfunc is deprecated (GstObject) > > > > > Traceback (most recent call last): > > > > > File "/home/claire/Applications/ProjetPython/projet particule et > > > > > objet/Nuclei/QuantSpots-skimage.py", line 49, in <module> > > > > > sk.io.use_plugin('freeimage', 'imread') > > > > > File "/usr/local/lib/python2.6/dist-packages/scikits_image-0.5dev- > > > > > py2.6-linux-i686.egg/skimage/io/_plugins/plugin.py", line 128, in use > > > > > _load(name) > > > > > File "/usr/local/lib/python2.6/dist-packages/scikits_image-0.5dev- > > > > > py2.6-linux-i686.egg/skimage/io/_plugins/plugin.py", line 190, in > > > > > _load > > > > > fromlist=[modname]) > > > > > File "/usr/local/lib/python2.6/dist-packages/scikits_image-0.5dev- > > > > > py2.6-linux-i686.egg/skimage/io/_plugins/freeimage_plugin.py", line 6, > > > > > in <module> > > > > > from numpy.compat import asbytes > > > > > ImportError: No module named compat > > > > > > What is that compat module? I have numpy 1.30 installed. > > > > > > Anyway,I can load the image with gdal (sk.io.use_plugin('gdal', > > > > > 'imread') with some warnings: > > > > > ** Message: pygobject_register_sinkfunc is deprecated (GstObject) > > > > > Warning 1: TIFFReadDirectory:Unknown field with tag 34959 (0x888f) > > > > > encountered > > > > > Warning 1: TIFFReadDirectory:Unknown field with tag 34960 (0x8890) > > > > > encountered > > > > > Warning 1: TIFFReadDirectory:Unknown field with tag 34959 (0x888f) > > > > > encountered > > > > > Warning 1: TIFFReadDirectory:Unknown field with tag 34960 (0x8890) > > > > > encountered > > > > > > best regards > > > > > Jean-Patrick

1 0

Pull request visualisation
by Stéfan van der Walt Dec. 26, 2011

Dec. 26, 2011

Here's an illustration of our GitHub pull-request activity over the last while. It's modest, still, but those humps are good news! https://plus.google.com/u/0/104831275312843762750/posts/dHzc6cQfyhV

1 0

Re: PKG: Cant seem to build scikits debian package from source
by Jerome Kieffer Dec. 23, 2011

Dec. 23, 2011

On Fri, 23 Dec 2011 12:28:33 +0530 harish badrinath <harishbadrinath(a)gmail.com> wrote: > Hello, > I cant seem to build scikits-image from source using debuild. > > Here are the steps i followed: > a)I checkout master and debian branches in seprate directories. > b) export master branch as a tar ball > c) run dh_make (a debian specific packaging helper script) > d) copy the contents of the debian directory to the directory in which > i ran dh_make > e) run debbuild I suggest you to install python-stdeb (on recent ubuntu/debian systems) then simply use the setup.py: python setup.py --command-packages=stdeb.command bdist_deb to create a debian package. Most of the case it works out of the box (I just tested on scikits-git and it is OK). Cheers -- Jerome Kieffer <google(a)terre-adelie.org>

1 0

Re: PKG: Cant seem to build scikits debian package from source
by Nadav Horesh Dec. 23, 2011

Dec. 23, 2011

There is a new import statement syntax. I think you should upgrade your python. I have no experience with Debian, but on the seveal other linux distros I used this upgrade is easy and harmless. Nadav. 2011/12/23 harish badrinath <harishbadrinath(a)gmail.com> > Hello, > I cant seem to build scikits-image from source using debuild. > > Here are the steps i followed: > a)I checkout master and debian branches in seprate directories. > b) export master branch as a tar ball > c) run dh_make (a debian specific packaging helper script) > d) copy the contents of the debian directory to the directory in which > i ran dh_make > e) run debbuild > > The log of the build is > <snip> > dh clean --buildsystem python_distutils > dh_testdir -O--buildsystem=python_distutils > dh_auto_clean -O--buildsystem=python_distutils > Traceback (most recent call last): > File "setup.py", line 101, in <module> > cmdclass={'build_py': build_py}, > File "/usr/lib/pymodules/python2.5/numpy/distutils/core.py", line > 152, in setup > config = configuration() > File "setup.py", line 42, in configuration > config.add_subpackage('skimage') > File "/usr/lib/pymodules/python2.5/numpy/distutils/misc_util.py", > line 957, in add_subpackage > caller_level = 2) > File "/usr/lib/pymodules/python2.5/numpy/distutils/misc_util.py", > line 926, in get_subpackage > caller_level = caller_level + 1) > File "/usr/lib/pymodules/python2.5/numpy/distutils/misc_util.py", > line 863, in _get_configuration_from_setup_py > config = setup_module.configuration(*args) > File "skimage/setup.py", line 8, in configuration > config.add_subpackage('graph') > File "/usr/lib/pymodules/python2.5/numpy/distutils/misc_util.py", > line 957, in add_subpackage > caller_level = 2) > File "/usr/lib/pymodules/python2.5/numpy/distutils/misc_util.py", > line 926, in get_subpackage > caller_level = caller_level + 1) > File "/usr/lib/pymodules/python2.5/numpy/distutils/misc_util.py", > line 848, in _get_configuration_from_setup_py > ('.py', 'U', 1)) > File "skimage/graph/setup.py", line 3, in <module> > from skimage._build import cython > File > "/home/code/projs/scikits-image-master/scikits-image-0.4/skimage/__init__.py", > line 64 > from .util.dtype import * > SyntaxError: 'import *' not allowed with 'from .' > dh_auto_clean: python2.5 setup.py clean -a returned exit code 1 > make: *** [clean] Error 1 > dpkg-buildpackage: error: fakeroot debian/rules clean gave error exit > status 2 > debuild: fatal error at line 1325: > dpkg-buildpackage -rfakeroot -D -us -uc failed > </snip> > Since i have very little experience with python, its build system, etc > any help would be appreciated. > > Regards, > Harish >

1 0

Re: Need an advice
by Nadav Horesh Dec. 23, 2011

Dec. 23, 2011

Of course. Thank you, Nadav. 2011/12/23 Stéfan van der Walt <stefan(a)sun.ac.za> > On Thu, Dec 22, 2011 at 10:44 PM, Nadav Horesh <nadavh.horesh(a)gmail.com> > wrote: > > The application is not too sophisticated, and I do use numpy, scipy and > > skimage for all I need. The only problem is speed, I need the processing > to > > be at l5-10 time faster. Convolutions take most of the processing time. I > > prefer a flexible solution that would speed up also common nonlinear > filters > > (i.e. a median filter). > > We have a PR in the pipeline for doing really, really fast > convolutions... but it needs a bit of work. Would you like to have a > look at it? > > Stéfan >

1 0