Mailman 3 August 2011 - scikit-image

Re: Progress towards the next release
by Chris Colbert Aug. 30, 2011

Aug. 30, 2011

What? It is was a clean patch :) On Aug 30, 2011 6:36 AM, "Stéfan van der Walt" <stefan(a)sun.ac.za> wrote: > Hi all, > > We've made some progress at the sprint last night towards the 0.3 > release. Thanks go out to Emmanuelle, Gaël, Thouis, David as well as > Chris (whom we never saw, but who merged a patch silently :). > > I'll be travelling for the next 10 days, but thereafter I plan to > round this up and cut the release. Here's what's cooking: > > - Emmanuelle is incorporating cellprofiler morphology funcs (watershed & others) > - Stefan is creating the Debian packages, sorting out the windows > builder, and writing up the dtype overview > - Thouis & Kyle are completing the functionality overview table > - Gael is improving the implementation of TV denoise using fista > - David is setting up our bento build script > - The team (that's you!) is reviewing Pieter's Google Summer of Code patches > > We need to hear from you! If there's anything you need to get in > before the release, now's a good time. > > Also, we need a new logo for the website--I know someone must be able > to draw prettier pictures than me, so don't be shy! > > http://github.com/scikits-image > > Regards > Stéfan

1 0

Progress towards the next release
by Stéfan van der Walt Aug. 30, 2011

Aug. 30, 2011

Hi all, We've made some progress at the sprint last night towards the 0.3 release. Thanks go out to Emmanuelle, Gaël, Thouis, David as well as Chris (whom we never saw, but who merged a patch silently :). I'll be travelling for the next 10 days, but thereafter I plan to round this up and cut the release. Here's what's cooking: - Emmanuelle is incorporating cellprofiler morphology funcs (watershed & others) - Stefan is creating the Debian packages, sorting out the windows builder, and writing up the dtype overview - Thouis & Kyle are completing the functionality overview table - Gael is improving the implementation of TV denoise using fista - David is setting up our bento build script - The team (that's you!) is reviewing Pieter's Google Summer of Code patches We need to hear from you! If there's anything you need to get in before the release, now's a good time. Also, we need a new logo for the website--I know someone must be able to draw prettier pictures than me, so don't be shy! http://github.com/scikits-image Regards Stéfan

1 0

Re: Removing OpenCV wrappers
by Pieter Holtzhausen Aug. 29, 2011

Aug. 29, 2011

Our version can prove as example of how it should have been done :) but yes the opencv wrapper now offers the same functionality while being more thorough and more actively maintained. 2011/8/29 Stéfan van der Walt <stefan(a)sun.ac.za>: > Hi all, > > The OpenCV wrappers currently in scikits.image did a great job, but > since the OpenCV project's wrappers have now caught up (in terms of > not copying data, etc.), it makes sense to deprecate ours for removal > in the next release. > > Please let me know if you have any thoughts on the topic. > > Regards > Stéfan >

1 0

Removing OpenCV wrappers
by Stéfan van der Walt Aug. 29, 2011

Aug. 29, 2011

Hi all, The OpenCV wrappers currently in scikits.image did a great job, but since the OpenCV project's wrappers have now caught up (in terms of not copying data, etc.), it makes sense to deprecate ours for removal in the next release. Please let me know if you have any thoughts on the topic. Regards Stéfan

1 0

Re: Usage of and relationship to OpenCV
by Thouis (Ray) Jones Aug. 29, 2011

Aug. 29, 2011

2011/8/29 Stéfan van der Walt <stefan(a)sun.ac.za>: > Should we declare OpenCV a dependency? If it's made a dependency, is it possible to automate the process of fetching and installing it (in a directory local to the package?) as part of the install process? Only if it's not installed already, of course. I think I would prefer: - not making it a dependency - adding code to (optionally) download and install it as part of the scikit install process - some functionality might require it be installed. Eventually, slower non-opencv versions might be added. Ray

1 0

Re: Usage of and relationship to OpenCV
by Emmanuelle Gouillart Aug. 29, 2011

Aug. 29, 2011

Hi, having opencv as a dependency has two important drawbacks for me. First, it makes the installation of the scikit more difficult. Second, the documentation of openCV does not match the standards of numpy/scipy docstrings (no examples, few references to articles). Performance is great, but I think we'll gain more users by putting effort on user-friendliness than if we put it on performance. For examples, a large fraction of my colleagues are using image processing, but on Windows, and they get discouraged by a package with unsufficient documentation. I'm at the Python for Physics day now and I don't have the time to write more, but I'll be glad to continue the discussion later. Cheers, Emmanuelle On Mon, Aug 29, 2011 at 11:25:06AM +0200, StÃ¯Â¿Â½fan van der Walt wrote: > Hi all, > After merging the GSoC contributions, I think we're in good shape for > a next release. Before we get there, however, I'd like to ask your > opinions. > Should we declare OpenCV a dependency? There are two sides to this: > 1) I want to stop focusing on optimising low-level routines, and > instead implement interesting algorithms but 2) OpenCV is quite a big > dependency (although it does run on Windows, Linux, Mac etc, comes > with built binaries in all of them, and has a permissive BSD license). > Fact is, our manpower is limited, so we should focus on where we can > make a difference. My aim is to provide an excellent image processing > toolbox for Python that at least covers the MATLAB functionality and > more, and to reach that goal with our limited manpower, we need to > carefully pick our strategy. > I'd be glad to hear what you have to say. > Regards > StÃ¯Â¿Â½fan

2 1

Report from EuroSciPy
by Stéfan van der Walt Aug. 29, 2011

Aug. 29, 2011

Hi everyone, A quick report from EuroSciPy: - Ray Jones gave a great talk about CellProfiler. As you may recall, CellProfiler opened their code under a BSD license so that we may include it in the scikit. - There will be a sprint this evening at 19:30 Paris time to move us closer to our 0.3 release. Emmanuelle, Ray and myself will be sprinting from Paris, and you are all invited to join via IRC or chat. - We've had good conversations with developers of other image processing packages, so let's hope for some more contributors! Stéfan

1 0

Usage of and relationship to OpenCV
by Stéfan van der Walt Aug. 29, 2011

Aug. 29, 2011

Hi all, After merging the GSoC contributions, I think we're in good shape for a next release. Before we get there, however, I'd like to ask your opinions. Should we declare OpenCV a dependency? There are two sides to this: 1) I want to stop focusing on optimising low-level routines, and instead implement interesting algorithms but 2) OpenCV is quite a big dependency (although it does run on Windows, Linux, Mac etc, comes with built binaries in all of them, and has a permissive BSD license). Fact is, our manpower is limited, so we should focus on where we can make a difference. My aim is to provide an excellent image processing toolbox for Python that at least covers the MATLAB functionality and more, and to reach that goal with our limited manpower, we need to carefully pick our strategy. I'd be glad to hear what you have to say. Regards Stéfan

1 0

Re: Greyscale reconstruction and merging CellProfiler code
by Stéfan van der Walt Aug. 21, 2011

Aug. 21, 2011

On Sun, Aug 21, 2011 at 7:21 AM, Tony Yu <tsyu80(a)gmail.com> wrote: > It sounds like there are plans to decide on a tutorial format at EuroScipy, > so I'm inclined to wait until the dust has settled. Also, do you think the > morphological reconstruction routine looks general/useful enough to include? I'll have to study the technique in a bit more detail. I'd like to compare it with the Discrete Pulse Transform, for example: http://dip.sun.ac.za/~stefan/dpt/ http://scholar.sun.ac.za/handle/10019.1/4883 Regards Stéfan

1 0

Re: scikits.image sprints at EuroScipy and SciPy 2011
by Emmanuelle Gouillart Aug. 21, 2011

Aug. 21, 2011

Hi StÃ¯Â¿Â½fan! > I'll definitely be there. I think one of the main aims of the sprint > for me will be to Great news! When do you arrive in Paris? > - Merge and test Pieter Holtzhausen's GSoC changes > - Add tools for easy writing of tutorials (we'll borrow from nipy, perhaps) > - Fill out the MATLAB compatibility chart that Kyle Mandli > implemented at SciPy2011 > - Prepare for the release of 0.3 OK, I'll try to give you a hand for this. As for me, I'm particularly interested in writing documentation (examples) and merging code from CellProfiler. > > Let's start: I'm free on Tuesday 23rd and Wednesday 24th; I'll have much > > less time during the conference itself, being one of the organizers. > I'll definitely be there those days as well. I'm teaching during the > conference, but apart from that I'll work on this and IPython. Great; in fact I'll have to be at work either on Tuesday morning or Wednesday morning, but I'll be at the Ecole Normale SupÃ¯Â¿Â½rieure (Physics Department) for the rest of the time. Anybody else? (MaÃ¯Â¿Â½l?)

2 1

Re: Greyscale reconstruction and merging CellProfiler code
by Tony Yu Aug. 21, 2011

Aug. 21, 2011

2011/8/21 Stéfan van der Walt <stefan(a)sun.ac.za> > Hi Tony > > On Tue, Aug 9, 2011 at 6:50 AM, Tony Yu <tsyu80(a)gmail.com> wrote: > > P.S. Stéfan: were you able to build the tutorial on my "reconstruction" > > branch? > > This is the result I get (had to fix a small problem in the data_path > first): > > http://mentat.za.net/refer/pp.tar.gz > > Could you verify that that looks right? > Wow, that's weird. The first 4 images (peak_detection-orig, peak_detection-smooth, peak_detection-thresh_bg, peak_detection-peaks) are rendered incorrectly. It looks like the ipython directive is taking a previous mpl figure (apparently from the Hough transform docs) and plotting into one of the subplots of that figure. I guess the state of the ipython directive is maintained across docs, so I should start the doc page by closing all other figures. It sounds like there are plans to decide on a tutorial format at EuroScipy, so I'm inclined to wait until the dust has settled. Also, do you think the morphological reconstruction routine looks general/useful enough to include? Best, -Tony

1 0

Re: Greyscale reconstruction and merging CellProfiler code
by Stéfan van der Walt Aug. 21, 2011

Aug. 21, 2011

Hi Tony On Tue, Aug 9, 2011 at 6:50 AM, Tony Yu <tsyu80(a)gmail.com> wrote: > P.S. Stéfan: were you able to build the tutorial on my "reconstruction" > branch? This is the result I get (had to fix a small problem in the data_path first): http://mentat.za.net/refer/pp.tar.gz Could you verify that that looks right? Thanks Stéfan

1 0

scikits.image sprints at EuroScipy and SciPy 2011
by Stéfan van der Walt Aug. 19, 2011

Aug. 19, 2011

Hi everyone Just a heads-up about two upcoming scikits.image sprints. The first will be at SciPy 2011, held in Austin, Tx from 11 to 16 July. Then, we'll do the same at the EuroSciPy conference from 25 to 28 August in Paris. Thank you to Emmanuelle Guillart for taking care of the arrangements! At EuroSciPy, we'll be in the room next to the scikits.learn guys, so there should be some great cross-pollination and collaboration happening. If you're around, come and join us to finalise the 3.0 release! Here are the important dates again: * SciPy: July 11 - July 16 2011, Austin, Tx * EuroSciPy: August 25 - 28 2011, Paris Regards Stéfan

2 3

Re: Greyscale reconstruction and merging CellProfiler code
by Tony Yu Aug. 9, 2011

Aug. 9, 2011

2011/8/8 Stéfan van der Walt <stefan(a)sun.ac.za> > > Here we go (I had the projects mixed up): > > https://github.com/nipy/nitime/tree/master/doc/examples > > There's also a tools directory that show how they are converted. It's > a quick hack, but it does the job! > In case anyone else is following this discussion, here's a specific example. This python file: https://github.com/nipy/nitime/blob/master/doc/examples/multi_taper_coh.py is rendered like this: http://nipy.sourceforge.net/nitime/examples/multi_taper_coh.html And here's the script that does most of the work generating rst from py files: https://github.com/nipy/nitime/blob/master/tools/ex2rst Should we adopt something like that for now? > I really like this approach. As I mentioned earlier, being able to run the example code directly (instead of having to build the docs to run the code) is a huge plus for me. One thing I like about ipython_directive: they define a "savefig" decorator which saved the current figure and generated rst to add the image. I presume a similar function can be defined for this approach (but I'm not sure I'd miss this feature enough to write the code myself). Cheers, -Tony P.S. Stéfan: were you able to build the tutorial on my "reconstruction" branch?

1 0

Re: Greyscale reconstruction and merging CellProfiler code
by Stéfan van der Walt Aug. 8, 2011

Aug. 8, 2011

2011/8/7 Stéfan van der Walt <stefan(a)sun.ac.za>: > On Sun, Aug 7, 2011 at 6:35 AM, Tony Yu <tsyu80(a)gmail.com> wrote: >> That's strange: I can't see any missing files. I don't get any missing image warnings from Sphinx, and I don't have any uncommitted files on my end. > > If I look at your Git repo, I don't see noisy_circles.png, for example. > > https://github.com/tonysyu/scikits.image/tree/master/scikits/image/data > >> That sounds like an interesting idea. I especially like the fact you can test *.py files without rebuilding all the docs. I can't find any examples of this on the IPython or PyMVPA github accounts (granted I didn't try *that* hard). Do you have any more info on this approach? > > I can't find them now either, but I'll find out and let you know tomorrow. Here we go (I had the projects mixed up): https://github.com/nipy/nitime/tree/master/doc/examples There's also a tools directory that show how they are converted. It's a quick hack, but it does the job! Should we adopt something like that for now? Ideally, we will soon be able to write IPython worksheets instead. Cheers Stéfan

1 0

Re: Greyscale reconstruction and merging CellProfiler code
by Tony Yu Aug. 7, 2011

Aug. 7, 2011

2011/8/7 Stéfan van der Walt <stefan(a)sun.ac.za> > On Sun, Aug 7, 2011 at 6:35 AM, Tony Yu <tsyu80(a)gmail.com> wrote: > > That's strange: I can't see any missing files. I don't get any missing > image warnings from Sphinx, and I don't have any uncommitted files on my > end. > > If I look at your Git repo, I don't see noisy_circles.png, for example. > > https://github.com/tonysyu/scikits.image/tree/master/scikits/image/data > Oh, the grey_reconstruction stuff is in a separate branch: https://github.com/tonysyu/scikits.image/tree/reconstruction/scikits/image/… which does have a file named noisy_circles.jpg. It seems like the grey, white and boolean morphology operators share > (or could share) quite a lot of code, and functionally belong > together. The main problem with a proliferation of namespaces is > helping the user to detangle the package. > > How about we just make the grey-level operators default, i.e. > white_tophat etc., and if we then add boolean operators name them > differently? > That sounds good to me. Cheers, -Tony

1 0

Re: Greyscale reconstruction and merging CellProfiler code
by Stéfan van der Walt Aug. 7, 2011

Aug. 7, 2011

On Sun, Aug 7, 2011 at 6:35 AM, Tony Yu <tsyu80(a)gmail.com> wrote: > That's strange: I can't see any missing files. I don't get any missing image warnings from Sphinx, and I don't have any uncommitted files on my end. If I look at your Git repo, I don't see noisy_circles.png, for example. https://github.com/tonysyu/scikits.image/tree/master/scikits/image/data > That sounds like an interesting idea. I especially like the fact you can test *.py files without rebuilding all the docs. I can't find any examples of this on the IPython or PyMVPA github accounts (granted I didn't try *that* hard). Do you have any more info on this approach? I can't find them now either, but I'll find out and let you know tomorrow. >> Why don't we just shorten those names to morphology.grey_tophat etc.? > > I like the flexibility with namespaces, actually. For example, I could be very verbose and write: > >>> from scikits.image import morphology > >>> morphology.grey.white_tophat(...) > or slightly less verbose and write > >>> from scikits.image.morphology import grey > >>> grey.white_tophat(...) It seems like the grey, white and boolean morphology operators share (or could share) quite a lot of code, and functionally belong together. The main problem with a proliferation of namespaces is helping the user to detangle the package. How about we just make the grey-level operators default, i.e. white_tophat etc., and if we then add boolean operators name them differently? Regards Stéfan

1 0

Re: Greyscale reconstruction and merging CellProfiler code
by Tony Yu Aug. 7, 2011

Aug. 7, 2011

2011/8/6 Stéfan van der Walt <stefan(a)sun.ac.za> > Hey Tony > > On Sat, Aug 6, 2011 at 3:04 PM, Tony Yu <tsyu80(a)gmail.com> wrote: > > (https://github.com/tonysyu/scikits.image/tree/reconstruction) > > Thanks! I'm very interested to have a look, but I see that one of the > data files (.png) is missing. Could you add that as well? > That's strange: I can't see any missing files. I don't get any missing image warnings from Sphinx, and I don't have any uncommitted files on my end. Also, I checked that all savefig filenames in the tutorial are in the ipython_images directory on the github repo. Was this from a Sphinx warning? Any more clues? > > A couple of random notes: > > * This branch adds the ipython_directive (and > ipython_console_hightlighting) > > sphinx extension, because it allows you to show code examples with text > > in-between (and still be able to access variables from previous blocks). > > No problem. I saw another pretty cool trick that the IPython and > PyMVPA guys use: they write tutorials as .py files, that are then > parsed to produce an rst file. I'm not sure what the best way is to > do this, but their approach does have the advantage that tutorials are > simply .py files, ready for distribution, and that we can provide a > template that is super easy to modify (we really need to encourage > more people to use, test, complain and write about this package). > > What do you think: what's the best way to do tutorials? > That sounds like an interesting idea. I especially like the fact you can test *.py files without rebuilding all the docs. I can't find any examples of this on the IPython or PyMVPA github accounts (granted I didn't try *that* hard). Do you have any more info on this approach? <snip> > > > * the morphology names seem a bit clunky to work with. For example, white > > top hat is called with "morphology.greyscale_white_top_hat". The > greyscale > > routines are already isolated to morphology.grey, so maybe > > morphology/__init__.py can just call "import grey" (instead of "from grey > > import *") and then the "greyscale_" prefix can be removed from the > > functions? > > Why don't we just shorten those names to morphology.grey_tophat etc.? > I like the flexibility with namespaces, actually. For example, I could be very verbose and write: >>> from scikits.image import morphology >>> morphology.grey.white_tophat(...) or slightly less verbose and write >>> from scikits.image.morphology import grey >>> grey.white_tophat(...) or just lazy and write >>> from scikits.image.morphology.grey import * >>> white_tophat(...) (BTW, the "white_" prefix is required because there's also a black_tophat). Plus, "Namespaces are one honking great idea". That's just my preference, though. Best, -Tony

1 0

Re: Greyscale reconstruction and merging CellProfiler code
by Stéfan van der Walt Aug. 7, 2011

Aug. 7, 2011

Hey Tony On Sat, Aug 6, 2011 at 3:04 PM, Tony Yu <tsyu80(a)gmail.com> wrote: > I'm still interested in getting greyscale reconstruction into scikits.image. > To that end, I've updated my branch with a tutorial demonstrating peak > detection using reconstruction > (https://github.com/tonysyu/scikits.image/tree/reconstruction), with Thanks! I'm very interested to have a look, but I see that one of the data files (.png) is missing. Could you add that as well? > A couple of random notes: > * This branch adds the ipython_directive (and ipython_console_hightlighting) > sphinx extension, because it allows you to show code examples with text > in-between (and still be able to access variables from previous blocks). No problem. I saw another pretty cool trick that the IPython and PyMVPA guys use: they write tutorials as .py files, that are then parsed to produce an rst file. I'm not sure what the best way is to do this, but their approach does have the advantage that tutorials are simply .py files, ready for distribution, and that we can provide a template that is super easy to modify (we really need to encourage more people to use, test, complain and write about this package). What do you think: what's the best way to do tutorials? > These extensions require the latest ipython (i.e. 0.11). > * morphology.greyscale_white_top_hat has issues with underflow because > there's a subtraction in the algorithm and it requires uint8 images (I > think). A similar issue came up in this > thread: http://groups.google.com/group/scikits-image/browse_thread/thread/c081f18d9…. > But I guess, that applied specifically to functions using ndimage.convolve. We should fix any such problems where we can. I should write up the proposed guidelines for usage of different image dtypes, but it comes down to interpreting the different types in a set way (uint8->[0,255], int8->[-127,127], etc.), providing the necessary utility functions, and allowing functions to specify what type of input they allow. Output can be whatever the functions can compute most efficiently, as long as it is well documented. Users can also then call a utility function such as img_as_uint8 to convert to the specified format. > * the morphology names seem a bit clunky to work with. For example, white > top hat is called with "morphology.greyscale_white_top_hat". The greyscale > routines are already isolated to morphology.grey, so maybe > morphology/__init__.py can just call "import grey" (instead of "from grey > import *") and then the "greyscale_" prefix can be removed from the > functions? Why don't we just shorten those names to morphology.grey_tophat etc.? Looking forward to reading your tutorial! Regards Stéfan

1 0

Re: Greyscale reconstruction and merging CellProfiler code
by Tony Yu Aug. 6, 2011

Aug. 6, 2011

2011/7/10 Stéfan van der Walt <stefan(a)sun.ac.za> > On Thu, Jul 7, 2011 at 6:30 PM, Tony Yu <tsyu80(a)gmail.com> wrote: > > I added a reference and tried to expand on the description, but the > > description is still poor. Even in the papers I've read, the descriptions > > are quite difficult for me to understand (the figures in those papers > really > > help). I've also added an example to the docstring, but it's quite ... > > verbose. > > I guess the question then becomes whether such a specialised algorithm > is of enough common interest to include it. In the end, every piece > of code requires maintenance, and our aim is to provide a fairly > generic set of tools that can be used to construct image processing > experiments. > > I don't know this algorithm specifically, so I can't say, but it is an > aspect worth considering. > > > I'd like to add a longer example in the docs, but that will have to wait > > until I have a bit more time. When I do add it, where should put it? I'd > > like for it to be more like a tutorial than a function-specific example > > (specifically, I'd like to compare reconstruction and top-hat for peak > > detection). > > We are eagerly awaiting tutorial contributions for the docs, but no > one's volunteered so far :) I'd love to have more of these, as it > shows common patterns (such as how to use io, displaying images, > etc.). > > Regards > Stéfan > Hi again, I'm still interested in getting greyscale reconstruction into scikits.image. To that end, I've updated my branch with a tutorial demonstrating peak detection using reconstruction ( https://github.com/tonysyu/scikits.image/tree/reconstruction), with comparisons to thresholding and white top hat. This example should demonstrate reconstruction better than I was able to explain it earlier. Let me know if you have trouble building the docs. There's still some changes I want to make, but first I want to know whether this example is enough to convince you that the reconstruction algorithm is worth including ;) Best, - Tony A couple of random notes: * This branch adds the ipython_directive (and ipython_console_hightlighting) sphinx extension, because it allows you to show code examples with text in-between (and still be able to access variables from previous blocks). These extensions require the latest ipython (i.e. 0.11). * morphology.greyscale_white_top_hat has issues with underflow because there's a subtraction in the algorithm and it requires uint8 images (I think). A similar issue came up in this thread: http://groups.google.com/group/scikits-image/browse_thread/thread/c081f18d9…. But I guess, that applied specifically to functions using ndimage.convolve. * the morphology names seem a bit clunky to work with. For example, white top hat is called with "morphology.greyscale_white_top_hat". The greyscale routines are already isolated to morphology.grey, so maybe morphology/__init__.py can just call "import grey" (instead of "from grey import *") and then the "greyscale_" prefix can be removed from the functions?

1 0