Mailman 3 August 2011 - scikit-image

Re: Greyscale reconstruction and merging CellProfiler code
by Tony Yu 21 Aug '11

21 Aug '11

2011/8/21 Stéfan van der Walt <stefan(a)sun.ac.za> > Hi Tony > > On Tue, Aug 9, 2011 at 6:50 AM, Tony Yu <tsyu80(a)gmail.com> wrote: > > P.S. Stéfan: were you able to build the tutorial on my "reconstruction" > > branch? > > This is the result I get (had to fix a small problem in the data_path > first): > > http://mentat.za.net/refer/pp.tar.gz > > Could you verify that that looks right? > Wow, that's weird. The first 4 images (peak_detection-orig, peak_detection-smooth, peak_detection-thresh_bg, peak_detection-peaks) are rendered incorrectly. It looks like the ipython directive is taking a previous mpl figure (apparently from the Hough transform docs) and plotting into one of the subplots of that figure. I guess the state of the ipython directive is maintained across docs, so I should start the doc page by closing all other figures. It sounds like there are plans to decide on a tutorial format at EuroScipy, so I'm inclined to wait until the dust has settled. Also, do you think the morphological reconstruction routine looks general/useful enough to include? Best, -Tony

1 0

Re: Greyscale reconstruction and merging CellProfiler code
by Stéfan van der Walt 21 Aug '11

21 Aug '11

Hi Tony On Tue, Aug 9, 2011 at 6:50 AM, Tony Yu <tsyu80(a)gmail.com> wrote: > P.S. Stéfan: were you able to build the tutorial on my "reconstruction" > branch? This is the result I get (had to fix a small problem in the data_path first): http://mentat.za.net/refer/pp.tar.gz Could you verify that that looks right? Thanks Stéfan

1 0

scikits.image sprints at EuroScipy and SciPy 2011
by Stéfan van der Walt 20 Aug '11

20 Aug '11

Hi everyone Just a heads-up about two upcoming scikits.image sprints. The first will be at SciPy 2011, held in Austin, Tx from 11 to 16 July. Then, we'll do the same at the EuroSciPy conference from 25 to 28 August in Paris. Thank you to Emmanuelle Guillart for taking care of the arrangements! At EuroSciPy, we'll be in the room next to the scikits.learn guys, so there should be some great cross-pollination and collaboration happening. If you're around, come and join us to finalise the 3.0 release! Here are the important dates again: * SciPy: July 11 - July 16 2011, Austin, Tx * EuroSciPy: August 25 - 28 2011, Paris Regards Stéfan

2 3

Re: Greyscale reconstruction and merging CellProfiler code
by Tony Yu 09 Aug '11

09 Aug '11

2011/8/8 Stéfan van der Walt <stefan(a)sun.ac.za> > > Here we go (I had the projects mixed up): > > https://github.com/nipy/nitime/tree/master/doc/examples > > There's also a tools directory that show how they are converted. It's > a quick hack, but it does the job! > In case anyone else is following this discussion, here's a specific example. This python file: https://github.com/nipy/nitime/blob/master/doc/examples/multi_taper_coh.py is rendered like this: http://nipy.sourceforge.net/nitime/examples/multi_taper_coh.html And here's the script that does most of the work generating rst from py files: https://github.com/nipy/nitime/blob/master/tools/ex2rst Should we adopt something like that for now? > I really like this approach. As I mentioned earlier, being able to run the example code directly (instead of having to build the docs to run the code) is a huge plus for me. One thing I like about ipython_directive: they define a "savefig" decorator which saved the current figure and generated rst to add the image. I presume a similar function can be defined for this approach (but I'm not sure I'd miss this feature enough to write the code myself). Cheers, -Tony P.S. Stéfan: were you able to build the tutorial on my "reconstruction" branch?

1 0

Re: Greyscale reconstruction and merging CellProfiler code
by Stéfan van der Walt 09 Aug '11

09 Aug '11

2011/8/7 Stéfan van der Walt <stefan(a)sun.ac.za>: > On Sun, Aug 7, 2011 at 6:35 AM, Tony Yu <tsyu80(a)gmail.com> wrote: >> That's strange: I can't see any missing files. I don't get any missing image warnings from Sphinx, and I don't have any uncommitted files on my end. > > If I look at your Git repo, I don't see noisy_circles.png, for example. > > https://github.com/tonysyu/scikits.image/tree/master/scikits/image/data > >> That sounds like an interesting idea. I especially like the fact you can test *.py files without rebuilding all the docs. I can't find any examples of this on the IPython or PyMVPA github accounts (granted I didn't try *that* hard). Do you have any more info on this approach? > > I can't find them now either, but I'll find out and let you know tomorrow. Here we go (I had the projects mixed up): https://github.com/nipy/nitime/tree/master/doc/examples There's also a tools directory that show how they are converted. It's a quick hack, but it does the job! Should we adopt something like that for now? Ideally, we will soon be able to write IPython worksheets instead. Cheers Stéfan

1 0

Re: Greyscale reconstruction and merging CellProfiler code
by Tony Yu 08 Aug '11

08 Aug '11

2011/8/7 Stéfan van der Walt <stefan(a)sun.ac.za> > On Sun, Aug 7, 2011 at 6:35 AM, Tony Yu <tsyu80(a)gmail.com> wrote: > > That's strange: I can't see any missing files. I don't get any missing > image warnings from Sphinx, and I don't have any uncommitted files on my > end. > > If I look at your Git repo, I don't see noisy_circles.png, for example. > > https://github.com/tonysyu/scikits.image/tree/master/scikits/image/data > Oh, the grey_reconstruction stuff is in a separate branch: https://github.com/tonysyu/scikits.image/tree/reconstruction/scikits/image/… which does have a file named noisy_circles.jpg. It seems like the grey, white and boolean morphology operators share > (or could share) quite a lot of code, and functionally belong > together. The main problem with a proliferation of namespaces is > helping the user to detangle the package. > > How about we just make the grey-level operators default, i.e. > white_tophat etc., and if we then add boolean operators name them > differently? > That sounds good to me. Cheers, -Tony

1 0

Re: Greyscale reconstruction and merging CellProfiler code
by Stéfan van der Walt 08 Aug '11

08 Aug '11

On Sun, Aug 7, 2011 at 6:35 AM, Tony Yu <tsyu80(a)gmail.com> wrote: > That's strange: I can't see any missing files. I don't get any missing image warnings from Sphinx, and I don't have any uncommitted files on my end. If I look at your Git repo, I don't see noisy_circles.png, for example. https://github.com/tonysyu/scikits.image/tree/master/scikits/image/data > That sounds like an interesting idea. I especially like the fact you can test *.py files without rebuilding all the docs. I can't find any examples of this on the IPython or PyMVPA github accounts (granted I didn't try *that* hard). Do you have any more info on this approach? I can't find them now either, but I'll find out and let you know tomorrow. >> Why don't we just shorten those names to morphology.grey_tophat etc.? > > I like the flexibility with namespaces, actually. For example, I could be very verbose and write: > >>> from scikits.image import morphology > >>> morphology.grey.white_tophat(...) > or slightly less verbose and write > >>> from scikits.image.morphology import grey > >>> grey.white_tophat(...) It seems like the grey, white and boolean morphology operators share (or could share) quite a lot of code, and functionally belong together. The main problem with a proliferation of namespaces is helping the user to detangle the package. How about we just make the grey-level operators default, i.e. white_tophat etc., and if we then add boolean operators name them differently? Regards Stéfan

1 0

Re: Greyscale reconstruction and merging CellProfiler code
by Tony Yu 07 Aug '11

07 Aug '11

2011/8/6 Stéfan van der Walt <stefan(a)sun.ac.za> > Hey Tony > > On Sat, Aug 6, 2011 at 3:04 PM, Tony Yu <tsyu80(a)gmail.com> wrote: > > (https://github.com/tonysyu/scikits.image/tree/reconstruction) > > Thanks! I'm very interested to have a look, but I see that one of the > data files (.png) is missing. Could you add that as well? > That's strange: I can't see any missing files. I don't get any missing image warnings from Sphinx, and I don't have any uncommitted files on my end. Also, I checked that all savefig filenames in the tutorial are in the ipython_images directory on the github repo. Was this from a Sphinx warning? Any more clues? > > A couple of random notes: > > * This branch adds the ipython_directive (and > ipython_console_hightlighting) > > sphinx extension, because it allows you to show code examples with text > > in-between (and still be able to access variables from previous blocks). > > No problem. I saw another pretty cool trick that the IPython and > PyMVPA guys use: they write tutorials as .py files, that are then > parsed to produce an rst file. I'm not sure what the best way is to > do this, but their approach does have the advantage that tutorials are > simply .py files, ready for distribution, and that we can provide a > template that is super easy to modify (we really need to encourage > more people to use, test, complain and write about this package). > > What do you think: what's the best way to do tutorials? > That sounds like an interesting idea. I especially like the fact you can test *.py files without rebuilding all the docs. I can't find any examples of this on the IPython or PyMVPA github accounts (granted I didn't try *that* hard). Do you have any more info on this approach? <snip> > > > * the morphology names seem a bit clunky to work with. For example, white > > top hat is called with "morphology.greyscale_white_top_hat". The > greyscale > > routines are already isolated to morphology.grey, so maybe > > morphology/__init__.py can just call "import grey" (instead of "from grey > > import *") and then the "greyscale_" prefix can be removed from the > > functions? > > Why don't we just shorten those names to morphology.grey_tophat etc.? > I like the flexibility with namespaces, actually. For example, I could be very verbose and write: >>> from scikits.image import morphology >>> morphology.grey.white_tophat(...) or slightly less verbose and write >>> from scikits.image.morphology import grey >>> grey.white_tophat(...) or just lazy and write >>> from scikits.image.morphology.grey import * >>> white_tophat(...) (BTW, the "white_" prefix is required because there's also a black_tophat). Plus, "Namespaces are one honking great idea". That's just my preference, though. Best, -Tony

1 0

Re: Greyscale reconstruction and merging CellProfiler code
by Stéfan van der Walt 07 Aug '11

07 Aug '11

Hey Tony On Sat, Aug 6, 2011 at 3:04 PM, Tony Yu <tsyu80(a)gmail.com> wrote: > I'm still interested in getting greyscale reconstruction into scikits.image. > To that end, I've updated my branch with a tutorial demonstrating peak > detection using reconstruction > (https://github.com/tonysyu/scikits.image/tree/reconstruction), with Thanks! I'm very interested to have a look, but I see that one of the data files (.png) is missing. Could you add that as well? > A couple of random notes: > * This branch adds the ipython_directive (and ipython_console_hightlighting) > sphinx extension, because it allows you to show code examples with text > in-between (and still be able to access variables from previous blocks). No problem. I saw another pretty cool trick that the IPython and PyMVPA guys use: they write tutorials as .py files, that are then parsed to produce an rst file. I'm not sure what the best way is to do this, but their approach does have the advantage that tutorials are simply .py files, ready for distribution, and that we can provide a template that is super easy to modify (we really need to encourage more people to use, test, complain and write about this package). What do you think: what's the best way to do tutorials? > These extensions require the latest ipython (i.e. 0.11). > * morphology.greyscale_white_top_hat has issues with underflow because > there's a subtraction in the algorithm and it requires uint8 images (I > think). A similar issue came up in this > thread: http://groups.google.com/group/scikits-image/browse_thread/thread/c081f18d9…. > But I guess, that applied specifically to functions using ndimage.convolve. We should fix any such problems where we can. I should write up the proposed guidelines for usage of different image dtypes, but it comes down to interpreting the different types in a set way (uint8->[0,255], int8->[-127,127], etc.), providing the necessary utility functions, and allowing functions to specify what type of input they allow. Output can be whatever the functions can compute most efficiently, as long as it is well documented. Users can also then call a utility function such as img_as_uint8 to convert to the specified format. > * the morphology names seem a bit clunky to work with. For example, white > top hat is called with "morphology.greyscale_white_top_hat". The greyscale > routines are already isolated to morphology.grey, so maybe > morphology/__init__.py can just call "import grey" (instead of "from grey > import *") and then the "greyscale_" prefix can be removed from the > functions? Why don't we just shorten those names to morphology.grey_tophat etc.? Looking forward to reading your tutorial! Regards Stéfan

1 0

Re: Greyscale reconstruction and merging CellProfiler code
by Tony Yu 07 Aug '11

07 Aug '11

2011/7/10 Stéfan van der Walt <stefan(a)sun.ac.za> > On Thu, Jul 7, 2011 at 6:30 PM, Tony Yu <tsyu80(a)gmail.com> wrote: > > I added a reference and tried to expand on the description, but the > > description is still poor. Even in the papers I've read, the descriptions > > are quite difficult for me to understand (the figures in those papers > really > > help). I've also added an example to the docstring, but it's quite ... > > verbose. > > I guess the question then becomes whether such a specialised algorithm > is of enough common interest to include it. In the end, every piece > of code requires maintenance, and our aim is to provide a fairly > generic set of tools that can be used to construct image processing > experiments. > > I don't know this algorithm specifically, so I can't say, but it is an > aspect worth considering. > > > I'd like to add a longer example in the docs, but that will have to wait > > until I have a bit more time. When I do add it, where should put it? I'd > > like for it to be more like a tutorial than a function-specific example > > (specifically, I'd like to compare reconstruction and top-hat for peak > > detection). > > We are eagerly awaiting tutorial contributions for the docs, but no > one's volunteered so far :) I'd love to have more of these, as it > shows common patterns (such as how to use io, displaying images, > etc.). > > Regards > Stéfan > Hi again, I'm still interested in getting greyscale reconstruction into scikits.image. To that end, I've updated my branch with a tutorial demonstrating peak detection using reconstruction ( https://github.com/tonysyu/scikits.image/tree/reconstruction), with comparisons to thresholding and white top hat. This example should demonstrate reconstruction better than I was able to explain it earlier. Let me know if you have trouble building the docs. There's still some changes I want to make, but first I want to know whether this example is enough to convince you that the reconstruction algorithm is worth including ;) Best, - Tony A couple of random notes: * This branch adds the ipython_directive (and ipython_console_hightlighting) sphinx extension, because it allows you to show code examples with text in-between (and still be able to access variables from previous blocks). These extensions require the latest ipython (i.e. 0.11). * morphology.greyscale_white_top_hat has issues with underflow because there's a subtraction in the algorithm and it requires uint8 images (I think). A similar issue came up in this thread: http://groups.google.com/group/scikits-image/browse_thread/thread/c081f18d9…. But I guess, that applied specifically to functions using ndimage.convolve. * the morphology names seem a bit clunky to work with. For example, white top hat is called with "morphology.greyscale_white_top_hat". The greyscale routines are already isolated to morphology.grey, so maybe morphology/__init__.py can just call "import grey" (instead of "from grey import *") and then the "greyscale_" prefix can be removed from the functions?

1 0