Mailman 3 November 2013 - scikit-image

Re: Test coverage
by Chintak Sheth Nov. 20, 2013

Nov. 20, 2013

Hi Ankit Yes, the number of function calls and the lines (?) that have tests for them. Tests are for ensuring that all the lines of code that is written is checked against test cases and are correct. Chintak On Nov 21, 2013 1:31 AM, "Ankit Agrawal" <aaaagrawal(a)gmail.com> wrote: > Hi Stefan, > > Can you elaborate briefly what the coverage percentage exactly > represents? Is it the fraction of API functions that have tests to cover > their back? Or something else? > > Regards, > Ankit Agrawal, > Communication and Signal Processing, > IIT Bombay. > > > On Thu, Nov 21, 2013 at 12:15 AM, Stéfan van der Walt <stefan(a)sun.ac.za>wrote: > >> Hello, everyone >> >> We now have automated test coverage for scikit-image: >> >> https://coveralls.io/r/scikit-image/scikit-image >> >> Most of the newer modules (those that went through rigorous PR >> review!) did pretty well, but we are still only at 82%. If ever you >> find yourself with a bit of spare time on your hands, please hunt down >> some of these corner cases and write tests for them so that we can >> start making our way towards 100%. >> >> Thanks! >> Stéfan >> >> -- >> You received this message because you are subscribed to the Google Groups >> "scikit-image" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to scikit-image+unsubscribe(a)googlegroups.com. >> For more options, visit https://groups.google.com/groups/opt_out. >> > > -- > You received this message because you are subscribed to the Google Groups > "scikit-image" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to scikit-image+unsubscribe(a)googlegroups.com. > For more options, visit https://groups.google.com/groups/opt_out. >

1 0

Re: Test coverage
by Ankit Agrawal Nov. 20, 2013

Nov. 20, 2013

Hi Stefan, Can you elaborate briefly what the coverage percentage exactly represents? Is it the fraction of API functions that have tests to cover their back? Or something else? Regards, Ankit Agrawal, Communication and Signal Processing, IIT Bombay. On Thu, Nov 21, 2013 at 12:15 AM, Stéfan van der Walt <stefan(a)sun.ac.za>wrote: > Hello, everyone > > We now have automated test coverage for scikit-image: > > https://coveralls.io/r/scikit-image/scikit-image > > Most of the newer modules (those that went through rigorous PR > review!) did pretty well, but we are still only at 82%. If ever you > find yourself with a bit of spare time on your hands, please hunt down > some of these corner cases and write tests for them so that we can > start making our way towards 100%. > > Thanks! > Stéfan > > -- > You received this message because you are subscribed to the Google Groups > "scikit-image" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to scikit-image+unsubscribe(a)googlegroups.com. > For more options, visit https://groups.google.com/groups/opt_out. >

1 0

Test coverage
by Stéfan van der Walt Nov. 20, 2013

Nov. 20, 2013

Hello, everyone We now have automated test coverage for scikit-image: https://coveralls.io/r/scikit-image/scikit-image Most of the newer modules (those that went through rigorous PR review!) did pretty well, but we are still only at 82%. If ever you find yourself with a bit of spare time on your hands, please hunt down some of these corner cases and write tests for them so that we can start making our way towards 100%. Thanks! Stéfan

1 0

Re: Advice on improving a analysis on images of nanoparticles
by Adam Hughes Nov. 20, 2013

Nov. 20, 2013

Cool suggestions, thanks! Although I don't have multiple images of the same region of one of our surfaces, I will try it out next time we take images. On Wed, Nov 20, 2013 at 1:15 PM, Stéfan van der Walt <stefan(a)sun.ac.za>wrote: > Hi Adam > > On Wed, Nov 20, 2013 at 8:11 PM, Adam Hughes <hughesadam87(a)gmail.com> > wrote: > > One last thing to add to this thread. We also have the issue that some > > images were taken at low resolution, or low magnification. As a result, > it > > becomes quite difficult to ascertain particle distributions. Unto this > > point, I mostly just made sure to get high quality images, but sometimes > > this is not possible due to microscope-induced artifacts at long image > > times. Have you any suggestions or examples for improving low res > images of > > small particles? Otherwise I'll just note the limitations in resolution > and > > leave it at that in my writeup. > > If you have multiple images of the same object, you could do > super-resolution. If you only have one image, you could try > single-frame super-resolution, which is essentially dictionary > learning on a large set of similar photos, used to update > high-frequency information in the current image. > > Stéfan > > -- > You received this message because you are subscribed to a topic in the > Google Groups "scikit-image" group. > To unsubscribe from this topic, visit > https://groups.google.com/d/topic/scikit-image/EbS7o2HvcUc/unsubscribe. > To unsubscribe from this group and all of its topics, send an email to > scikit-image+unsubscribe(a)googlegroups.com. > For more options, visit https://groups.google.com/groups/opt_out. >

1 0

Re: Advice on improving a analysis on images of nanoparticles
by Stéfan van der Walt Nov. 20, 2013

Nov. 20, 2013

Hi Adam On Wed, Nov 20, 2013 at 8:11 PM, Adam Hughes <hughesadam87(a)gmail.com> wrote: > One last thing to add to this thread. We also have the issue that some > images were taken at low resolution, or low magnification. As a result, it > becomes quite difficult to ascertain particle distributions. Unto this > point, I mostly just made sure to get high quality images, but sometimes > this is not possible due to microscope-induced artifacts at long image > times. Have you any suggestions or examples for improving low res images of > small particles? Otherwise I'll just note the limitations in resolution and > leave it at that in my writeup. If you have multiple images of the same object, you could do super-resolution. If you only have one image, you could try single-frame super-resolution, which is essentially dictionary learning on a large set of similar photos, used to update high-frequency information in the current image. Stéfan

1 0

Re: Advice on improving a analysis on images of nanoparticles
by Adam Hughes Nov. 20, 2013

Nov. 20, 2013

Hey guys, One last thing to add to this thread. We also have the issue that some images were taken at low resolution, or low magnification. As a result, it becomes quite difficult to ascertain particle distributions. Unto this point, I mostly just made sure to get high quality images, but sometimes this is not possible due to microscope-induced artifacts at long image times. Have you any suggestions or examples for improving low res images of small particles? Otherwise I'll just note the limitations in resolution and leave it at that in my writeup. On Wed, Nov 20, 2013 at 1:04 PM, Adam Hughes <hughesadam87(a)gmail.com> wrote: > Hi Juan, thanks for your helpful response. See my replies inline: > > > On Tue, Nov 19, 2013 at 9:39 PM, Juan Nunez-Iglesias <jni.soma(a)gmail.com>wrote: > >> Adam, >> >> On Wed, Nov 20, 2013 at 7:44 AM, Adam Hughes <hughesadam87(a)gmail.com>wrote: >> >>> To download without an account, I am not familiar with any hosting >>> solution, but if you guys have any recommendations I'd love to hear them. >>> >> >> Dropbox can be used for this... >> > > I tried this but to share a link, it asks for the emails of the > recipients. You have used Dropbox to host a publicly accessible link? If > so, I will certainly start doing this, thanks. > > >> >> Thank you for you help, I will test out the methods you suggested >>> >> >> Is the goal only to count particles? In that case, I think a local >> thresholding (threshold_adaptive) would work on all these images. Then, >> just do a labelling (scipy.ndimage.label) and draw a histogram of particle >> sizes. You'll get a sharp peak around the true particle size, with bigger >> peaks for clumps. Once you have the mean particle size you can estimate the >> number of particles in each clump (barring occlusion in 3D, in which case >> you're stuffed anyway), and then the total number of particles in your >> image. >> > > Yes, that is the goal. We had done a similar process ImageJ, but did > thersholding manually. I will read into the adaptive threshold a bit more. > We had hoped that some of these corrections, such as histogram > equilization, would make the automatic threshold more likely to give > correct results. > > >> >> Looking at your images, I don't think watershed (or anything else that I >> know) will do very well with the clumps. The contrast between adjacent >> particles is too low. >> > > Hmm I see. I will still try it out, but thanks for the heads up. I'll > feel better now if it doesn't work well. > > >> >> Low-contrast-4 looks tricky... Are the smaller "points" particles of >> different sizes or just image noise? >> >> Finally, Watershed-f3 also looks hard, because it appears all the >> particles are touching... Again, I don't think watershed will help you >> here, nor anything else that doesn't have an a-priori knowledge of the >> particle size. >> > > We do have an a-prior knowledge actually. What I've been doing already is > putting a lower limit on particle size, with anything under it being noise. > After doing particle counts and binning the data, we fit it with a > guassian, and optionally scale the data so that the guassian is centered > around the mean partitcle diameter (which believe we know to about 3nm > based on TEM imaging and indirect spectroscopic techniques). Based on the > size distribution, we try to further bin the data into small > (dimers/trimers) and large aggregates. For all the particles that are > large enough to be considered an aggregate, we *assume *that they fill a > half-sphere volume, and then we infer the true particle due to these > aggregates. It's pretty ad-hoc, but we certainly apply some knowledge of > the expected particle size distributions. I realize watershedding won't > split up huge clumps, but maybe could assist in the dimers and trimers? In > any case, even if it doesn't significantly enhance our results, it would > still be helpful to explore that option and I'll try it out. > > Thanks for this, and other examples! > > >> Juan. >> >> -- >> You received this message because you are subscribed to a topic in the >> Google Groups "scikit-image" group. >> To unsubscribe from this topic, visit >> https://groups.google.com/d/topic/scikit-image/EbS7o2HvcUc/unsubscribe. >> To unsubscribe from this group and all of its topics, send an email to >> scikit-image+unsubscribe(a)googlegroups.com. >> For more options, visit https://groups.google.com/groups/opt_out. >> > >

1 0

Re: Advice on improving a analysis on images of nanoparticles
by Adam Hughes Nov. 20, 2013

Nov. 20, 2013

Hi Juan, thanks for your helpful response. See my replies inline: On Tue, Nov 19, 2013 at 9:39 PM, Juan Nunez-Iglesias <jni.soma(a)gmail.com>wrote: > Adam, > > On Wed, Nov 20, 2013 at 7:44 AM, Adam Hughes <hughesadam87(a)gmail.com>wrote: > >> To download without an account, I am not familiar with any hosting >> solution, but if you guys have any recommendations I'd love to hear them. >> > > Dropbox can be used for this... > I tried this but to share a link, it asks for the emails of the recipients. You have used Dropbox to host a publicly accessible link? If so, I will certainly start doing this, thanks. > > Thank you for you help, I will test out the methods you suggested >> > > Is the goal only to count particles? In that case, I think a local > thresholding (threshold_adaptive) would work on all these images. Then, > just do a labelling (scipy.ndimage.label) and draw a histogram of particle > sizes. You'll get a sharp peak around the true particle size, with bigger > peaks for clumps. Once you have the mean particle size you can estimate the > number of particles in each clump (barring occlusion in 3D, in which case > you're stuffed anyway), and then the total number of particles in your > image. > Yes, that is the goal. We had done a similar process ImageJ, but did thersholding manually. I will read into the adaptive threshold a bit more. We had hoped that some of these corrections, such as histogram equilization, would make the automatic threshold more likely to give correct results. > > Looking at your images, I don't think watershed (or anything else that I > know) will do very well with the clumps. The contrast between adjacent > particles is too low. > Hmm I see. I will still try it out, but thanks for the heads up. I'll feel better now if it doesn't work well. > > Low-contrast-4 looks tricky... Are the smaller "points" particles of > different sizes or just image noise? > > Finally, Watershed-f3 also looks hard, because it appears all the > particles are touching... Again, I don't think watershed will help you > here, nor anything else that doesn't have an a-priori knowledge of the > particle size. > We do have an a-prior knowledge actually. What I've been doing already is putting a lower limit on particle size, with anything under it being noise. After doing particle counts and binning the data, we fit it with a guassian, and optionally scale the data so that the guassian is centered around the mean partitcle diameter (which believe we know to about 3nm based on TEM imaging and indirect spectroscopic techniques). Based on the size distribution, we try to further bin the data into small (dimers/trimers) and large aggregates. For all the particles that are large enough to be considered an aggregate, we *assume *that they fill a half-sphere volume, and then we infer the true particle due to these aggregates. It's pretty ad-hoc, but we certainly apply some knowledge of the expected particle size distributions. I realize watershedding won't split up huge clumps, but maybe could assist in the dimers and trimers? In any case, even if it doesn't significantly enhance our results, it would still be helpful to explore that option and I'll try it out. Thanks for this, and other examples! > Juan. > > -- > You received this message because you are subscribed to a topic in the > Google Groups "scikit-image" group. > To unsubscribe from this topic, visit > https://groups.google.com/d/topic/scikit-image/EbS7o2HvcUc/unsubscribe. > To unsubscribe from this group and all of its topics, send an email to > scikit-image+unsubscribe(a)googlegroups.com. > For more options, visit https://groups.google.com/groups/opt_out. >

1 0

Re: scikit-image paper
by Stuart Mumford Nov. 20, 2013

Nov. 20, 2013

The paper I have contributed to that uses skimage references the website. Stuart On 20 Nov 2013 17:58, "Stéfan van der Walt" <stefan(a)sun.ac.za> wrote: > On Wed, Nov 20, 2013 at 7:52 PM, Adam Hughes <hughesadam87(a)gmail.com> > wrote: > > Is there currently a preferred way to cite scikit image while this paper > is > > in the works? > > I think unfortunately not until we at least upload the pre-print on arxiv. > > Stéfan > > -- > You received this message because you are subscribed to the Google Groups > "scikit-image" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to scikit-image+unsubscribe(a)googlegroups.com. > For more options, visit https://groups.google.com/groups/opt_out. >

1 0

Re: scikit-image paper
by Stéfan van der Walt Nov. 20, 2013

Nov. 20, 2013

On Wed, Nov 20, 2013 at 7:52 PM, Adam Hughes <hughesadam87(a)gmail.com> wrote: > Is there currently a preferred way to cite scikit image while this paper is > in the works? I think unfortunately not until we at least upload the pre-print on arxiv. Stéfan

1 0

Re: scikit-image paper
by Johannes Schönberger Nov. 20, 2013

Nov. 20, 2013

> Speaking of development practices, I've recently noticed that there's a bit of heterogeneity in the API, with some parts being object-oriented in style and others (most) being functional, which is my preference. Do we want to emphasise that this is the preferred style? The functional style is imo the preferred style, especially for raw image processing functionality, which only takes some image and produces some output. But there are parts where object-oriented style makes a lot of sense. I only know of the following excessive cases: - skimage.transform.GeometricTransforms - skimage.measure.fit.*Models - skimage.viewer And in some places internally, mostly where actually appropriate. But only the interface is relevant in my opinion. I can only speak for the estimation functionality in skimage, where objects help a lot to make a sane implementation - a purely functional interface would be horribly complicated. Maybe it’s easier to keep track of the status of the discussion for everyone with a new PR: https://github.com/scikit-image/scikit-image-paper/pull/3 Please, raise your voice for more ideas and if you are volunteering for a section :-)

2 1