Feature Detectors and Descriptors in scikit-image

[Anders Boesen Lindbo Larsen]

On Friday, June 7, 2013 3:32:25 PM UTC+2, Ankit Agrawal wrote:

> Hi all,
>
>        I have some queries about feature Detectors and Descriptors that I 
> have to implement as a part of my GSoC project<https://github.com/scikit-image/scikit-image/wiki/GSoC-2013-Ankit-Agrawal-Implementation-of-STAR-and-Binary-Feature-Detectors-and-Descriptors>
> .
>
>        Practically, a descriptor is more useful around feature points that 
> are more distinctive in nature and hence will then produce a greater 
> accuracy for the tasks where descriptors are used - Correspondence matching 
> in two image frames for Stereo, Image Alignment, Object Recognition and 
> Tracking etc. On the same note, the feature descriptors in OpenCV take 
> input argument as a Keypoint object which is mainly a vector of 
> keypoints(extracted using a feature detector). OpenCV thus has classes for 
> Keypoint<http://docs.opencv.org/modules/features2d/doc/common_interfaces_of_feature_detectors.html#keypoint>, 
> Feature Detector<http://docs.opencv.org/modules/features2d/doc/common_interfaces_of_feature_detectors.html#featuredetector>, 
> DescriptorExtractor<http://docs.opencv.org/modules/features2d/doc/common_interfaces_of_descriptor_extractors.html>etc. This enables the flexibility of using any FeatureDescriptor an 
> keypoints extracted using any FeatureDetector.
>
>        I took a look at the implementation<https://github.com/scikit-image/scikit-image/blob/master/skimage/feature/_daisy.py>of Daisy feature descriptor in skimage and noticed that it finds 
> descriptors around points that are spread uniformly with density based on 
> step argument as the input. For this I checked its paper<http://www.ele.puc-rio.br/~visao/Topicos/Tola%202010%20Efficient%20Dense%20Descriptor%20Applied%20to%20Wide%20Baseline%20Stereo.pdf>(Pg 
> 3, section 3) and it said it can be used around feature-points as well as 
> non feature-points. Almost all the Feature Descriptors(that I know of) 
> including the ones that I am going to implement are calculated about 
> keypoints. Because of the above reasons, I think an option should be 
> provided in the functions of feature detectors to return the output as a 
> vector containing location of feature-points. I would like to know the 
> views/suggestions of community members experienced in this part of Computer 
> Vision on this point and to suggest the best possible data-flow between 
> functions of Feature Detectors and Feature Descriptors. Thank you.
>

Feature description is a messy business - there is little consensus in the 
literature and in the implementations available! 

For an overview of the feature extraction pipeline, I recommend reading 
until and including section 2.3.2 in
http://www.vlfeat.org/~vedaldi/assets/pubs/vedaldi10knowing.pdf
Here, different types of interest points are described (disk, oriented 
disk, ellipse, etc.). Moreover, the feature description pipeline is divided 
into 3 steps (detection, canonization, description). This means that for 
each interest point type, you will have to make a canonization method that 
can bring the underlying image patch can to a form suitable for the 
description algorithm, e.g. a 64x64 image patch.
I recommend this approach because it is more flexible than if the detection 
and description code is combined as it is done in e.g. SIFT. However, I 
should mention that the approach is not ideal for 2 reasons:
  - It requires more computations. In SIFT, the scale-space pyramid 
generated in the detection step can be reused for description. 
  - The canonization step introduces noise because we typically will have 
to warp the image.

I hope some of it made sense. Returning to your question on the data flow 
between detectors and descriptors: I would recommend making the detectors 
return a list of interest points. This list of interest points can then be 
given to a descriptor function. It is up to the descriptor to canonize the 
interest points if needed.

BTW, some time ago I wrote some code to canonize an affine interest point 
(ellipse):
https://github.com/andersbll/jetdesc/blob/master/util.py#L50
Feel free to copy-paste whatever you might find useful in that repository. 
:)

Cheers,
Anders
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/scikit-image/attachments/20130607/db76d68e/attachment.html>