Image Mining

Question

I have several thousand spare parts each of which has multiple photographs taken from various angles. I want to build a model to identify which spare part is pictured in a new photograph. In data mining terms I want to: Extract features from the images Train a learner to categorize the images based on the features If I use the global features extraction I get good results on a limited sample of training images with this process: A cross-validation evaluating a decision tree model. But I think it will struggle when I expand the number of objects and the real-world image data comes in with wide variations in background colour, perspective, illumination, etc. Having read a little about object recognition and image processing, it would seem that more promising methods of feature extraction for this type of task are SIFT and SURF. I can't find any Rapidminer operator for SIFT but the image mining extension has an operator called descriptor_surf, in the Feature Extraction--->Local Features My problem is that when I run this operator all the output attributes V0 to V63 are Zero! This is the process: 1) Am I correct in thinking that the descriptor_surf is the operator I need ? 2) Why are all the values zero, do I need to pre-process the images or do some other conversion before running it? Thanks, James

jameshickman · Answer

Thanks, I have read a bit more in the are of image recognition understand that I need to extract the POI and then use a region descriptor to capture the features specific to those points. In the SIFT approach the interest points are detected using the DoG (Difference of Gaussians) detector and the region descriptor is a histogram of gradients (HoG) or alternatively CCH (Contrast Context Histograms). I can't find a DoG interest point detector but can identify POI's with the Harris or Hessian detectors. I can extract local features, but none of the local feature extraction operators seem to correspond with HoG or CCH (Although there is a HoG Detector) 1) What is the operator equivalent of HoG in local feature extraction ? 2) What is the operator equivalent of CCH in local feature extraction 3) If neither exist what would be the best choice to for a SIFT like approach ? 4) Does the HoG detector operator work on the whole image or does it extract POI as part of its process (If so, using what algortihm)? Thanks for your help, James

StaryVena · Answer

Hello James,1) Am I correct in thinking that the descriptor_surf is the operator I need ?

There are several types of Interest Points detectors, you can combine them with Local features extractor with PoI. This could be one of approaches, but nobody knows if it will work unless you will try it.2) Why are all the values zero, do I need to pre-processed the images or do some other conversion before running it?

It it wrongly implemented in this old version. It needs defined widow RoI with some scale, but I can't now find any operator which generates these windows.

Best,
Vaclav