Permutations with string distance for near duplicates detection

tib
tib New Altair Community Member
edited November 5 in Community Q&A
Dear Rapid-Miners,

A RapidMiner newbie needs your help :-)

We are trying to use RapidMiner to analyse a huge dataset stored in an Oracle Database. The data represents organisation (companies) information such as addresses, emails, description, market sector...
The idea would be to compute the similitude (using string distance functions ?) of each of these companies with each other (permutations ?)... The goal being to find the near duplicates in the database.

Would RapidMiner be able to achieve such task ? If yes, how should I procede ? Any help would be really appreciated :)

Thanks a lot

Thibault
Tagged:

Answers

  • el_chief
    el_chief New Altair Community Member
    RapidMiner offers very good consulting services :)

    for an outline of where to start check out my videos on my blog, in particular Text Analytics #4 (links in signature)
  • land
    land New Altair Community Member
    Thanks Neil! :)

    But you are of course right. We offer consulting service to help you with such project setups as this is a little bit too large for starting with RapidMiner from scratch on such a big problem.

    Greetings,
    Sebastian

Categories