🎉Community Raffle - Win $25

An exclusive raffle opportunity for active members like you! Complete your profile, answer questions and get your first accepted badge to enter the raffle.
Join and Win

Text Mining- Select Token based on Dictionary File

User: "mbjasser"
New Altair Community Member
Updated by Jocelyn
Hi every one,

I'm trying to work on a text mining workflow to filter specific language contents based on  a specific language dictionary (TXT file most probably).

I was able to filter stopwords using the operator "Filter Stopwords (Dictionary)" to filter the content depending on a dictionary, but I'm still trying to select tokens based also on a dictionary, but it seems that the only operator offered is Filter Tokens (by contents) (which enables selecting tokens based on a regular expression, there is no option for selecting tokens based on a dictionary file).

I need your support if you have an idea if there exists any operator to do that, or if I'm missing something.

Thank you in advance

Find more posts tagged with

Sort by:
1 - 5 of 51
    User: "JEdward"
    New Altair Community Member
    Seconded. 

    See my post here: http://rapid-i.com/rapidforum/index.php/topic,8008.msg27328.html#msg27328 suggesting that a group of us band together to pay for RM to improve language support in the text extension.  Particularly for more difficult languages such as Arabic, Chinese, Indonesian or even TxtSpk. 
    User: "fras"
    New Altair Community Member
    There is a operator "Filter Stopwords" where you can place a
    txt file with one stopword per line.
    User: "mbjasser"
    New Altair Community Member
    OP
    Thanks fras, I'm already using the operator of Filter Stop words based on dictionary, but this operator is filtering words, I want to select tokens based on dictionary not filtering (removing). I'm not able to find any up to now. The only option offered is filter tokens by content using the regular option field.

    User: "mbjasser"
    New Altair Community Member
    OP
    it's god idea JEdward, hopefully it'll work
    User: "JEdward"
    New Altair Community Member
    Yes, definitely something I'm going to follow up on in the next couple of weeks.