Problem with text preprocessing

gham
gham New Altair Community Member
edited November 5 in Community Q&A

hi

How do I use the stop word dictionary to abbreviate words and replace them with the original words and delete negative words like not?
Where do I download the stopword dictinary?
Thankful

Tagged:

Answers

  • gham
    gham New Altair Community Member

    help me...:smileysad:

  • kypexin
    kypexin New Altair Community Member

    Hi @gham

     

    I suggest you to look at 'Text processing' extension for RapidMiner which contains lots of very useful operators for working with texts; among them, there's a group of operators named 'Filter stopwords' which are doing the exact thing you need, for different languages. 

     

    As for downloading the dictionary, I guess there's no 'unified' dictionary as every text processing task would basically need different stoword list, so this part need to be completed manually. Maybe you can just google and download any ready dictionaries / lists someone has built before and shared online.

  • Telcontar120
    Telcontar120 New Altair Community Member

    If you are using one of the supported languages, there are operators with built-in dictionaries (e.g., English, German, etc.) , so you don't need to download them.  If you are using another language, or if you want a custom stopword dictionary, you will need to create it yourself, or find one on the web and download it.  That's not functionality which is handled inside RapidMiner, but a little bit of web searching will turn up many useful references.

  • sgenzer
    sgenzer
    Altair Employee

    hello @gham Some quick recommendations for you:
    • Post your XML process here in this thread (see https://youtu.be/KkgB5QXWXJ8 and "Read Before Posting" on right when you reply)
    • Attach your dataset if possible (use a fictionalized version if there are privacy concerns)
    • Make sure you have all necessary extensions installed (see https://youtu.be/pjBqG3xtXx4)

    Scott

  • gham
    gham New Altair Community Member

    Please help my friend help me out.

     

  • Thomas_Ott
    Thomas_Ott New Altair Community Member

    @gham did you post your process like what @sgenzer said? You have to do some work here, help us troubleshoot were you're getting stuck. We're not going to buildyour process for you. 

  • gham
    gham New Altair Community Member

    Hello
    How do I remove the negation?

    help me please

    12.JPG

  • sgenzer
    sgenzer
    Altair Employee

    I'm sorry - I have no idea what "remove the negation" means.


    Scott

     

Categories