text classification with dialect in Arabic language

Badr
Badr New Altair Community Member
edited November 5 in Community Q&A
I made text classification for Arabic and it works very well with standard Arabic language but now I will use   with dataset with different dialect in Arabic ? can I use same operators( tokenize and stem(Arabic)  and stopword 
Tagged:

Best Answer

  • Telcontar120
    Telcontar120 New Altair Community Member
    Answer ✓
    It should as long as the underlying characters are not different, even though the vocabulary, syntax and usage may vary in the dialect.  RapidMiner (or any NLP algorithm) doesn't really understand languages, it just transforms them all into numerical representations to manipulate.

Answers

  • Telcontar120
    Telcontar120 New Altair Community Member
    Answer ✓
    It should as long as the underlying characters are not different, even though the vocabulary, syntax and usage may vary in the dialect.  RapidMiner (or any NLP algorithm) doesn't really understand languages, it just transforms them all into numerical representations to manipulate.

Categories