"count syllables based on a predefined dictionary"

markus_dressel
markus_dressel New Altair Community Member
edited November 5 in Community Q&A

Hi community,

I want to count the syllables within in document. Therefore, I have a predefined dictionary (in excel) which contains 85000 words and its corresponding syllable values. 

Now I want to tokenize the document and and count the sylables within the document. As result I would to retrieve the number of syllables in the text. What operators do I need ?

I hope you can help me with that topic.

 

Best regards and thanks in advance,

 

Markus 

Tagged:

Best Answer

  • Telcontar120
    Telcontar120 New Altair Community Member
    Answer ✓

    Sure, this is actually pretty straightforward.  First tokenize the document using "Process Document" and then output the wordlist using the "Wordlist to Data" operator, which will give you an exampleset of the wordlist with the counts.  Then you should be able to join your syllable count in using "Join" (you'll join on the words/tokens) and use "Generate Attributes" to compute the product of the word count and the syllable count per word, and then use "Aggregate" to get the sum of that product.  And you should have total syllables in the document!

     

     

Answers

  • Telcontar120
    Telcontar120 New Altair Community Member
    Answer ✓

    Sure, this is actually pretty straightforward.  First tokenize the document using "Process Document" and then output the wordlist using the "Wordlist to Data" operator, which will give you an exampleset of the wordlist with the counts.  Then you should be able to join your syllable count in using "Join" (you'll join on the words/tokens) and use "Generate Attributes" to compute the product of the word count and the syllable count per word, and then use "Aggregate" to get the sum of that product.  And you should have total syllables in the document!

     

     

  • sgenzer
    sgenzer
    Altair Employee

    Thanks, @Telcontar120, for that nice solution.  @markus_dressel - would you be willing to share your excel "syllable" sheet with the community?  It may be a resource that others find useful.

     

    Scott

  • markus_dressel
    markus_dressel New Altair Community Member

    That was exactly the solution I was looking for. It worked perfectly. Thank you so much @Telcontar120

     

    @sgenzer I use the Business dictionary provided by Lougrhan&McDonald. 

    The list and a comprehensive explanation can be found here:

     

    Thank you so much for the help,

     

    Best regards,

     

    Markus 

  • TFJ95
    TFJ95 New Altair Community Member
    Thanks for your help @Telcontar120
    I just started using RapidMiner yesterday, so I'm really new and a bit overwhelmed right now. Could you maybe explain to me with a screenshot what the solution looks like in the end?


Categories