"Text Mining Missing Values"

wessel
wessel New Altair Community Member
edited November 5 in Community Q&A
Dear All,

Using
"Data to Documents"
and
"Process Documents"

I get a nice document array.
Where rows are documents and columns are word frequencies.
How to set word frequencies to missing, when text length (e.g. document length) is smaller than some value?

Like
Document 1 has no occurrences for any word, because text is missing (or has some very small length).
Document 2 has a count of 10 on word1, and a count of 0 on word2.

Right now document 1 has a count of 0 for all words. This is true, but I'd like to be able to set a special case when when text is missing or of small length.

Best regards,

Wessel

Answers

  • Andrew2
    Andrew2 New Altair Community Member
    Hello Wessel

    You could count the number of tokens using Extract Information (I think). This gets added as a special attribute. Then a bit of example filtering or attribute generation would be needed.

    Regards

    Andrew