operating generate N-Grams

MarcosRL
MarcosRL New Altair Community Member
edited November 5 in Community Q&A
Hello. friends of comunity .One question.
What heuristic applies  operator "generate N-Grams" ?
The question I have is,
if my text delete the stop words and generate N-Grams applied with N = 4.
How to recognize the terms of 4 or 3 or 2 words?
regards
Tagged:

Answers

  • MariusHelf
    MariusHelf New Altair Community Member
    Hi,

    to be honest, I don't understand the question. N-Grams does not apply any heuristic, it simply combines N, in your case 4, tokens to one new token. If you removed the stopwords before, no stopwords will be used for the N-Grams.


    Best regards,
    Marius

Categories