"Truncated words when using WVTool (Text tool)"

drstevekramer

I have been using the default WVToolConfiguration (with no stemmer requested explicitly). Unfortunately, when I create a WVTWordList from a number of input documents using createWordList(WVTInputList input, WVTConfiguration config, java.util.List initialWords, boolean addWords), quite a few of the words end up being truncated when I iterate through the WVTWordList. For example, "time" -> "tim" and "country" -> "countr".

Is there a reason that this is happening with the sample, standard configuration? I did not explicitly set any stemmer or tokenizer options.

Thanks in advance for your help.

Cheers,
Steve