What's the meaning of PARAMETER_ON_THE_FLY_PRUNING?

amy

Hi there,
Here is a parameter named on_the_fly_pruning like
<parameter key="on_the_fly_pruning" value="-1"/>
in the TextInput operator. What does it mean?
Thanks a lot.

IngoRM

Hi,

if this parameter is set to any value greater than 0 the following happens: each time the specified number of documents were read, all terms which occured only once are removed. If, for example, you set this parameter to 3, after 3 documents all terms with a frequency of 1 are removed, then after the next three documents this is checked again and so on.

Cheers,
Ingo