how to tokenize documents? - Altair Community

kayman · 2019-08-10T10:17:13+00:00

I'm wondering if it's feasible to tokenize documents, as the tokenize operator itself only offers the option to tokenize on expressions and the likes. As an example consider the following scenario : a collection of similar documents in a folder is loaded and combined in a single document using the combine document…