Data preparation

LindsayKelevra

hi everybody! I made the data preparation shown in the picutre, but watching the tf-idf's weighting schemas i notice that there are some strange charaters (for example â€œoptionâ€), how can eliminate them?  thank u

Telcontar120

This is likely the artifact of characters that can't be encoded normally (like emojis or other similar graphical items) so you can simply filter them out using the "Filter Tokens" after you transform, or you can deal with them before you even process your document by using the Replace operator.

Telcontar120

This is likely the artifact of characters that can't be encoded normally (like emojis or other similar graphical items) so you can simply filter them out using the "Filter Tokens" after you transform, or you can deal with them before you even process your document by using the Replace operator.