Process Documents operator ends up with more documents than example set

Question

I have an example set with 733 examples and with text as an attribute, I process it into TF-IDF using the data to documents and the process documents operators, inside the process documents I clean up and tokenize the text then I use the Write Document and I end up with 1466 files

Why do I end up with more twice as many files as examples?
How do I ensure that 1 document in = 1 document out ?

I have extract content set to negate every tag possible but I end up with 2 outputs for every 1 input. From a brief look it seems file 734 is simiilar to file 1 so its like the whole thing loops twice for some reason

mob · Answer

Can't share the data but here is the process to take a text field from an example set, pre-process it, write the documents to disk before outputting a td-idf example set Need to select the attributes in the pro settings of the Data to Document Process the entire dataset as if it were DIT data and use the merged stop phrase list to remove the boiler plate Uses the regex from rapidminer forum to split where capitialised letters are in the middle of words because punctuation is missing from the original text. It finds captialised words and replaces them with a space and the captured text Remove �� from the start of some words and sometime �� and replace it with just the word found after it Regular Expression Replacement ��{1,2}() $1

JEdward · Answer

Can you post up the XML of your process (or a simplified version) showing it?

That way I can properly see what's happening.

Thanks