"fil" and "out" are not Operator objects, they are the connectors on the left and right border of the window that look like knobs.

Click on Select Attributes and select the parameter attribute filter type to either subset or regular_expression.

For subset, click on the Select Attributes... button, and add the attributes (columns) of your CSVs that you want to have in your merged output. Add them in the right list of the window by typing the name and clicking the plus icon.

For regular_expression you can define a list of attriutes (columns) like this: .*attribute1.*|.*attribute2.*|.*attribute3.*

example:

 

.*mail.*|.*date_submitted.*|.*page_url.*

Then you are done.

 

 

Merge columns with different names into the same column (attribute):

In case you have columns in your CSV with different naming, like: e-mail, eMail, e_mail you can do the following:

In your existing Select Attributes Object choose regular_expressions. Define a regular_expression that contains all columns you want, and also the variants. If I have the following columns:

I would create the following regular_expression:

 

 

.*mail.*|.*date_submitted.*|.*name.*

This will still create an output with different columns (attribute). To merge the 3 Email columns into one, you have to rename them to be identical. Add a Rename by Replacing Object after the already existing Select Attributes.

 

On the Rename by Replacing Object select regular_expression as the attribute filter type. Then fill out the fields below like:

regular expression: .*mail.*

replace what: .*

replace by: mail

 

I attached a gif of my process, to clarify.

Photo of sgenzer sgenzer

very nice explanation, @underlines !

 

Scott

 

 

Photo of penmen penmen
Great answer @underlines!

Could someone please suggest what to do:
if the columns being imported don't have column headers? or even,
What should be provided in the sub-process while setting the filter for Select Attribute, in the line above?
Photo of RolandJones RolandJones
Hi @penmen,

When you import a file without headers, it will assign default names of att1, att2, att3 and so on. If the order of attributes is always the same , then it doesn't cause a problem, you can just select the attributes you need (e.g. att1 and att3), and then rename the attributes after the append described above. If there's isn't a consistent order, then you will probably have to implement some more complicated logic to assign the correct attribute names prior to the Select Attributes step.

Hope this helps,

Best,

Roland
Photo of penmen penmen
Thanks @rjones13 !
I was able to resolve my issue above by setting 'all attributes' in the Select Attribute first.
Then, after Renaming by Generic names, I was able to any required column with 'Select Attributes' and by providing the concatenated string value of my generic name + column number.
Photo of MartinLiebig MartinLiebig
Hi,
another option is to use the "data set meta data configuration" parameter in the operator. This gives you the option the give a given index a name.
If you run the wizard its automatically configured. Its a good practice to first run the wizard on one file and then c/p the configured opertor into your loop.
BR,
Martin
Photo of nataliarelish nataliarelish
Hello there
In my experience first organize all the CSV file in a single folder then open RapidMiner and explore the Repository choose Import Data from the toolbar then select Import Data from Files organize the schema make sure the option is selected then confirm setting