[SOLVED] Example Set Transformation - Performance Problem

South2wood

Hi,

I have a large data set (10 millionn lines)  where each line/example of the table is identified by two id-attributes, for example like this:

domain (id), week (id), value (attribute)
facebook.com; 1; 0,5
facebook.com; 2; 0,6
google.com; 1; 0,9
google.com; 2; 0,4
...

Now I want to transform the table into a time series, like this:
domain (id), value_week1 (attribute), value_week2 (attribute)
facebook.com; 0,5; 0,6
google.com; 0,9; 0,4

I tried to solve it by importing my data into a mysql database and do the transformation process with a php script, but it lasts for hours and didn't terminate yet. Is there a transformation operator in rapidminer which can do the job?

Another question. If I define two attributes as ID, does rapidminer understand that each line is defined by two IDs? I tried some set operations (Set Minus, Join) and it doesn't seem to work?

Happy New Year by the way ;-)

Best regards,
Matthias

earmijo

For your first question: Use the operator Pivot (using domain as the group attribute and week as the index attribute).

Regards,

E