Aggregate Duplicates

iason

Can you suggest a method to remove duplicate examples and add a "count" attribute to the remaining unique items?
I would like to do that to reduce the size of the dataset and then use this counter attribute with a k-NN operator. Is that even possible in RM?

Andrew2

Hello

The aggregate operator is your friend - here's an example

<?xml version="1.0" encoding="UTF-8" standalone="no"?>
<process version="5.1.011">
  <context>
    <input/>
    <output/>
    <macros/>
  </context>
  <operator activated="true" class="process" compatibility="5.1.011" expanded="true" name="Process">
    <process expanded="true" height="206" width="413">
      <operator activated="true" class="generate_data" compatibility="5.1.011" expanded="true" height="60" name="Generate Data" width="90" x="112" y="75">
        <parameter key="target_function" value="three ring clusters"/>
        <parameter key="number_of_attributes" value="2"/>
      </operator>
      <operator activated="true" class="aggregate" compatibility="5.1.011" expanded="true" height="76" name="Aggregate" width="90" x="313" y="75">
        <list key="aggregation_attributes">
          <parameter key="label" value="count"/>
        </list>
        <parameter key="group_by_attributes" value="|label"/>
      </operator>
      <connect from_op="Generate Data" from_port="output" to_op="Aggregate" to_port="example set input"/>
      <connect from_op="Aggregate" from_port="example set output" to_port="result 1"/>
      <portSpacing port="source_input 1" spacing="0"/>
      <portSpacing port="sink_result 1" spacing="0"/>
      <portSpacing port="sink_result 2" spacing="0"/>
    </process>
  </operator>
</process>

regards

Andrew