"[solved]K means clustering for text data to cluster similar data"
nachiket
New Altair Community Member
Hello I am new rapidminer
I have taken sample data as (for general study purpose)
Task
a
a
a
a
f
f
f
f
f
d
d
d
r
r
d
.
.
.
(repeating these characters i.e. "a", "r"," d"," f")
in excel file
and now I am applying kmeans clustering with 4 cluster (as I used 4 characters)
for smaller number of occurrences of "a" "r" "d" "f" when almost evenly distributed it works fine
but when a particular character dominates one cluster becomes empty and others are filled
what should be used in-order to avoid this situation ?
this is what I tried
<?xml version="1.0" encoding="UTF-8" standalone="no"?>
<process version="5.1.011">
<context>
<input/>
<output/>
<macros/>
</context>
<operator activated="true" class="process" compatibility="5.1.011" expanded="true" name="Process">
<process expanded="true" height="396" width="751">
<operator activated="true" class="read_excel" compatibility="5.1.011" expanded="true" height="60" name="Read Excel" width="90" x="45" y="75">
<parameter key="excel_file" value="C:\Users\hp\Desktop\tempclusterdel.xls"/>
<parameter key="imported_cell_range" value="A1:A54"/>
<parameter key="first_row_as_names" value="false"/>
<list key="annotations">
<parameter key="0" value="Name"/>
</list>
<list key="data_set_meta_data_information">
<parameter key="0" value="Task.true.text.attribute"/>
</list>
</operator>
<operator activated="true" class="generate_id" compatibility="5.1.011" expanded="true" height="76" name="Generate ID" width="90" x="71" y="142"/>
<operator activated="true" class="multiply" compatibility="5.1.011" expanded="true" height="94" name="Multiply" width="90" x="112" y="210"/>
<operator activated="true" class="nominal_to_numerical" compatibility="5.1.011" expanded="true" height="94" name="Nominal to Numerical" width="90" x="179" y="30">
<list key="comparison_groups"/>
</operator>
<operator activated="true" class="multiply" compatibility="5.1.011" expanded="true" height="94" name="Multiply (2)" width="90" x="312" y="94"/>
<operator activated="true" class="k_means" compatibility="5.1.011" expanded="true" height="76" name="Clustering" width="90" x="447" y="75">
<parameter key="k" value="4"/>
</operator>
<operator activated="true" class="join" compatibility="5.1.011" expanded="true" height="76" name="Join" width="90" x="514" y="210">
<list key="key_attributes"/>
</operator>
<connect from_op="Read Excel" from_port="output" to_op="Generate ID" to_port="example set input"/>
<connect from_op="Generate ID" from_port="example set output" to_op="Multiply" to_port="input"/>
<connect from_op="Multiply" from_port="output 1" to_op="Nominal to Numerical" to_port="example set input"/>
<connect from_op="Multiply" from_port="output 2" to_op="Join" to_port="left"/>
<connect from_op="Nominal to Numerical" from_port="example set output" to_op="Multiply (2)" to_port="input"/>
<connect from_op="Multiply (2)" from_port="output 1" to_op="Clustering" to_port="example set"/>
<connect from_op="Multiply (2)" from_port="output 2" to_port="result 3"/>
<connect from_op="Clustering" from_port="cluster model" to_port="result 2"/>
<connect from_op="Clustering" from_port="clustered set" to_op="Join" to_port="right"/>
<connect from_op="Join" from_port="join" to_port="result 1"/>
<portSpacing port="source_input 1" spacing="0"/>
<portSpacing port="sink_result 1" spacing="0"/>
<portSpacing port="sink_result 2" spacing="0"/>
<portSpacing port="sink_result 3" spacing="0"/>
<portSpacing port="sink_result 4" spacing="0"/>
</process>
</operator>
</process>
I have taken sample data as (for general study purpose)
Task
a
a
a
a
f
f
f
f
f
d
d
d
r
r
d
.
.
.
(repeating these characters i.e. "a", "r"," d"," f")
in excel file
and now I am applying kmeans clustering with 4 cluster (as I used 4 characters)
for smaller number of occurrences of "a" "r" "d" "f" when almost evenly distributed it works fine
but when a particular character dominates one cluster becomes empty and others are filled
what should be used in-order to avoid this situation ?
this is what I tried
<?xml version="1.0" encoding="UTF-8" standalone="no"?>
<process version="5.1.011">
<context>
<input/>
<output/>
<macros/>
</context>
<operator activated="true" class="process" compatibility="5.1.011" expanded="true" name="Process">
<process expanded="true" height="396" width="751">
<operator activated="true" class="read_excel" compatibility="5.1.011" expanded="true" height="60" name="Read Excel" width="90" x="45" y="75">
<parameter key="excel_file" value="C:\Users\hp\Desktop\tempclusterdel.xls"/>
<parameter key="imported_cell_range" value="A1:A54"/>
<parameter key="first_row_as_names" value="false"/>
<list key="annotations">
<parameter key="0" value="Name"/>
</list>
<list key="data_set_meta_data_information">
<parameter key="0" value="Task.true.text.attribute"/>
</list>
</operator>
<operator activated="true" class="generate_id" compatibility="5.1.011" expanded="true" height="76" name="Generate ID" width="90" x="71" y="142"/>
<operator activated="true" class="multiply" compatibility="5.1.011" expanded="true" height="94" name="Multiply" width="90" x="112" y="210"/>
<operator activated="true" class="nominal_to_numerical" compatibility="5.1.011" expanded="true" height="94" name="Nominal to Numerical" width="90" x="179" y="30">
<list key="comparison_groups"/>
</operator>
<operator activated="true" class="multiply" compatibility="5.1.011" expanded="true" height="94" name="Multiply (2)" width="90" x="312" y="94"/>
<operator activated="true" class="k_means" compatibility="5.1.011" expanded="true" height="76" name="Clustering" width="90" x="447" y="75">
<parameter key="k" value="4"/>
</operator>
<operator activated="true" class="join" compatibility="5.1.011" expanded="true" height="76" name="Join" width="90" x="514" y="210">
<list key="key_attributes"/>
</operator>
<connect from_op="Read Excel" from_port="output" to_op="Generate ID" to_port="example set input"/>
<connect from_op="Generate ID" from_port="example set output" to_op="Multiply" to_port="input"/>
<connect from_op="Multiply" from_port="output 1" to_op="Nominal to Numerical" to_port="example set input"/>
<connect from_op="Multiply" from_port="output 2" to_op="Join" to_port="left"/>
<connect from_op="Nominal to Numerical" from_port="example set output" to_op="Multiply (2)" to_port="input"/>
<connect from_op="Multiply (2)" from_port="output 1" to_op="Clustering" to_port="example set"/>
<connect from_op="Multiply (2)" from_port="output 2" to_port="result 3"/>
<connect from_op="Clustering" from_port="cluster model" to_port="result 2"/>
<connect from_op="Clustering" from_port="clustered set" to_op="Join" to_port="right"/>
<connect from_op="Join" from_port="join" to_port="result 1"/>
<portSpacing port="source_input 1" spacing="0"/>
<portSpacing port="sink_result 1" spacing="0"/>
<portSpacing port="sink_result 2" spacing="0"/>
<portSpacing port="sink_result 3" spacing="0"/>
<portSpacing port="sink_result 4" spacing="0"/>
</process>
</operator>
</process>
Tagged:
0