How to get index of a word in the list and can we extract the value based on index?
Anusha
New Altair Community Member
Hi All!
My dataset has six columns. All columns are in the list format.
first column(list) has common words which are present in 2nd(ch_1) and 3rd(ch_2) columns. The respective values are present in 4th(va_1) and 5th(va_2) columns. So, I need to loop over the each word present in "list" column and extract the index of that particular in ch_1& ch_2 and get the respective values in va_1&va_2. After getting va_1 and va_2, I have to compare if va_1 not equal to va_2 flag "mis" value becomes "1" else "0".
Source:
list ch_1 ch_2 va_1 va_2
typ,shap typ,shap,fgh typ,fgi,shap 8,"ghk",90 8,97, "oki"
----here typ,shap are common words in ch_1&ch_2 and their respective indexes are (0,1)in ch_1 and (0,2) in ch_2. so, I need values from va_1&va_2 based on indexes. As (0,1) in ch_1, need to extract (0,1) values from va_1 and in the same way (0,2) are indexes of ch_2, need to get values of (0,2) from va_2.
Required output:
list ch_1 ch_2 va_1 va_2 mis
typ,shap typ,shap,fgh typ,fgi,shap 8,"ghk",90 8,97, "oki" 1
----"mis" value is "1" because for the "typ"----va_1=8 and va_2=8 matched in the case.
But for the "shap"----va_1="ghk" and va_2="oki", here the values are mismatched. so, If one value is mismatched, then the flag should be "1".
Can anyone help me?
Thanks in Advance!
My dataset has six columns. All columns are in the list format.
first column(list) has common words which are present in 2nd(ch_1) and 3rd(ch_2) columns. The respective values are present in 4th(va_1) and 5th(va_2) columns. So, I need to loop over the each word present in "list" column and extract the index of that particular in ch_1& ch_2 and get the respective values in va_1&va_2. After getting va_1 and va_2, I have to compare if va_1 not equal to va_2 flag "mis" value becomes "1" else "0".
Source:
list ch_1 ch_2 va_1 va_2
typ,shap typ,shap,fgh typ,fgi,shap 8,"ghk",90 8,97, "oki"
----here typ,shap are common words in ch_1&ch_2 and their respective indexes are (0,1)in ch_1 and (0,2) in ch_2. so, I need values from va_1&va_2 based on indexes. As (0,1) in ch_1, need to extract (0,1) values from va_1 and in the same way (0,2) are indexes of ch_2, need to get values of (0,2) from va_2.
Required output:
list ch_1 ch_2 va_1 va_2 mis
typ,shap typ,shap,fgh typ,fgi,shap 8,"ghk",90 8,97, "oki" 1
----"mis" value is "1" because for the "typ"----va_1=8 and va_2=8 matched in the case.
But for the "shap"----va_1="ghk" and va_2="oki", here the values are mismatched. so, If one value is mismatched, then the flag should be "1".
Can anyone help me?
Thanks in Advance!
Tagged:
0
Answers
-
Hi there,the first step is to convert your data set into a proper data frame. I recommend to do this in the ETL process which pushes the data into your data lake. This is 100% standard thing to do.
Attached is a process to convert it into a format we may use more easily. I think afterwards its just another generate Attributes + maybe pivot it back.Best,Martin<?xml version="1.0" encoding="UTF-8"?><process version="9.9.000">
<context>
<input/>
<output/>
<macros/>
</context>
<operator activated="true" class="process" compatibility="9.9.000" expanded="true" name="Process">
<parameter key="logverbosity" value="init"/>
<parameter key="random_seed" value="2001"/>
<parameter key="send_mail" value="never"/>
<parameter key="notification_email" value=""/>
<parameter key="process_duration_for_mail" value="30"/>
<parameter key="encoding" value="SYSTEM"/>
<process expanded="true">
<operator activated="true" class="utility:create_exampleset" compatibility="9.9.000" expanded="true" height="68" name="Create ExampleSet" width="90" x="45" y="34">
<parameter key="generator_type" value="comma separated text"/>
<parameter key="number_of_examples" value="100"/>
<parameter key="use_stepsize" value="false"/>
<list key="function_descriptions"/>
<parameter key="add_id_attribute" value="false"/>
<list key="numeric_series_configuration"/>
<list key="date_series_configuration"/>
<list key="date_series_configuration (interval)"/>
<parameter key="date_format" value="yyyy-MM-dd HH:mm:ss"/>
<parameter key="time_zone" value="SYSTEM"/>
<parameter key="input_csv_text" value="list;ch_1;ch_2;va_1;va_2 typ,shap;typ,shap,fgh;typ,fgi,shap; 8,"ghk",90;8,97, "oki" "/>
<parameter key="column_separator" value=";"/>
<parameter key="parse_all_as_nominal" value="true"/>
<parameter key="decimal_point_character" value="."/>
<parameter key="trim_attribute_names" value="true"/>
</operator>
<operator activated="true" class="generate_id" compatibility="9.9.000" expanded="true" height="82" name="Generate ID" width="90" x="380" y="34">
<parameter key="create_nominal_ids" value="false"/>
<parameter key="offset" value="0"/>
</operator>
<operator activated="true" class="split" compatibility="9.9.000" expanded="true" height="82" name="Split" width="90" x="514" y="34">
<parameter key="attribute_filter_type" value="subset"/>
<parameter key="attribute" value=""/>
<parameter key="attributes" value="va_1|va_2"/>
<parameter key="use_except_expression" value="false"/>
<parameter key="value_type" value="nominal"/>
<parameter key="use_value_type_exception" value="false"/>
<parameter key="except_value_type" value="file_path"/>
<parameter key="block_type" value="single_value"/>
<parameter key="use_block_type_exception" value="false"/>
<parameter key="except_block_type" value="single_value"/>
<parameter key="invert_selection" value="false"/>
<parameter key="include_special_attributes" value="false"/>
<parameter key="split_pattern" value=","/>
<parameter key="split_mode" value="ordered_split"/>
</operator>
<operator activated="true" class="de_pivot" compatibility="9.9.000" expanded="true" height="82" name="De-Pivot" width="90" x="648" y="34">
<list key="attribute_name">
<parameter key="first_list" value="va_1_(\d+)"/>
<parameter key="secondlist" value="va_2_(\d+)"/>
</list>
<parameter key="index_attribute" value="rowId"/>
<parameter key="create_nominal_index" value="false"/>
<parameter key="keep_missings" value="false"/>
</operator>
<connect from_op="Create ExampleSet" from_port="output" to_op="Generate ID" to_port="example set input"/>
<connect from_op="Generate ID" from_port="example set output" to_op="Split" to_port="example set input"/>
<connect from_op="Split" from_port="example set output" to_op="De-Pivot" to_port="example set input"/>
<connect from_op="De-Pivot" from_port="example set output" to_port="result 1"/>
<portSpacing port="source_input 1" spacing="0"/>
<portSpacing port="sink_result 1" spacing="0"/>
<portSpacing port="sink_result 2" spacing="0"/>
</process>
</operator>
</process>
0 -
Hi @mschmitz,
Sorry. Maybe you didn't get my question right. For your understanding, I'm twisting my data set as key-value pairs.
Source:("list" attribute has words that are common in other two attributes ch_val_1 & ch_val_2)
list ch_val_1 ch_val_2
typ,shap shap: "cir",coun:1,typ:23 jkl:90,shap:"cir",typ:32
fit coun:4,fit:23 fit:23,jkl:09
Output required:
list ch_val_1 ch_val_2 mis
typ,shap shap: "cir",coun:1,typ:23 jkl:90,shap:"cir",typ:32 1
fit coun:4,fit:23 fit:23,jkl:09 0
Explanation:
In first record, "typ" value in ch_val_1 attribute is 23 and in ch_val_2 attribute, it is 32. so, there is a mismatch of values. Hence mis flag should be "1". Even though second-word "shap" value is matched between ch_val_1 & ch_val_2. if one value is mismatched in that row flag should be highlighted.
In the second record,"fit" word value in ch_val_1 is 23 and ch_val_2 also it's 23. so as there is no mismatch, flag should be "0".
Could you please help me?
0 -
Hi @Anusha,this looks quite similar to your problem in this post: https://community.rapidminer.com/discussion/comment/69757If you use the "Split" Operator with the separator "," you should receive an ExampleSet which looks like the one mentioned in above post.Then you could use the solution created there.Happy Mining and Stay safe,Edin
0
Altair Employee