"LearningCurve First_fraction Bug"
wessel
New Altair Community Member
Okay, the bug goes like this:
When I open Repidminer, load the xml, and press play, there is no result at the first fraction.
When I play the "Play" button again, there is a result for the first fraction, but it seems to be the result for the last fraction!!!
First open RM:
Performance Fraction
NaN 0.05
0.4 0.1
0.85 0.15000000000000002
0.85 0.2
0.8833333333333333 0.25
0.9166666666666666 0.3
0.9166666666666666 0.35
2nd, 3rd, 4th, ... time press play
0.9166666666666666 0.05 <== THIS SEEMS BUGGED
0.4 0.1
0.85 0.15000000000000002
0.85 0.2
0.8833333333333333 0.25
0.9166666666666666 0.3
0.9166666666666666 0.35
xml
[tt]<operator name="Root" class="Process" expanded="yes">
<description text="This process plots the learning curve, i.e. the performance with respect to the number of examples which is used for learning."/>
<parameter key="logverbosity" value="warning"/>
<parameter key="random_seed" value="2004"/>
<operator name="Iris" class="ArffExampleSource">
<parameter key="data_file" value="C:\rm_workspace\sample\data\iris.arff"/>
<parameter key="label_attribute" value="class"/>
</operator>
<operator name="split at 90%, step 5%" class="LearningCurve" expanded="yes">
<parameter key="training_ratio" value="0.6"/>
<parameter key="start_fraction" value="0.0"/>
<operator name="OperatorChain" class="OperatorChain" expanded="yes">
<operator name="W-J48" class="W-J48">
<parameter key="keep_example_set" value="true"/>
</operator>
</operator>
<operator name="ApplierChain" class="OperatorChain" expanded="yes">
<operator name="ModelApplier" class="ModelApplier">
<list key="application_parameters">
</list>
</operator>
<operator name="ClassificationPerformance" class="ClassificationPerformance">
<parameter key="accuracy" value="true"/>
<list key="class_weights">
</list>
</operator>
</operator>
<operator name="ProcessLog" class="ProcessLog">
<list key="log">
<parameter key="performance" value="operator.split at 90%, step 5%.value.performance"/>
<parameter key="fraction" value="operator.split at 90%, step 5%.value.fraction"/>
</list>
</operator>
</operator>
</operator>
[/tt]
When I open Repidminer, load the xml, and press play, there is no result at the first fraction.
When I play the "Play" button again, there is a result for the first fraction, but it seems to be the result for the last fraction!!!
First open RM:
Performance Fraction
NaN 0.05
0.4 0.1
0.85 0.15000000000000002
0.85 0.2
0.8833333333333333 0.25
0.9166666666666666 0.3
0.9166666666666666 0.35
2nd, 3rd, 4th, ... time press play
0.9166666666666666 0.05 <== THIS SEEMS BUGGED
0.4 0.1
0.85 0.15000000000000002
0.85 0.2
0.8833333333333333 0.25
0.9166666666666666 0.3
0.9166666666666666 0.35
xml
[tt]<operator name="Root" class="Process" expanded="yes">
<description text="This process plots the learning curve, i.e. the performance with respect to the number of examples which is used for learning."/>
<parameter key="logverbosity" value="warning"/>
<parameter key="random_seed" value="2004"/>
<operator name="Iris" class="ArffExampleSource">
<parameter key="data_file" value="C:\rm_workspace\sample\data\iris.arff"/>
<parameter key="label_attribute" value="class"/>
</operator>
<operator name="split at 90%, step 5%" class="LearningCurve" expanded="yes">
<parameter key="training_ratio" value="0.6"/>
<parameter key="start_fraction" value="0.0"/>
<operator name="OperatorChain" class="OperatorChain" expanded="yes">
<operator name="W-J48" class="W-J48">
<parameter key="keep_example_set" value="true"/>
</operator>
</operator>
<operator name="ApplierChain" class="OperatorChain" expanded="yes">
<operator name="ModelApplier" class="ModelApplier">
<list key="application_parameters">
</list>
</operator>
<operator name="ClassificationPerformance" class="ClassificationPerformance">
<parameter key="accuracy" value="true"/>
<list key="class_weights">
</list>
</operator>
</operator>
<operator name="ProcessLog" class="ProcessLog">
<list key="log">
<parameter key="performance" value="operator.split at 90%, step 5%.value.performance"/>
<parameter key="fraction" value="operator.split at 90%, step 5%.value.fraction"/>
</list>
</operator>
</operator>
</operator>
[/tt]
Tagged:
0
Answers
-
Hi,
this is not a bug, but instead the only possible behavior. You are logging the values from the parent operator. This operator will update it's values as soon as all inner operators have been executed. Since the ProcessLog is an inner operator, it tries to get this value before it's written the first time.
You simply have to log the performance value of the Performance Operator itself. It should work with this process:<operator name="Root" class="Process" expanded="yes">
Greetings,
<description text="This process plots the learning curve, i.e. the performance with respect to the number of examples which is used for learning."/>
<parameter key="logverbosity" value="warning"/>
<parameter key="random_seed" value="2004"/>
<operator name="Iris" class="ArffExampleSource">
<parameter key="data_file" value="C:\rm_workspace\sample\data\iris.arff"/>
<parameter key="label_attribute" value="class"/>
</operator>
<operator name="split at 90%, step 5%" class="LearningCurve" expanded="yes">
<parameter key="training_ratio" value="0.6"/>
<parameter key="start_fraction" value="0.0"/>
<operator name="OperatorChain" class="OperatorChain" expanded="yes">
<operator name="W-J48" class="W-J48">
<parameter key="keep_example_set" value="true"/>
</operator>
</operator>
<operator name="ApplierChain" class="OperatorChain" expanded="yes">
<operator name="ModelApplier" class="ModelApplier">
<list key="application_parameters">
</list>
</operator>
<operator name="ClassificationPerformance" class="ClassificationPerformance">
<parameter key="accuracy" value="true"/>
<list key="class_weights">
</list>
</operator>
</operator>
<operator name="ProcessLog" class="ProcessLog">
<list key="log">
<parameter key="performance" value="operator.ClassificationPerformance.value.accuracy"/>
<parameter key="fraction" value="operator.split at 90%, step 5%.value.fraction"/>
</list>
</operator>
</operator>
</operator>
Sebastian0