two problems: create Threshold and T-test results
inceptorfull
New Altair Community Member
I am creating two models and compare there accuracy: first the Logistic and second Neural network for binary classification
I didnot configure threshold , so is this should make the results wrong?
I tried to create threshold using create and apply threshold after applying the model and before perfomance to be 0.5 but the results is the same, although I look to NN threshould in the results it say threshold is 3.24 so why it is not 0.5??
Second Question I made T-test for the accuracy of both Models with the same data inputs it give me the following results
Probabilities for random values with the same result:
----- 0.892
----- -----
Values smaller than alpha=0.050 indicate a probably significant difference between the mean values!
List of performance values:
0: 0.822 +/- 0.049
1: 0.825 +/- 0.074
so what :0 mean and 1 mean?? as the table in the form of
A B C
I didnot configure threshold , so is this should make the results wrong?
I tried to create threshold using create and apply threshold after applying the model and before perfomance to be 0.5 but the results is the same, although I look to NN threshould in the results it say threshold is 3.24 so why it is not 0.5??
Second Question I made T-test for the accuracy of both Models with the same data inputs it give me the following results
Probabilities for random values with the same result:
----- 0.892
----- -----
Values smaller than alpha=0.050 indicate a probably significant difference between the mean values!
List of performance values:
0: 0.822 +/- 0.049
1: 0.825 +/- 0.074
so what :0 mean and 1 mean?? as the table in the form of
A B C
Tagged:
0
Answers
-
Hi,
dont get confused by thresholds. In the NN the threshold is something internal (i think something like a constant/bias). The create and apply threshold operators change the decision boundery for confidences. If you set it to 0.7 i needs to have at least a confidence of 0.7 to be classified into a specific class.
~Martin0 -
thanks a lot, so when I apply the threshold to 0.5 or I didnit apply it it gives me the same results, is that means threshold is already 0.5? do I need to make threshold for Logisitic?
also can I export rapidminer results in a form to be input for other programs like spss? to make wilcoxon test?
0 -
Hi,
the default threshold to define whether it is a or b is 0.5, yes.
You can of course export tables to any other formats (e.g. Write Excel). You can not export models. By the way wilcoxon test is included in the stats extension. See: https://oldworldcomputing.com/products/statistics-extension-for-rapidminer . The extension is commercial but there is a free version with limited number of rows.
~Martin0 -
thanks a lot for reply, it works fine but
is there a way to export the descriptive statics of my data ??
also I want to know how to export the parameters of Neural network for example in a file ? to put in my research ?
I found write performance and parametrs but the file extenstion is per? so what to do ?0 -
Stats: Only via Aggregate operator
Parameters for NN: you can use a log and then write the log, for performance: Try performance to data first. this gives you an example set.
~Martin0 -
thanks a lot for reply , I am really gratefull for you
I have a problem I tried to make t-test for the models accuracy and ANOVA test, it work fine using X-validation , but when using Split validation, it gives me error " degree freedoom -2", donot know why that? would you help me in that?
although I want to know , when using the t-test for accuracy, what data to export to input it in SPPSS? is the accuracy rate? so I write it to Excel then put to in SPPSS? or what? I am confused so I can do more statistical tests in my output?0 -
Regarding your error message:
Could you provide me a copy of your process?
The error message indicates, that your test set is either to small or to large.0
Altair Employee