Compare ROC Parameter
Hello,
I have some questions but I am not anglophone and may be it will difficult to you to understand my questions and difficult to me to understand your answer.
Please if I am not understandable say it to me.
I try to make a learning machine to a stage in computer science and I want compare some models to this.
I have very few examples (20) but I have to try anyway.
My troubles are to adjust the parameters of "compare ROC", I think understand what is a ROC curve and how interpreted but may be I make a mistake cause I don't understand why they are two parameters :
- number_of_folds.
- split_ratio.
According to me if i put 4 in number_of_folds for each models Rapidminer compares 4 training machine with 15 examples to learn and evaluate them with the last 5 and after create for each model the ROC Curves. And it match with that I want (compare some models to my 20 examples).
But split ratio can not be disable so it can play a role in the roc curve but i don't understand which. May be if I put 1 in this parameter it work as I think?
Thank for your help
Find more posts tagged with
Sort by:
1 - 2 of
21
Hi @lionelderkrikor and thank for your response
I also search in documentation before ask the question but I read and read again and no see that "if number of folds is in Range ]1, infinite[ [...] (and in this case, the defined split ratio is not taken into account)". I think your right cause it seems more logic than use the two options in the same time but I regret than it is not well-defined I the documentation.
I am reassure by your response but I hope we can have a second opinion.
(And may be somebody know how to ask modifications in the documentation? If two people have doubts may be we are not the only ones.)
thanks
Hi @PlatyQ,
Good question.
I asked myself the same question when I used this operator for the first time, but I never .......dared to ask the question.
Your post prompted me to look for an answer in the documentation :
According to the documentation, Compare ROC behaves like a Split Validation or a Cross Validation operator :
- if number of folds = -1, then Compare ROC behaves like a Split Validation with the defined split ratio.
- if number of folds is in Range ]1, infinite[, then Compare ROC behaves like a Cross Validation with the defined number of folds. (and in this case, the defined split ratio is not taken into account).
- if number of folds = 0 or 1 , then RapidMiner raise an error .... and it's logic.
NB : That's what I understand from the documentation, but thanks to the experts to confirm or..... invalidate this explanation of the behaviour of Compare ROC operator.
I hope it helps,
Regards,
Lionel