Home
Discussions
Community Q&A
"Compare model performance graphs"
dupolas
Hi there,
I was wondering if it is possible to run multiple classification models on the same dataset and compare the different performance curves.
For example I run a decision tree, neural net and logistic regression on a training and test set and would like to compare training and test performance.(eg. sq. avg error)
is this possible with Rapidminer? I know it is with SAS and it is pretty usefull for me
Thanks a lot...
I googled for this and could not see any tutorials/posts on it..
Regards,
Geoffrey
Find more posts tagged with
AI Studio
Performance
Accepted answers
All comments
MariusHelf
Hi,
I suppose you already know how to evaluate a single model in RapidMiner, e.g. with the X-Validation. The X-Validation puts out a performance vector, which should contain the root mean squared error. This value is only a scalar value, so I don't know from which data you want to create curves. Of course you can compare the values from different models, and if you extract the performance values, you also can write them into a single example set for easier comparison.
For classification problems we can create ROC comparison charts.
For the performance curves, please give us a more detailed description of what you want to do. Please read the post linked in my signature on how to write good questions.
Happy Mining!
~Marius
dupolas
Marius
Thanks for the hints.... Still struggling with some of the concepts being new to data mining .
Yes I was looking for ROC curves for classification problems.
However conceptually trying this on the iris data set didn't work out because the label needs to be binomial instead of multiclass. So will have to manipulate my data set a bit to get the 'one vs the rest' labels. Some practical learning to be done there
Regards
Geoffrey
MariusHelf
Fine. if you get stuck anywhere just post a message here!
Aprile
Hi, I have the same question too and I've tried this process. But I'm not sure if it right in theory but it actually present the Comparing ROC for my 3 models. Would you mind helping me discuss about the process, if I did anything wrong.
Aprile
https://community.rapidminer.com/discussion/58946/how-to-compare-classification-models-in-the-same-time-with-compare-their-rocs-this-is-my-process#latest
Quick Links
All Categories
Recent Discussions
Activity
Unanswered
日本語 (Japanese)
한국어(Korean)