Altair RISE
A program to recognize and reward our most engaged community members
Nominate Yourself Now!
Home
Discussions
Community Q&A
Performance evalution in the recommender extension
bernardo_pagnon
Hello,
I am using the recommender extension and there is one thing I don't get it. In the item recommendation, the operator gives a ranking prediction of unseen movies for a user (unlike the rating prediction). In this case, how can one evaluate the quality of the estimator? Does it make sense to use the split operator for instance? There is no label, no benchmark!
Best,
Bernardo
Find more posts tagged with
AI Studio
Extensions
Performance
Accepted answers
Telcontar120
Right, this is inherent to unsupervised machine learning outcomes, there is no performance metric that is pre-defined. You could try to do some diagnostics like look at the variety of different top rated movies or something similar, but that is not really a performance metric strictly speaking. Another thing you could do is wait for users to watch and rate the movies recommended, and then compare those rankings with the original rankings generated. But this is going to take some time to collect the relevant data.
All comments
Telcontar120
Right, this is inherent to unsupervised machine learning outcomes, there is no performance metric that is pre-defined. You could try to do some diagnostics like look at the variety of different top rated movies or something similar, but that is not really a performance metric strictly speaking. Another thing you could do is wait for users to watch and rate the movies recommended, and then compare those rankings with the original rankings generated. But this is going to take some time to collect the relevant data.
bernardo_pagnon
Hello Brian,
thank you for your reply. I agree with everything that you said, validating an unsupervised ML method is usually tricky. I am doing exactly as you said, looking at the different top-rated movies
and trying to gain some insight.
Best,
Bernardo
Quick Links
All Categories
Recent Discussions
Activity
Unanswered
日本語 (Japanese)
한국어(Korean)
Groups