Is there a way to optimize the Cost / Benefit Matrix in Auto Model

Tyche
Tyche New Altair Community Member
edited November 5 in Community Q&A

Hello everyone, I am relatively new to the field and currently working on a university project.

I tried to experiment a bit with the Cost / Benefit Matrix of Auto Model and was able to get more precise models.
I am now at the point where I ask myself if there is any way of optimizing the matrix by for example calculating the distribution?

The labels are distributed with 49% in the first group, 18,5% in the second and third group and 14 % in the fourth group.

Thanks for help in advance


Tagged:

Best Answer

  • Telcontar120
    Telcontar120 New Altair Community Member
    Answer ✓
    @Tyche I don't think so, because there can be large differences in cost that are totally independent of the distribution of classes.  Think about cases like false positives vs false negatives for certain medical tests, or the cost of undetected fraud in financial institutions, etc.  So I would say you should probably consult a domain expert if you want to use the cost matrix as the main model performance metric.  Otherwise you can use something simpler like accuracy or ROC which does not require you to make different assumptions about the costs of different types of misclassification errors.


Answers

  • Telcontar120
    Telcontar120 New Altair Community Member
    I am not at all clear what you mean by "optimize the cost matrix"---since the cost matrix is itself a measure of model performance.  Can you explain further what you have in mind?  Typically the cost matrix is something you bring to your problem from your domain knowledge, not something you are trying to solve for using the model.  
  • Tyche
    Tyche New Altair Community Member
    edited May 2020
    Typically the cost matrix is something you bring to your problem from your domain knowledge, not something you are trying to solve for using the model. 

    @Telcontar120 That's what I expected, since the domain not really has a cost assigned to the problem I was curious if there is an optimal way to configure the cost matrix based on the distribution of the attributes.



  • Telcontar120
    Telcontar120 New Altair Community Member
    Answer ✓
    @Tyche I don't think so, because there can be large differences in cost that are totally independent of the distribution of classes.  Think about cases like false positives vs false negatives for certain medical tests, or the cost of undetected fraud in financial institutions, etc.  So I would say you should probably consult a domain expert if you want to use the cost matrix as the main model performance metric.  Otherwise you can use something simpler like accuracy or ROC which does not require you to make different assumptions about the costs of different types of misclassification errors.


Categories