Robustness metrics

teio
teio New Altair Community Member
edited November 2024 in Community Q&A

Hello,

Is there any metric provided to assess robustness / stability of a ML module? As far as I understand the ones provided are more related with the functionality.
I understand reliability as the ability to perform the intended function in the presence of abnormal or unknown inputs, related to ‘reliability’.

Any comment is welcome
Thanks in advance

Tagged:

Answers

  • MartinLiebig
    MartinLiebig
    Altair Employee
    Hi there,
    anything in particular you search for? You could for example use SHAPly and see how much the output changes?

    Best,
    Martin
  • teio
    teio New Altair Community Member
    Hello,

    Thanks for your answer. I have to analyse it but it could work. I am Quality Assurance people and we are searching for metrics provided by the commercial tools to assess the quality of a ML/AI model. In our research, the metrics provided by the mostly used tools in the market, are related to functionality and not to others characteristics mentioned in ISO/IEC standards for AI systems as Robustness, Transparency, User Controllability.

    Our first option is always to use a tool to asssess quality than manual.

    Thanks for your support, Regards Teresa

Categories