Altair RISE

A program to recognize and reward our most engaged community members

Nominate Yourself Now!

Significance testing

Hi all,

I'm interested in comparing the performance of an algorithm to chance with a significance test. I understand there's a t-test operator that may allow me to do this but I'm not sure how exactly. Could anyone offer some advice?

Thanks!

Find more posts tagged with

AI Studio

Accepted answers

All comments

earmijo

There is an operator T-test. There is an example process that illustrates its use. Click on the operator (Evaluation/Significante) and in the help Tab you'll get a description of the operator and a link to the illustrative process.

lsevel

I'm aware of this operator but it assumes a paired t-test and requires two input vectors. Is it necessary to build a 'chance' classifier or is there another way to compare one model to chance?

rachel_lomasky

I also have this question. I'd like to do a proportion test on rows of a data grid, not model performance vectors. Is this possible?

Thanks,

Rachel

sgenzer

Hello - not sure what you mean by a “proportion test”. Maybe you’re looking for a correlation matrix?

Scott

rachel_lomasky

What is prop.test in R. Which I just ended up using as a RapidMiner operator.

sgenzer

Ah I see. Never heard of prop.test but it looks like basically chi-squared independence / goodness of fit to me. @land does your stats extension do this? Anything in the toolbox, @mschmitz?

Scott

land

Hi,

we have Chi-Square test but not no direct prop.test equivalent. But seems it is solved by integrating R?

Greetings,

Sebastian

rachel_lomasky

Yeah, I'm happy with just using the R function.