validation

vps

Hi,
What are types of validation in Auto Model (Bootstrapping Validation, Cross Validation, Split Validation, Wrapper Split Validation, Wrapper-X-Validation) ?

jacobcybulski

As I can see the Auto Models are validated using multiple hold-out validations, which are then averaged using Performance Average (Robust). So this is not a "standard" RM process. When you generate an Auto Model, you can inspect each process individually by clicking into "Open Process".

varunm1

Hello @vps

Its a multi hold out validation as informed by @jacobcybulski. The data is split into 60:40 (train:test) ratio. The 40 % of test data is again split into 7 subsets and then all the 7 subsets are tested and their performances are aggregated to get the final performance in auto model.

lionelderkrikor

HI @vps,

Yes, there is a short description of how is validated the model in the final screen of Auto-Model (the results screen):
just click on the "information (i)" button : 



Regards,

Lionel

jacobcybulski

As I can see the Auto Models are validated using multiple hold-out validations, which are then averaged using Performance Average (Robust). So this is not a "standard" RM process. When you generate an Auto Model, you can inspect each process individually by clicking into "Open Process".

varunm1

Hello @vps

Its a multi hold out validation as informed by @jacobcybulski. The data is split into 60:40 (train:test) ratio. The 40 % of test data is again split into 7 subsets and then all the 7 subsets are tested and their performances are aggregated to get the final performance in auto model.

vps

Thanks!

Is there any documentation regarding the validation in Auto Model?

lionelderkrikor

HI @vps,

Yes, there is a short description of how is validated the model in the final screen of Auto-Model (the results screen):
just click on the "information (i)" button : 



Regards,

Lionel

vps

Thanks a lot, Lionel. That seems to completely fix the issue.