how to get performance of test data where the label has no values
Hi All,
I tried to find out how my model is performing on Training data and I was able to do it successfully
Now I wanted to see how it is going to perform on test data so I added another apply model and performance and of course my test data like below
I got below errors: That's probably because my "label parameter" is blank in "test data" as I wanted to see what values it will predict....... I am able to get results of prediction but to see how my model is performing on completely new set of data with no values in label.... can we do that if yes then how?
if I am trying to put "Set Role" in between "Apply model" and "Performance" I am able to set that predicted variable as my "label" which is not right because that predicted variable column is not present in the original test data so that's not working
I tried to find out how my model is performing on Training data and I was able to do it successfully
Now I wanted to see how it is going to perform on test data so I added another apply model and performance and of course my test data like below
I got below errors: That's probably because my "label parameter" is blank in "test data" as I wanted to see what values it will predict....... I am able to get results of prediction but to see how my model is performing on completely new set of data with no values in label.... can we do that if yes then how?
squared_error: unknown
root_mean_squared_error: unknown
if I am trying to put "Set Role" in between "Apply model" and "Performance" I am able to set that predicted variable as my "label" which is not right because that predicted variable column is not present in the original test data so that's not working
Find more posts tagged with
Sort by:
1 - 7 of
71
to see how my model is performing on completely new set of data with no values in label.... can we do that if yes then how?
Nope, regular performance metrics cannot be calculated without the original known label.@varunm1
Thank you for your response.
My next step was to run it without performance and save the results in an excel file then I ran that excel as an input to the same model to see the error rate and it came as 0. Can you tell me why?
Please see below screenshot
Thank you for your response.
My next step was to run it without performance and save the results in an excel file then I ran that excel as an input to the same model to see the error rate and it came as 0. Can you tell me why?
Please see below screenshot
I did one more thing and I think I did it right this time.
The result set that was generated above was from the model and feeding same data to the model obviously would show 0 deviation.
Now I put original data for example I predicted response rate for Nov 2019 and I already have the actual/original so I fed that as an input to see how much the result set would deviate from original and I got root mean squared error as 0.016
which isn't bad what do you think?
The result set that was generated above was from the model and feeding same data to the model obviously would show 0 deviation.
Now I put original data for example I predicted response rate for Nov 2019 and I already have the actual/original so I fed that as an input to see how much the result set would deviate from original and I got root mean squared error as 0.016
which isn't bad what do you think?
If this nov 2019 data is not in your training then the RMSE is low, which is fine. You can try different models and see if you could get better
Yes. Nov 19 data was not in training set but I feel I have very limited options on how many models I can run. I see only 3 models, mostly 2 GBT, & random forest to work with my data as it has only 1 real/int variable which is response rate and all others are polynomial.
Now I put original data for example I predicted response rate for Nov 2019 and I already have the actual/original so I fed that as an input to see how much the result set would deviate from original and I got root mean squared error as 0.016
Basically to find the performance of newly scored data you will need to wait until enough time passes for you to assign the label using the same logic that was embedded in your original model development sample. Then you can load that in and merge it with the dataset containing the predictions, and then use the typical performance operators on that combined dataset to see how the model did.
Sort by:
1 - 1 of
11
Yes. Nov 19 data was not in training set but I feel I have very limited options on how many models I can run. I see only 3 models, mostly 2 GBT, & random forest to work with my data as it has only 1 real/int variable which is response rate and all others are polynomial.