Missing Test Date Column in Forecast Validation

SkyTrader

Hi
I was following the Elaborate Your Time Series Analysis video: https://www.youtube.com/watch?time_continue=2182&v=Hvdh8ItfiGA&feature=emb_logo

It clearly shows a date column when you're doing a performance (regression) analysis using ARIMA. Pls see YouTube "Elaborate Your Time Series" video image.


I set up my process the same as the video but don't have this test date to refer to?
Pls see images:



and yet I set the attribute as Date as in the video instructions? 



So could someone please tell me how do I get a date reference?

Also is there a way to swap the order of the close column and the forecast column so that the forecast column comes first before the actual close?

Cheers,

tftemme

That's sound like a problem with the meta data propagation. For the background: Meta data is used to provide such information as attribute names and the like in the parameters, when the process is not yet executed (while you configure it). Sometimes it can happen that there is a problem with it, or it can also happen that the meta data cannot be evaluated, cause the information would depend on the execution of the operators itself and are not known upfront.

Without looking into the whole process, I cannot figure out what happens in your case. You can always check the meta data (so what information are available), by hovering with your mouse courser over the ports. So check the 'training' output port before the ARIMA operator and maybe also the 'example set' input port of the Forecast Validation operator.

Also you can always just type in the name of the attributes directly. So just add 'Date' for the indices attribute.

Hopes this helps
Best regards,
Fabian

SkyTrader

Thanks for the help @tftemme, much appreciated.

I decided to go back to my Read Excel and used the import wizard again and all the attributes are now populating the dropdown menu and now I've set the "has indices" to Date, and the date column has appeared and everything looks good!

tftemme

Great.

What you also can do, to better handle Meta Data and the like is to put the Read Excel operator into another process, and then directly Store the data set in the repository. Then you can retrieve the data set in the original process from repository, instead of always reading it from disc.

This has the advantage that it can potentially speed up the process (the reading can be slow), and that data sets in the repository always have proper Meta Data stored.

Don't forget to execute the reading process, when you have updated input data.

tftemme

Hi @SkyTrader

Have you set your Date attribute as the indices attribute as well at the ARIMA operator? 

Normally you can reorder attributes by using the "Reorder Attributes" operator. Unfortunately in the result view, attributes with special roles (in this case the 'Label' and the 'Prediction' role) are shown in the beginning of the table and also in a fixed order. 

If you really need a different order, you can use Set Role operator to convert them into regular attributes and then use Reorder Attributes to order as you need them. You loose the special role information by doing this, though.

Hopes this helps
Best regards,
Fabian

SkyTrader

Hi @tftemme, cheers for getting back to me and thanks for the Elaborating Times Series video!

"Have you set your Date attribute as the indices attribute as well at the ARIMA operator?"

I don't think so... but when I go to do that the attributes drop down values that automatically populated the drop down menu are now all missing? They used to show all the attributes of my Excel sheet. Pls see image:



Thank you for the attribute sort order tips.

Best regards
Sky Trader

tftemme

That's sound like a problem with the meta data propagation. For the background: Meta data is used to provide such information as attribute names and the like in the parameters, when the process is not yet executed (while you configure it). Sometimes it can happen that there is a problem with it, or it can also happen that the meta data cannot be evaluated, cause the information would depend on the execution of the operators itself and are not known upfront.

Without looking into the whole process, I cannot figure out what happens in your case. You can always check the meta data (so what information are available), by hovering with your mouse courser over the ports. So check the 'training' output port before the ARIMA operator and maybe also the 'example set' input port of the Forecast Validation operator.

Also you can always just type in the name of the attributes directly. So just add 'Date' for the indices attribute.

Hopes this helps
Best regards,
Fabian

SkyTrader

Thanks for the help @tftemme, much appreciated.

I decided to go back to my Read Excel and used the import wizard again and all the attributes are now populating the dropdown menu and now I've set the "has indices" to Date, and the date column has appeared and everything looks good!

tftemme

Great.

What you also can do, to better handle Meta Data and the like is to put the Read Excel operator into another process, and then directly Store the data set in the repository. Then you can retrieve the data set in the original process from repository, instead of always reading it from disc.

This has the advantage that it can potentially speed up the process (the reading can be slow), and that data sets in the repository always have proper Meta Data stored.

Don't forget to execute the reading process, when you have updated input data.

SkyTrader

Cheers @tftemme
I actually do have a Read/Store process saved in processes but for some reason I think because I was going to be updating the Excel file so regularly I went with using the Read Excel operator (from my drive) on it's own.