"Collect Test Results from Sliding Window Validation"

Question

Hi,

i want to accomplish the following:

I have a time series dataset of ~3000 examples. i'm using a sliding window validation that learns a model on 500 datapoints and then applies the model on the next 50. then i want the labeled data of each testing period and concatenate these results to a single example set.

i already have a solution for this, but it's very unflexible:

in the testing period, i have a store operator, that stores the labeled data as an example set in my repository. i use the %{a} macro, to store the data in different locations. as a result i get many, many example sets, which i concatenate in a separate process.

is there an easier way to accomplish this? my solution is very unflexible if i want to change the testing window length, etc.

alex

Eric005 · Answer

Is there any way the individual example sets from a sliding window operator can be stored?  I would like to be able to visually inspect these and correlate them to our master dataset for a more granular look at the predictions/estimations.

land · Answer

Hi,
well there's unfortunately no single operator solving the issue, but your process setup can be tuned. You can head for collections of ExampleSets. This would make it more easier, because: You can simply add an exampleSet to the collection and store it back then, overwriting the previous stored (smaller) collection. Then after all exampleSets have been collection you can retrieve the collection and use one single Append operator to merge all sets at once without loop or something like that.

Greetings,
  Sebastian