Loop through exampleset and identify a public holiday and then set all 24 hours to 1 in variable

Hi,

how are your data structured? Do you have a list of public holidays (dates) and the timestamps (on hour level)? 

You could try something like this in Generate Attributes to create an additional attribute with just the date part of your timestamp:

<div>date_add(timestamp_attribute,&nbsp;
<span style="background-color: transparent; color: inherit; font-size: inherit;">    <font face="Open Sans, sans-serif">- date_get(timestamp_attribute, DATE_UNIT_HOUR),&nbsp;</font>
</span><span style="background-color: transparent; color: inherit; font-size: inherit;">    </span><span style="background-color: transparent; color: inherit; font-size: inherit; font-family: "Open Sans", sans-serif;">DATE_UNIT_HOUR)</span></div>

Hi Balazs
Many thanks the data set has a date-time variable so yes can extract hour of day for days that are flagged as public holidays - holiday variable is either "None" or the name of the public holiday such as "Christmas Day" so it is a polynominal variable in RapidMiner

However the problem I have if that data set only flags the first hour of each public holiday so the other 23 hours for that holiday are incorrectly flagged as "None" so using the example above the first hour of Christmas day 25/12/2012 is flagged as "Christmas Day" the problem is that other 23 hours are not currently flagged as "Christmas Day" but rather "None".

I am thinking of using an if statement to identify a public holiday such as != "None" and use a macro and loop to set the other 23 hours to "Christmas Day" and run through the entire exampleset for each holiday could work, see snapshot of data set below


Regards Michael

Hi,

if you have the large table (the one with the timestamps) and create the date attribute, it will have 24 entries for each day, with the date.

Then you have a second table, I assume, with the holiday dates. 

In this case you can simply join the two tables using the Join operator, selecting the date in both tables as the join attribute. This will assign the holiday to the date/time entries. Use "left join" to keep all entries of the timestamp table, then fill missing values in the resulting table with "None".

Regards,

Balázs

Hi Balázs
I am using date to nominal to reformat the date_time to date format MM/dd/yyyy and then I am using the Filter Examples operator to get the dates with days that are flagged as public holidays then as you suggest I am using join operator and doing left join on the date_time with full data set with date_time also reformatted to date format MM/dd/yyyy see rapidminer process screen capture now I am getting all dates across 24 hours for each holiday but I am not getting the full data set with other days listed as None in holiday variable regards Michael

Hi, 

you're sending the filtered list to the "left" input and the unfiltered one to the "right" input. But you're doing left outer join.
For this setup you need a right join. This keeps everything from the right input and the matching data from the left input.

Check out this short video on Academy:
https://academy.rapidminer.com/learn/video/joining-and-cleansing-intro

Regards,
Balázs

Hi Balazs many thanks this sorted out my problem although I still needed to remove duplicate attributes and replace ? with None after watching the intro video on cleansing data I also used the trim operator and remove duplicates operator really appreciate your help with this regards Michael