Evaluating Anomaly and signature detection methods

Hi, 

I am a 4th year student trying to do an experiment comparing signature based and anomaly based detection methods. I would like to do this using decision trees and random forest algorithms. The end goal would be to measure the rate of false positives in both methods and to conclude with which is better for deterring cyber attacks.  

I am not too sure how I am going to undertake this experiment but RM looks very helpful. I have publicly available security logs to use as data, have downloaded the anomaly detection extension, text mining extension and have set up an IDS system in Sec Onion to monitor my network. 

Any advice/solutions would be much-appreciated apologises if ive not made what im doing clear enough I am far from an expert on the subject 

Cheers