"store a ROC plot for each iteration of a subprocess"

Legacy User

Hello,

I routinely use ROC plots to compare different learning algorithms and parameters employed. When not using rapidMiner I generally dump multiple ROC plots to disk for every parameter value, feature selection round, etc.

I can't seem to find a way to do the same in RapidMiner. Is it possible?

I would prefer a solution that did not require to use the GUI, because even though I use it to design workflows, whenever I need to run RapidMiner on a full dataset I need to use it from the command line on a computing cluster.
Moreover, I usually prefer to store the raw data so I can then reproduce the plots in my graphic library of choice (R or matplotlib).

Hence, I was wondering if there was a way to automatically export or save to disk ROC plots (as images or even better as raw data)
For eg. in backward/forward attribute selection, I'd like to compare the ROC curve for every generation.

Things I have thought/tried so far:

- I don't see a 'write ROC' operator

- I tried using the 'write Performance' operator, but I find that RapidMiner cannot read the result file thus generated (neither opening it through the GUI or through the 'Read Performance' operator)

- I have thought of using 'write Performance' and then parse the resulting XML file via python outside of RapidMiner, but I still can't figure out how to write a separate file for every iteration of the subprocess. Is there a particular operator that can add a suffix to the filename and increment its value for every loop, or something similar?

Many thanks,
eli

land

Hi Eli,
give the Reporting Extension a try. It offers a ReportGenerator to open a Report into various file formats. Then insert a Report operator to add a specific IOObject to the report. For example a plot of the roc chart.
Of course you can additionally add text for example describing the current parameter setting. Macros help you a big deal there.

Greetings,
  Sebastian