Log generation in Rapidminer Studio

ZahraSadeghiboga92

I have seen the execution log generated by AI Hub which reports individual steps of execution of a process.

INFO: Started operator: Process
Jan 13, 2022 7:35:55 PM com.rapidminer.execution.jobcontainer.execution.ExecutionProcessListener processStartedOperator
INFO: Started operator: Loop Files
Jan 13, 2022 7:35:55 PM org.rapidprom.operators.extract.ExtractXLogOperator read
INFO: Start: importing event log
Jan 13, 2022 7:35:55 PM org.rapidprom.operators.extract.ExtractXLogOperator read
INFO: Start: importing event log
Jan 13, 2022 7:35:55 PM org.rapidprom.operators.extract.ExtractXLogOperator read
INFO: Start: importing event log
Jan 13, 2022 7:35:55 PM org.rapidprom.operators.extract.ExtractXLogOperator read
INFO: End: importing event log (0 sec)
Jan 13, 2022 7:35:55 PM org.rapidprom.operators.logmanipulation.AddClassifierOperator doWork
INFO: Start: add classifier
Jan 13, 2022 7:35:55 PM org.rapidprom.operators.logmanipulation.AddClassifierOperator doWork

However, I need more verbose logging which can be used to chain together the sequence of events that is taking place in a complex process.

At this point, the only solution seems to be modifying Rapidminer Studio source code (github.com/rapidminer/rapidminer-studio) to emit such a process log. Can you please point out to me wherein the source code does it make sense to perform this?

Thanks in advance,

MartinLiebig

Hi,

not sure if it helps, but the main process as a parameter called log verbosity. Maybe you just want to increase it?

BR,

Martin

MartinLiebig

Oh, and you can write into this log using the Write to Console operator.

ZahraSadeghiboga92

Hello,

I think that log level parameter only affects the operational log of the RapidMiner and not the execution log of the processes.

To clarify, I should add, that the only information about an operator in the execution log is the start and end time. But there is no information that can help us chain together a sequence of related events when there is a parallel Loop Files operator in a process.

For instance, the log will have several starts and end times for executing an operator without knowing which one is for which iteration of the loop.

Inside "loop files"

Moreover, I want to log the parameters that are passed to each operator.

I have attached a sample of an existing execution log, and also what it should ideally look like:

Actual execution log:

INFO: Started operator : Process
Jan 15, 2022 4:51:04 PM com.rapidminer.execution.jobcontainer.execution.ExecutionProcessListener processStartedOperator
INFO: Started operator : Loop Files
Jan 15, 2022 4:51:04 PM org.rapidprom.operators.extract.ExtractXLogOperator read
INFO: Start: importing event log
Jan 15, 2022 4:51:04 PM org.rapidprom.operators.extract.ExtractXLogOperator read
INFO: Start: importing event log
Jan 15, 2022 4:51:04 PM org.rapidprom.operators.extract.ExtractXLogOperator read
INFO: Start: importing event log
Jan 15, 2022 4:51:04 PM org.rapidprom.operators.extract.ExtractXLogOperator read
INFO: End: importing event log (0 sec)
Jan 15, 2022 4:51:04 PM org.rapidprom.operators.extract.ExtractXLogOperator read
INFO: End: importing event log (0 sec)
Jan 15, 2022 4:51:05 PM org.rapidprom.operators.extract.ExtractXLogOperator read
INFO: End: importing event log (0 sec)
Jan 15, 2022 4:51:05 PM com.rapidminer.execution.jobcontainer.execution.ExecutionProcessListener processFinishdOperator
INFO: Finished operator : Loop Files
Jan 15, 2022 4:51:05 PM com.rapidminer.execution.jobcontainer.execution.ExecutionProcessListener processFinishedOperator
INFO: Finished operator : Process

Expected execution log:

INFO: Started operator : Process

Jan 15, 2022 5:18:35 PM com.rapidminer.execution.jobcontainer.execution.ExecutionProcessListener processStartedOperator

INFO: Started operator : Loop Files

Jan 15, 2022 5:18:35 PM org.rapidprom.operators.extract.ExtractXLogOperator read

INFO: Start: importing event log second_file.xes

Jan 15, 2022 5:18:35 PM org.rapidprom.operators.extract.ExtractXLogOperator read
INFO: Start: importing event log first_file.xes

Jan 15, 2022 5:18:35 PM org.rapidprom.operators.extract.ExtractXLogOperator read
INFO: End: importing event log (0 sec) first_file.xes

Jan 15, 2022 5:18:38 PM org.rapidprom.operators.extract.ExtractXLogOperator read
INFO: End: importing event log (3 sec) second_file.xes

Jan 15, 2022 5:18:38 PM com.rapidminer.execution.jobcontainer.execution.ExecutionProcessListener processFinishedOperator
INFO: Finished operator : Loop Files

Jan 15, 2022 5:18:38 PM com.rapidminer.execution.jobcontainer.execution.ExecutionProcessListener processFinishedOperator
INFO: Finished operator : Process

My guess is that I will have to modify the source code of RapidMiner.

Can you please point out to be the appropriate place in the source code where this can be done?

MartinLiebig

Hi,

why don't you use the normal Log operator for it? It can just log parameter and execution times of operators. Not sure why you  want to code something.

Best,

Martin

ZahraSadeghiboga92

Hi

I want to get execution log from any existing workflow, as is, without modifying it.

So for example, some else might have provided a large complex workflow, and I want to get execution log from it. modifying it may not be a feasible option.

wadhvakartik22

I think that log level parameter only affects the operational log of the RapidMiner and not the execution log of the processes.  mobdro

MartinLiebig

@ZahraSadeghiboga92 even if we would add a log to every operator execution, you cannot just add code to the AIHub software. Those code pieces of course need to be signed by RM.

ZahraSadeghiboga92

Hi,

I probably could change the source code of RapidMiner Studio to get a log like what I said, right?

Rapidminer has a large codebase. Can you please let me know how can I get some hints about which part of the code is the best place to get started with this?

Thanks in advance