"Trouble Running Market Basket Analysis in Radoop Nest"

asl_lestari

I'm running Market Basket Analysis in Radoop Nest. I retrieve my data from Hive. And since there's no FP-Growth and Create Asociation Rules in Radoop Operator, I run them in Spark RM but always finding this trouble:

 

Process failed abnormally

Ooops. Seems like you have found a bug. Please report it in our community at https://community.rapidminer.com. Reason: ERROR : Ended Job = job_1531799644051_135092 with errors Message: Error while processing statement: FAILED: Execution Error, return code 2 from org.apache.hadoop.hive.ql.exec.mr.MapRedTask

eu.radoop.exception.NoStackTraceOperatorException: ERROR : Ended Job = job_1531799644051_135092 with errors Message: Error while processing statement: FAILED: Execution Error, return code 2 from org.apache.hadoop.hive.ql.exec.mr.MapRedTask at eu.radoop.RadoopTools.formattedOperatorException(RadoopTools.java:1665) at eu.radoop.datahandler.hive.HiveHandler.runScriptsOrQueriesKillableAction(HiveHandler.java:1548) at eu.radoop.datahandler.hive.HiveHandler.lambda$runScriptsOrQueriesKillable$5(HiveHandler.java:1250) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:422) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1807) at eu.radoop.datahandler.hive.HiveHandler.runScriptsOrQueriesKillable(HiveHandler.java:1249) at eu.radoop.datahandler.hive.HiveHandler.runScriptsKillable(HiveHandler.java:1204) at eu.radoop.datahandler.hive.HiveHandler.runScriptKillable(HiveHandler.java:1130) at eu.radoop.datahandler.hive.HiveExampleTable.materializeInFileFormat(HiveExampleTable.java:383) at eu.radoop.datahandler.HadoopExampleSet.materializeInFileFormat(HadoopExampleSet.java:880) at eu.radoop.operator.meta.PushdownOperator.deliverOrMaterializeInputHes(PushdownOperator.java:617) at eu.radoop.operator.meta.PushdownOperator.doWork(PushdownOperator.java:236) at com.rapidminer.operator.Operator.execute(Operator.java:1004) at com.rapidminer.operator.execution.SimpleUnitExecutor.execute(SimpleUnitExecutor.java:77) at com.rapidminer.operator.ExecutionUnit$3.run(ExecutionUnit.java:812) at com.rapidminer.operator.ExecutionUnit$3.run(ExecutionUnit.java:807) at java.security.AccessController.doPrivileged(Native Method) at com.rapidminer.operator.ExecutionUnit.execute(ExecutionUnit.java:807) at com.rapidminer.operator.OperatorChain.doWork(OperatorChain.java:428) at eu.radoop.RadoopNest.doWork(RadoopNest.java:662) at com.rapidminer.operator.Operator.execute(Operator.java:1004) at com.rapidminer.operator.execution.SimpleUnitExecutor.execute(SimpleUnitExecutor.java:77) at com.rapidminer.operator.ExecutionUnit$3.run(ExecutionUnit.java:812) at com.rapidminer.operator.ExecutionUnit$3.run(ExecutionUnit.java:807) at java.security.AccessController.doPrivileged(Native Method) at com.rapidminer.operator.ExecutionUnit.execute(ExecutionUnit.java:807) at com.rapidminer.operator.OperatorChain.doWork(OperatorChain.java:428) at com.rapidminer.operator.Operator.execute(Operator.java:1004) at com.rapidminer.Process.execute(Process.java:1310) at com.rapidminer.Process.run(Process.java:1285) at com.rapidminer.Process.run(Process.java:1176) at com.rapidminer.Process.run(Process.java:1129) at com.rapidminer.Process.run(Process.java:1124) at com.rapidminer.Process.run(Process.java:1114) at com.rapidminer.execution.jobcontainer.execution.SimpleExecutor.executeProcess(SimpleExecutor.java:74) at com.rapidminer.execution.jobcontainer.EngineRunner.onApplicationEvent(EngineRunner.java:68) at com.rapidminer.execution.jobcontainer.EngineRunner.onApplicationEvent(EngineRunner.java:30) at org.springframework.context.event.SimpleApplicationEventMulticaster.invokeListener(SimpleApplicationEventMulticaster.java:167) at org.springframework.context.event.SimpleApplicationEventMulticaster.multicastEvent(SimpleApplicationEventMulticaster.java:139) at org.springframework.context.support.AbstractApplicationContext.publishEvent(AbstractApplicationContext.java:393) at org.springframework.context.support.AbstractApplicationContext.publishEvent(AbstractApplicationContext.java:347) at org.springframework.boot.context.event.EventPublishingRunListener.finished(EventPublishingRunListener.java:101) at org.springframework.boot.SpringApplicationRunListeners.callFinishedListener(SpringApplicationRunListeners.java:79) at org.springframework.boot.SpringApplicationRunListeners.finished(SpringApplicationRunListeners.java:72) at org.springframework.boot.SpringApplication.run(SpringApplication.java:305) at com.rapidminer.execution.jobcontainer.Application.main(Application.java:43) Caused by: java.sql.SQLException: Error while processing statement: FAILED: Execution Error, return code 2 from org.apache.hadoop.hive.ql.exec.mr.MapRedTask at org.apache.hive.jdbc.HiveStatement.execute(HiveStatement.java:297) at eu.radoop.datahandler.hive.PooledStatement.execute(PooledStatement.java:365) at eu.radoop.datahandler.hive.ContainerStatementPool$ContainerStatement.execute(ContainerStatementPool.java:67) at eu.radoop.datahandler.hive.HiveHandler.runScriptsOrQueriesKillableAction(HiveHandler.java:1465) ... 45 more

 

which is I don't know what to do. Please help

JEdward

Are you able to share your process so we can see? 

 

Thanks.

 

asl_lestari

Hi,

 

Attached are the screenshots of processes I used to run MBA inside Radoop-Nest operator.

asl_lestari

Hi,

 

Attached are the screenshots of processes I used to run MBA inside Radoop-Nest operator.

phellinger

Hi,

 

The main error message is a general remote Hadoop error message that does not provide too much detail:

"Execution Error, return code 2 from org.apache.hadoop.hive.ql.exec.mr.MapRedTask"

The submitted job failed, but this kind of error can only be examined via checking the particular job log on e.g. the Resource Manager user interface. I hope you can get access to that interface -- it is usually accessible via a browser at <resource_manager_address>:8088 port. You can see your submitted jobs there and follow the "Logs" links (on the right side) for your job until you get to a log where you can see the underlying error. For example, running out of memory, or something other unexpected issue.

 

If there are many jobs running on the cluster, you can figure out the one that was submitted from Radoop by checking the Log panel in RapidMiner (View -> Show Panel -> Log, then right click -> Set log level -> FINER) during / after process run.

 

I hope this helps.

Peter