test failed hive connection when configuring radoop

asmahassani08

hi community,

 

I have successfully installed and configured hadoop 2.6.2, hive 2.1.1 and spark 2.3.1. when i run jps i have this:

1472 NameNode
3856 DataNode
12756 Jps
14164 SparkSubmit
14228 NodeManager
7544 GUILauncher
10860 RunJar
12524 ResourceManager

<SPAN>when i m trying to configure hadoop with rapidminer it gives the below error:<BR /></SPAN>

[Aug 12, 2018 6:19:50 PM]: --------------------------------------------------
[Aug 12, 2018 6:19:50 PM]: Connection test for 'hadoop' started.
[Aug 12, 2018 6:19:50 PM]: Removed test Kerberos beacuse Security is not enabled
[Aug 12, 2018 6:19:50 PM]: Using Radoop version 9.0.0.
[Aug 12, 2018 6:19:50 PM]: Running 11 tests: [Hive connection, Fetch dynamic settings, Java version, HDFS, MapReduce, Radoop temporary directory, MapReduce staging directory, Spark staging directory, Spark assembly jar existence, UDF jar upload, Create permanent UDFs]
[Aug 12, 2018 6:19:50 PM]: Running test 1/11: Hive connection
[Aug 12, 2018 6:19:50 PM]: Hive server 2 connection (localhost:10000) test started.
[Aug 12, 2018 6:19:51 PM] SEVERE: Test failed: Hive connection
[Aug 12, 2018 6:19:51 PM]: Cleaning after test: Hive connection
[Aug 12, 2018 6:19:51 PM]: Total time: 0.059s
[Aug 12, 2018 6:19:51 PM]: java.lang.NoClassDefFoundError: Could not initialize class org.apache.hive.jdbc.HiveConnection
	at org.apache.hive.jdbc.HiveDriver.connect(HiveDriver.java:107)
	at eu.radoop.classloader.HiveDriverShim.connect(HiveDriverShim.java:71)
	at java.sql.DriverManager.getConnection(DriverManager.java:664)
	at java.sql.DriverManager.getConnection(DriverManager.java:247)
	at eu.radoop.datahandler.hive.PooledStatement.lambda$connect$0(PooledStatement.java:214)
	at eu.radoop.tools.ExceptionTools.check(ExceptionTools.java:301)
	at eu.radoop.datahandler.hive.PooledStatement.connect(PooledStatement.java:189)
	at eu.radoop.datahandler.hive.StatementPool.getStatement(StatementPool.java:314)
	at eu.radoop.datahandler.hive.StatementPool.getStatement(StatementPool.java:181)
	at eu.radoop.datahandler.hive.HiveHandler.runFastQueryPrivilegedAction(HiveHandler.java:1010)
	at eu.radoop.datahandler.hive.HiveHandler.runFastQuery(HiveHandler.java:986)
	at eu.radoop.datahandler.hive.HiveHandler.getSimpleListFast(HiveHandler.java:2575)
	at eu.radoop.datahandler.hive.HiveHandler.getTableList(HiveHandler.java:2507)
	at eu.radoop.connections.service.test.connection.TestHiveConnection.call(TestHiveConnection.java:51)
	at eu.radoop.connections.service.test.connection.TestHiveConnection.call(TestHiveConnection.java:25)
	at eu.radoop.connections.service.test.RadoopTestContext.lambda$runTest$1(RadoopTestContext.java:279)
	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	at java.lang.Thread.run(Thread.java:748)

[Aug 12, 2018 6:19:51 PM] SEVERE: java.lang.NoClassDefFoundError: Could not initialize class org.apache.hive.jdbc.HiveConnection
[Aug 12, 2018 6:19:51 PM] SEVERE: Hive server 2 connection test failed. Please check that the server/daemon runs and is accessible on the address and port you specified.
[Aug 12, 2018 6:19:51 PM] SEVERE: Test failed: Hive connection
[Aug 12, 2018 6:19:51 PM] SEVERE: Connection test for 'hadoop' failed.

I work on apache distribution 2.1.1 on windows 8.1 and apache port are default port (10000).

I don't know what is the problem, I'm new with hadoop and hive, is missing some jar files or versions are not compatible??

Thanks in advance,

<SPAN><BR /><BR /><BR /></SPAN>

 

csp75

Hi,

 

Unfortunately, we could not reproduce the issue, two quick questions:

1. Do you have any Hive drivers added to Studio? (Connections > Manage Database Drivers)

2. Could you please attach a full test log? (click Extract logs... on Manage Radoop Connections after quick test fails)

 

Thanks,

Peter

asmahassani08

hi, 

 

for the first point: here you can find the configuration of hive drivers.

the connection status for the hive is unknown !!

in attached the test log 

 

thank you for your help

 

 

asmahassani08

is there any suggestion? I'm still blocked because i'm newbie with those tools.

I think that the problem is because the hiveserver is not starting. I still have this error when I start hiveserver2 :

ERROR StatusLogger No log4j2 configuration file found. Using default configuration: logging only errors to the console.

asmahassani08

[Oct 11, 2018 8:39:57 PM]: Integration test for 'hadoop' started.
[Oct 11, 2018 8:39:57 PM]: Removed test Kerberos beacuse Security is not enabled
[Oct 11, 2018 8:39:57 PM]: Using Radoop version 9.0.0.
[Oct 11, 2018 8:39:57 PM]: Running 25 tests: [Hive connection, Fetch dynamic settings, Java version, NameNode networking, DataNode networking, YARN services networking, HDFS, MapReduce, Radoop temporary directory, MapReduce staging directory, Spark staging directory, Spark assembly jar existence, UDF jar upload, Create permanent UDFs, HDFS upload, Radoop jar upload, Hive load data, Import job, SQL query (aggregation), Spark job, Hive export, UDF query, Distributed Cache, UDAF query, Job kill]
[Oct 11, 2018 8:39:57 PM]: Running test 1/25: Hive connection
[Oct 11, 2018 8:39:57 PM]: Hive server 2 connection (localhost:10000) test started.
[Oct 11, 2018 8:39:57 PM] SEVERE: Test failed: Hive connection
[Oct 11, 2018 8:39:57 PM]: Cleaning after test: Hive connection
[Oct 11, 2018 8:39:57 PM]: Total time: 0.073s
[Oct 11, 2018 8:39:57 PM]: java.lang.NoClassDefFoundError: Could not initialize class org.apache.hive.jdbc.HiveConnection
	at org.apache.hive.jdbc.HiveDriver.connect(HiveDriver.java:107)
	at eu.radoop.classloader.HiveDriverShim.connect(HiveDriverShim.java:71)
	at java.sql.DriverManager.getConnection(DriverManager.java:664)
	at java.sql.DriverManager.getConnection(DriverManager.java:247)
	at eu.radoop.datahandler.hive.PooledStatement.lambda$connect$0(PooledStatement.java:214)
	at eu.radoop.tools.ExceptionTools.check(ExceptionTools.java:301)
	at eu.radoop.datahandler.hive.PooledStatement.connect(PooledStatement.java:189)
	at eu.radoop.datahandler.hive.StatementPool.getStatement(StatementPool.java:314)
	at eu.radoop.datahandler.hive.StatementPool.getStatement(StatementPool.java:181)
	at eu.radoop.datahandler.hive.HiveHandler.runFastQueryPrivilegedAction(HiveHandler.java:1010)
	at eu.radoop.datahandler.hive.HiveHandler.runFastQuery(HiveHandler.java:986)
	at eu.radoop.datahandler.hive.HiveHandler.getSimpleListFast(HiveHandler.java:2575)
	at eu.radoop.datahandler.hive.HiveHandler.getTableList(HiveHandler.java:2507)
	at eu.radoop.connections.service.test.connection.TestHiveConnection.call(TestHiveConnection.java:51)
	at eu.radoop.connections.service.test.connection.TestHiveConnection.call(TestHiveConnection.java:25)
	at eu.radoop.connections.service.test.RadoopTestContext.lambda$runTest$1(RadoopTestContext.java:279)
	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	at java.lang.Thread.run(Thread.java:748)

[Oct 11, 2018 8:39:57 PM] SEVERE: java.lang.NoClassDefFoundError: Could not initialize class org.apache.hive.jdbc.HiveConnection
[Oct 11, 2018 8:39:58 PM] SEVERE: Hive server 2 connection test failed. Please check that the server/daemon runs and is accessible on the address and port you specified.
[Oct 11, 2018 8:39:58 PM] SEVERE: Test failed: Hive connection
[Oct 11, 2018 8:39:58 PM] SEVERE: Integration test for 'hadoop' failed.

Radoop connection log is attached.

can you help me please, I'm stuck for months?

csp75

Hi,

 

Still couldn't reproduce your error

1. Can you connect to that Hive Server with another tool (e.g. beeline)?

2. Could you please remove the Hive driver on the "Manage Database Drivers" dialog and re-run the Radoop tests? (Radoop uses its own set of database drivers, there is no need to add it manually.)

 

 

Peter

asmahassani08

1- beeline command doesn't work, I have a problem to connect to hiveserver  beeline.

2- for the second point, I removed the hive driver and Radoop still not working.

The problem is from hive configuration I'm missing something.

asimon

Hi,

Beeline is the fundamental cli for accessing hive. I usually test the connection with it first since if it doesn't work then it is a clear sign of a misconfiguration: either hive server or network. Based on the last attached gg.zip the connection cannot be established on the transport level which probably means that your Hive server is not listening on the specified localhost:10000 port. (maybe it is not even running as you already indicated, and confirmed by the missing entry for hive from jps output)

I would suggest you to make sure you have hiveserver2 running then beeline is able to connect to it and only then retry with Radoop. If even beeline is not able to connect then a more complex tool like Radoop won't be functional. For further help with hive or beeline I would suggest to contact the apache hive community lists. 

Attila

asmahassani08

I have successfully configured radoop but the problem persists for hive connector.

attached the interfaces of configuring hive. I can't detect the source of the error. always the connection status is unknown.

 

asmahassani08

on prompt : hiveserver2 is started and I can connect to hive beeline on this port.

I'm really blocked for months and the problem persists