Hive-Spark Data Import

David_21136
David_21136 Altair Community Member
edited March 2019 in Community Q&A
Hi, I want to use Spark to do a Hive Table Import, but all my boxes are greyed out. Do I need to change any setting in the software to allow me to do this?
 
Thanks!
 
Dave Roughton
Analysis, Insights and Metrics
COUNTRY Financial
< <mailto:dave.roughton@countryfinancial.com>>    
(309) 821-5188
 
 
 
Tagged:

Answers

  • Christopher_22023
    Christopher_22023
    Altair Employee
    edited March 2019
    Hi David,

    You can only do a Hive import using a Hive ODBC driver with the standard KnowledgeSTUDIO/SEEKER application.   Those icons indicating Apache SPARK functionality are only available with a "KnowledgeSTUDIO for Apache SPARK" license, but then using a SPARK import doesn't actually import to your server instance or desktop, but to a shared memory space as a data frame, which is the primary functionality of SPARK.

    Hope this helps.     

    .Chris

    ------------------------------
    Christopher Long
    Head of Presales KnowledgeStudio/Knowledge Seeker
    Datawatch Corporation
    Toronto ON
    (416) 593-2422
    ------------------------------
    -------------------------------------------
    Original Message:
    Sent: 03-25-2019 12:23 PM
    From: David Roughton
    Subject: Hive-Spark Data Import

    Hi, I want to use Spark to do a Hive Table Import, but all my boxes are greyed out. Do I need to change any setting in the software to allow me to do this?
     
    Thanks!
     
    Dave Roughton
    Analysis, Insights and Metrics
    COUNTRY Financial
    < <mailto:dave.roughton@countryfinancial.com>>    
    (309) 821-5188
     
     
     
  • dmitri_1
    dmitri_1
    Altair Employee
    edited March 2019
    Christopher_22023 said:

    Hi David,

    You can only do a Hive import using a Hive ODBC driver with the standard KnowledgeSTUDIO/SEEKER application.   Those icons indicating Apache SPARK functionality are only available with a "KnowledgeSTUDIO for Apache SPARK" license, but then using a SPARK import doesn't actually import to your server instance or desktop, but to a shared memory space as a data frame, which is the primary functionality of SPARK.

    Hope this helps.     

    .Chris

    ------------------------------
    Christopher Long
    Head of Presales KnowledgeStudio/Knowledge Seeker
    Datawatch Corporation
    Toronto ON
    (416) 593-2422
    ------------------------------
    -------------------------------------------
    Original Message:
    Sent: 03-25-2019 12:23 PM
    From: David Roughton
    Subject: Hive-Spark Data Import

    Hi, I want to use Spark to do a Hive Table Import, but all my boxes are greyed out. Do I need to change any setting in the software to allow me to do this?
     
    Thanks!
     
    Dave Roughton
    Analysis, Insights and Metrics
    COUNTRY Financial
    < <mailto:dave.roughton@countryfinancial.com>>    
    (309) 821-5188
     
     
     
    Hi Dave, 

    To add to what Chris wrote above: 
    If you do not have a Hive ODBC driver on your machine, you can download and install one from the Cloudera or Hortonworks, for example. The drivers are free. Alternatively, you may ask your Hadoop Database Admin team what driver they would recommend.
    Here is a link to the latest version of Hive ODBC Connector for Cloudera:
    https://www.cloudera.com/downloads/connectors/hive/odbc/2-6-1.html

    After you install the driver, you can use the ODBC import node to import tables from Hive. If you need further guidance, a call or a WebEx meeting to assist you, feel free to email AngossSupport@datawatch.com.

    Best regards, 
    Dmitri Dotsenko

    ------------------------------
    Dmitri Dotsenko
    Analyst
    Datawatch Corporation
    Toronto ON
    (416) 593-1122 x2483
    ------------------------------
    -------------------------------------------
    Original Message:
    Sent: 03-26-2019 07:56 AM
    From: Christopher Long
    Subject: Hive-Spark Data Import

    Hi David,

    You can only do a Hive import using a Hive ODBC driver with the standard KnowledgeSTUDIO/SEEKER application.   Those icons indicating Apache SPARK functionality are only available with a "KnowledgeSTUDIO for Apache SPARK" license, but then using a SPARK import doesn't actually import to your server instance or desktop, but to a shared memory space as a data frame, which is the primary functionality of SPARK.

    Hope this helps.

    .Chris

    ------------------------------
    Christopher Long
    Head of Presales KnowledgeStudio/Knowledge Seeker
    Datawatch Corporation
    Toronto ON
    (416) 593-2422
    ------------------------------

    Original Message:
    Sent: 03-25-2019 12:23 PM
    From: David Roughton
    Subject: Hive-Spark Data Import

    Hi, I want to use Spark to do a Hive Table Import, but all my boxes are greyed out. Do I need to change any setting in the software to allow me to do this?
     
    Thanks!
     
    Dave Roughton
    Analysis, Insights and Metrics
    COUNTRY Financial
    < <mailto:dave.roughton@countryfinancial.com>>    
    (309) 821-5188
     
     
     

Categories