Connecting Databricks from Azure App Service using Hive-JDBC Driver

Image for post
Image for post

You may have a use case where you need to query and report data from Hive. Azure Databricks supports various Business Intelligence tools (ref: https://docs.azuredatabricks.net/user-guide/bi/index.html) to serve the purpose.

In this example, we’ll be creating an Azure App Service which will fetch records using Hive-JDBC driver.

  1. Create a sample table at Azure Databricks.

2. Generate a user token

3. Note down the JDBC URL. The JDBC/ODBC option is available only with the Premium plan.

4. Download a JDBC Driver with all the dependencies from: https://docs.azuredatabricks.net/user-guide/bi/workbenchj.html. In my case, it’s uber-hive-jdbc-1.2.1.jar

5. Create a Web Project (follow the link for steps: https://docs.microsoft.com/en-us/azure/app-service/app-service-web-get-started-java) and include the jar under /WEB-INF/lib/.

6. Write the following code in index.jsp (this is just for an example, ideally we shouldn’t write the logic inside the JSP).

6. Once done, deploy to publish the web app to Azure. While uploading/publishing through Eclipse, it may take some time. For faster deployment, you can evaluate other Azure App Service deployment options.

Once deployed and service started, open the App Service URL:

The above approach (connecting to Databricks to query data in tables) should be used in case we’re building an interactive application and the returned records are not large. In case, our query could fetch large records we should rather submit a job at Azure Databricks cluster (e.g. using Jobs API — https://docs.azuredatabricks.net/user-guide/jobs.html).

Written by

Tech enthusiast, Azure Big Data Architect.

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store