Accessing Azure Databricks Resources with Service Principals via OAuth

OAuth is the preferred protocol for secure user authentication and authorization when accessing Databricks resources outside of the UI — whether you’re automating jobs, managing clusters, or integrating with external systems.

In this blog, I’ll walk you through how to generate OAuth tokens after setting up and authorizing Service Principals — from both Azure (Microsoft Entra) and Databricks — to interact with your Databricks workspace securely and efficiently.

What is a Service Principal?

A service principal (SP) is a non-human identity or a security principal used by applications or automated tools to access resources within a cloud environment like Azure or Databricks.

Option 1 — Azure Service Principal

An Azure Service Principal is created by going into Microsoft Entra ID and creating a new app registration. This process will create an application/client id. Once created, you go to the Certificates & secrets to generate a new client secret.