Azure Databricks Cost Optimizations

Databricks has now become a default choice of service for big data computation in Azure, by its own merit. As more and more clients are embracing it (and Apache Spark) with their versatile use cases, some people started complaining about the hefty Azure bill they’re getting and Azure Databricks’ contribution on that!

Though cloud services has brought infrastructures & services provisioning time from months to seconds however, appropriate governance & controls have become more important.

So, instead of blaming the cloud services (here, Databricks) why not we learn the cost optimization techniques and spend money based on our business needs only.

Identifying Costs

First, we’ll check on how to get the cost information for Azure Databricks. So, straight away we’ll go to the Cost Management + Billing section & will select the Cost Management > Cost analysis for the subscription.

Cost Management > Cost analysis — Actual & Forecast Costs.

Though we generally look for the azure databricks from the Service name dashboard but, that’ll only give the cost of the Azure Databricks service; the actual cost should be more if we consider the cost contributed by the Azure…