[Databricks on AWS #3] Compute Governance on Databricks: Instance Pools, Cluster Policies, and Shared Clusters

Short summary

Compute governance on Databricks across AWS prevents runaway cloud costs and enforces role-based access using three layers: instance pools (pre-warmed VMs for performance), cluster policies (constraints on instance types and autotermination), and entitlement gates (allow_cluster_create by role). Includes concrete AWS EC2 configurations and policy templates—essential operational knowledge for Databricks platform teams.

• •Three-layer governance model: instance pools → cluster policies → entitlement gates
• •Concrete AWS instance configurations and autotermination rules prevent five-figure bill surprises
• •Role-based access patterns (admin/engineer/analyst) with shared cluster fallback for restricted users

Generated with AI, which can make mistakes.