[Databricks on AWS #5] Fixing Databricks BOOTSTRAP_TIMEOUT with AWS PrivateLink: Control Plane Over the Backbone, Zero New Subnets

Short summary

Databricks on AWS Part 5 solves BOOTSTRAP_TIMEOUT—the error that occurs when cluster nodes can't reach the control plane—by routing traffic through AWS PrivateLink instead of public internet paths. The elegant implementation places interface VPC endpoint ENIs directly into existing cluster subnets, consuming only four private IPs and requiring zero new subnets, CIDR ranges, or route changes. Private DNS rewrites only Databricks domains, leaving all other workloads and infrastructure completely untouched.

• •AWS PrivateLink eliminates BOOTSTRAP_TIMEOUT by routing Databricks control plane traffic through the AWS backbone
• •Implementation requires no new subnets or CIDR changes—interface endpoints use existing cluster subnet IPs
• •Private DNS targeting is surgical: only Databricks domains are rewritten, leaving other services and workloads unchanged

Generated with AI, which can make mistakes.