[Databricks on AWS #4] The BOOTSTRAP_TIMEOUT Mystery: Tracing a Databricks Cluster from Data Plane to Control Plane (Transit Gateway + Firewall)

Short summary

Databricks cluster bootstrap failed with BOOTSTRAP_TIMEOUT despite healthy EC2 instances and correct routing in a customer-managed VPC. Systematic tracing revealed a centralized egress firewall was silently dropping outbound traffic to the Databricks control plane relay. Solution: allow-list the workspace CIDR for outbound 443 traffic to the Databricks relay, or use AWS PrivateLink for encrypted connectivity.

• •BOOTSTRAP_TIMEOUT + healthy EC2 + correct routes = egress firewall blocking control plane connectivity
• •Root cause identified via firewall logs: new workspace CIDR not in allow-list for outbound 443 traffic
• •Fix: allow outbound 443 to tunnel.<region>.cloud.databricks.com, or use AWS PrivateLink

Generated with AI, which can make mistakes.