Pin Databricks Clusters

This post is for anyone who is unaware that interactive Databricks clusters can be deleted 30 days after termination, unless the cluster is “pinned”.

After 30 days…

Rather than write more on this well documented aspect of managing clusters, I’ll refer you to the Azure Databricks documentation for more on how to pin your all-purpose clusters.

Other than the 30 days mentioned earlier in this post, be aware that this is for all-purpose clusters, with the ability to only pin 70 clusters (as of Aug 2021) via the UI or the clusters API.

Summary

Coincidentally, a colleague had an interactive cluster that “suddenly disappeared” and hadn’t accounted for the fact that it hadn’t been used in over 30 days, since it was last terminated. Annoying!

That’s it for today, short and sweet. Until next time.

Thanks for reading.