Secret redaction caution

Databricks
2 min
In this article

Secret redaction within Databricks is a great feature that helps to prevent exposure of your secrets unintentionally. This post will look at a short demo of why we need to remain cautious of secret exposure, even with secret redaction in place.

Setup

To follow along:

1. Create a secret scope (either Key vault or Databricks is fine, although I have both)

2. Add at least 1 secret in your secret

3. Optionally install Databricks CLI

Happy path

For cases where only authorised personnel have access to the workspace and they intentionally or unintentionally issue commands that may expose secrets, secret redaction will stop that.

I have 2 secret scopes, one key vault backed and another, Databricks backed as shown in the Databricks CLI output below:

Databricks CLI output

For my “demo” secret scope, I have a number of secrets that I can observe using dbutils.secrets.list(“demo”). The following example shows this:

List secret scopes, secrets and redaction

Printing any one of those secrets will make it redacted, which is great!

Exactly what we would want to happen. Fantastic!

Not so happy path

If you pay the following URL a visit (aka RTFM), you’ll notice a warning related to secret redaction: Secret redaction — Databricks Documentation

The cautious bit here is that even with redaction in place, it is still possible to expose secrets we’ve worked hard to hide away. Do not simply rely on secret redaction to halt exposure.

Look at the following example of said exposure and notice the last 2 characters in plain text. Shock horror!

No redaction

Take away

Enforce the principle of least privilege to limit access for users.

Principle of least privilege - Wikipedia

It is important not only to grant the right users access but also, the right level of access of the users.

In Databricks you can limit access by preventing users from issuing run commands and for more on this you can see the Databricks documentation on workspace access control: Workspace object access control — Databricks Documentation

All in all, it is wonderful to have features such as secret redaction so long as we continue to remain vigilant.

Thanks for reading.