Lesson 8 of 17Intermediate15 min

Secrets and Configuration

By the end of this lesson

Store credentials in a secret service and read them at run time.

Every application needs values that differ by environment: which database to connect to, how verbose the logs should be, which storage container holds employee photos. Some of those values are harmless. A few of them grant access to something, and those are secrets.

The rule that follows is short and it has no exceptions worth arguing about: a secret does not live in source control. Where it does live is the interesting part, and the answer is a secret store that your application reads at run time using an identity the platform gave it.

Sorting configuration from secrets is the first decision, and it is easier than it looks. Ask whether you would be comfortable with the value appearing in a pull request that anyone in the company can read.

Configuration: log level, page size, feature flag defaults, the storage account or bucket name, the secret store's address, the environment name. These describe your system, and describing your system to a colleague is not a breach
Secret: database password, third-party API key, token signing key, client secret for an identity provider, the shared secret on an inbound webhook
A connection string is usually both. A host and a database name sit alongside a password. Where the driver allows it, keep the password separate so only the sensitive half needs protecting — and prefer a platform identity, which removes the password entirely
Internal hostnames and resource names are not secrets, and they are also not worth publishing. Keep them in configuration and keep the repository private for ordinary reasons
The awkward middle case is anything that is harmless today and sensitive later. A test API key that gets pointed at a real account is the usual example. Treat keys as secrets from the start, whatever they currently unlock

The pieces, with both providers' names:

Secret store: A service built to hold sensitive values, with access control, an audit trail and versioning. Azure Key Vault; AWS Secrets Manager, or Systems Manager Parameter Store with encryption enabled for cheaper, simpler cases.
Secret version: Each write creates a new version and the old one stays readable for a while. This is what makes rotation survivable, because two versions can be valid at the same time.
Rotation: Replacing a credential with a new one on a schedule, and after any suspected exposure. Some stores can rotate a managed database password for you; for everything else it is your process.
Reference: A pointer to a secret, placed in application configuration instead of the value. The platform resolves it at start-up. Azure App Service calls it a Key Vault reference; AWS container services inject a secret by ARN into the task definition.
Platform identity: The managed identity or IAM role from the identity lesson, attached to your running service. It is how the application proves who it is to the secret store without holding a credential of its own.

Both of these put a working value in front of your code. They behave very differently the day something goes wrong.

	Pasted into an application setting	Read from a secret store
Where the value lives	In the platform's configuration screen, and in whatever document or chat message it was copied from	In one place, encrypted, with the store as the single source of truth
Who can read it	Anyone with configuration access to the app, which is usually far more people than need the credential	Only identities granted read access on that secret
Rotation	Find every place it was pasted. This is the step that gets abandoned halfway	Write a new version in one place; instances pick it up on their refresh interval or restart
Audit trail	A configuration change entry, at best. No record of reads	Who read which secret and when, which is the first thing you want after a suspected exposure
Sharing across environments	Tempting, because copying is quick. One credential then unlocks staging and production	One secret per environment falls out naturally, because each environment points at its own store
Setup effort	Almost none	Create the store, enable the identity, grant read access, add the configuration provider
Reasonable when	A throwaway experiment with a credential that unlocks nothing you care about	Anything that reaches real data, including your own development database if it holds a copy of real records

Program.cs — the employees API holds no credential of its own

// The vault address is configuration, not a secret. It grants nothing on its own.
var vaultUri = new Uri(builder.Configuration["Secrets:VaultUri"]
    ?? throw new InvalidOperationException("Secrets:VaultUri is not configured."));

builder.Configuration.AddAzureKeyVault(
    vaultUri,
    // Managed identity when deployed; the signed-in developer locally.
    new DefaultAzureCredential(),
    new AzureKeyVaultConfigurationOptions
    {
        // Rotated values arrive without a redeployment.
        ReloadInterval = TimeSpan.FromMinutes(15),
    });

// Resolve at the point of use so a reloaded value is actually picked up.
builder.Services.AddDbContext<EmployeesContext>((sp, options) =>
{
    var config = sp.GetRequiredService<IConfiguration>();
    var connectionString = config["employees-db-connection"]
        ?? throw new InvalidOperationException(
            "employees-db-connection is missing. Refusing to start.");

    options.UseNpgsql(connectionString);
});

The vault address goes in ordinary configuration and can be committed. Knowing where your vault is does not let anyone read it — access is decided by the identity asking, not by knowledge of the address.
DefaultAzureCredential tries a sequence of sources: the managed identity when the code runs on the provider's compute, and the developer's own sign-in on a laptop. That single line is what removes the key from your repository, because there is no longer a key to remove.
ReloadInterval makes the configuration provider re-read the vault periodically. Without it, a rotated password only reaches your application on the next restart, which means rotation and deployment become coupled.
Resolving the value inside the factory rather than into a variable at start-up is the part that makes the reload useful. A value copied into a local variable during start-up stays at its original version for the life of the process.
Failing to start when a secret is missing is deliberate. An application that starts with a null connection string fails later, on a user's request, with a less obvious error.
On AWS the shape is the same: attach an IAM role to the task, read from Secrets Manager or Parameter Store through the SDK behind a configuration provider, and cache with a refresh interval. The provider changes; the arrangement does not.

Rotating the employees database password without an outage. The order matters, and the reason is in step two.

Create the new credential alongside the old one
Add a second password or a second database login rather than changing the existing one in place. For a moment, two credentials are valid. That overlap is the whole trick.
Write the new value as a new secret version
Your instances are not synchronised. One refreshes now, another in fourteen minutes, a third when it restarts. If you revoked the old credential at this point, every instance that has not refreshed yet would start failing.
Let the fleet pick it up, or restart it deliberately
Either wait out the refresh interval or do a rolling restart. A rolling restart is faster and more predictable, and it is worth preferring when you want to know exactly when the change took effect.
Confirm nothing is still using the old credential
Check the database's connection logs, or the secret store's read audit, for use of the old version. Do not skip this on the assumption that the refresh worked everywhere — background jobs and scheduled tasks are the usual stragglers.
Revoke the old credential
Now delete the old password or login. Until you do, rotation has added a credential rather than replaced one, and the value you were worried about still works.

Common mistake

Common mistakes when starting out

Baking a secret into a container image, or passing it as a build argument. Image layers are stored in a registry, pulled by anything that can reach it, and kept until someone prunes them. The value is now in a place with a different access list from the one you were protecting.

Logging configuration at start-up to check it loaded. That writes the connection string into a log search system that far more people can read, with a retention period measured in weeks. The same applies to exception messages that helpfully include the connection string.

Reading secrets once at start-up and never again, then rotating on a Friday. Instances keep the old value until something restarts them, so the rotation appears to work and then breaks at the next deployment.

Using one secret across development, staging and production. A laptop now holds production access, and rotating the credential breaks three environments at once.

Revoking the old credential first. It feels tidier and it guarantees a short outage, because nothing that has not refreshed yet can connect.

Storing a secret in a pipeline variable that is not marked as secret. Unmarked variables appear in build logs, and build logs are often readable by more people than the production configuration is.

Trade-off

What a secret store costs you

It becomes a dependency in your start-up path. If the store is unreachable, your application does not start. That is usually the correct behaviour, and it does mean your availability now includes the store's availability. Cache the values you have and tolerate a brief outage rather than crashing a running process.

Reads are billed and throttled. Reading a secret on every request is wasteful and will eventually hit a rate limit. A refresh interval measured in minutes is the normal compromise.

Local development needs a path that works without the cloud. A developer sign-in against a non-production store, or the .NET user secrets store on the machine, both work. What does not work is telling people to paste the production value into a local file, which is how secrets spread.

Summary

Configuration describes your system; a secret grants access. The test is whether knowing the value is enough to get in
Secrets belong in a secret store, read at run time by a platform identity, never in source control or an image
A refresh interval decouples rotation from deployment, and only helps if the value is resolved where it is used
Rotate by adding the new credential, letting the fleet pick it up, confirming, then revoking the old one
The store becomes a start-up dependency and a billed, throttled service, so cache with a short refresh rather than reading per request

Practice

Attempt each one before opening the solution. Getting it wrong first is how the idea sticks.

Think about it

Sort the list

Decide which of these are secrets: the vault address; the storage container name; the database password; the log level; a third-party mapping API key; the employees API public hostname; the webhook signing secret.

For each one you called configuration, say what an attacker gains by knowing it.

Show solution

Secrets: the database password, the mapping API key, and the webhook signing secret. Each one grants access or allows something to be forged.

Configuration: the vault address, the container name, the log level and the public hostname. An attacker who learns them gains knowledge of your layout and no access, because every one of those resources checks the identity of the caller.

The second half of the question is the point. If knowing a value gives access, it is a secret. If access is decided by an identity check, the value is a name, and treating names as secrets makes your configuration harder to review for no security benefit.

The mapping API key is worth a second look. Third-party keys are usually billed per call, so an exposed key costs money rather than data. That is still a secret, and it is the kind teams are most casual about.

Try it yourself

Rotate something and time it

In a non-production environment, add a second database credential, write it to your secret store, and record how long each running instance took to start using it.

Then revoke the old one and note whether anything broke.

Show solution

The measurement is the deliverable. Whatever number you get is the minimum length of the overlap window you need during a real rotation, and it is almost always longer than the configured refresh interval because of scheduled jobs and idle connection pools.

If something broke when you revoked the old credential, you found a component that reads its configuration once at start-up and holds it. That is useful to know now rather than during an incident, and it is usually fixed by resolving the value at the point of use.

Doing this in a safe environment first is what turns rotation from an event into a routine. A rotation nobody has rehearsed tends to be attempted for the first time under pressure, immediately after a suspected exposure.

Knowledge check

Nothing is recorded and there is no score. The explanation appears either way.

Why is putting a secret in a container image worse than reading it from a secret store?

Images cannot hold text valuesThe image is stored in a registry with its own access list, and changing the value needs a rebuildSecret stores encrypt values and images do not support encryption at allIt makes the image slower to start

During rotation, why must the old and new credentials both work for a period?

Because secret stores cannot delete old versionsBecause the database needs time to index the new passwordBecause instances pick up the new value at different times, so revoking early breaks the ones that have not refreshedBecause two credentials are needed to encrypt the connection

Saved in this browser only.