Caching Strategies

A cache is a copy of an answer, kept somewhere cheaper to reach than the place that produced it. The outstanding-balance figure took 180 milliseconds to calculate across four tables; held in memory it comes back in one. Nothing about the calculation improved. You are reusing the result.

Two questions decide whether a cache helps or hurts, and only the first one gets attention. What is worth copying, and how do you know when the copy has stopped being true? The second question is the whole difficulty. A cache is a second source of truth, and two sources of truth can disagree.

public sealed class CachedInvoiceSummaries(
    IDistributedCache cache,
    IInvoiceSummaryQuery inner,
    ITenantContext tenant) : IInvoiceSummaryQuery
{
    private static readonly DistributedCacheEntryOptions Expiry = new()
    {
        AbsoluteExpirationRelativeToNow = TimeSpan.FromMinutes(2),
    };

    public async Task<InvoiceSummary> GetAsync(int customerId, CancellationToken token)
    {
        // Every input that changes the answer appears in the key, including
        // the tenant. "v2" is bumped when InvoiceSummary changes shape.
        string key = "invoice-summary:v2:t" + tenant.TenantId + ":c" + customerId;

        byte[]? cached = await cache.GetAsync(key, token);
        if (cached is not null)
            return JsonSerializer.Deserialize<InvoiceSummary>(cached)!;

        InvoiceSummary fresh = await inner.GetAsync(customerId, token);

        await cache.SetAsync(key, JsonSerializer.SerializeToUtf8Bytes(fresh), Expiry, token);
        return fresh;
    }
}

• The class implements the same interface it wraps, so callers are unchanged and the cache can be removed by altering one registration. That also means no caller can accidentally read around it.
• The tenant identifier is in the key because the answer depends on it. Any input that changes the answer belongs there: tenant, user, role, currency, locale, permission scope. Leave one out and the cache serves one caller's answer to another.
• The version segment is an escape hatch. Add a field to InvoiceSummary and bump it to v3, and existing entries are ignored rather than deserialised into the wrong shape after a deployment.
• Two minutes is a decision, not a default. It says: the business accepts that this figure can be up to two minutes old. Someone should agree with that sentence before it ships.
• This code has a gap worth spotting now. Under load, two hundred concurrent requests for the same missing key all miss, and all two hundred run the underlying query. The cache was meant to protect the database and at that moment it does the opposite.
• The serialisation is not free either. Caching an object that takes three milliseconds to serialise in order to avoid a two-millisecond query makes things worse, which is why hit rate and timing are worth measuring rather than assuming.

There are two ways to stop serving something that is no longer true. Let it expire, or remove it when the underlying data changes. They fail differently, so most systems need both.

Expiry is self-healing and needs no cooperation from anyone. It also guarantees a window in which you knowingly serve the wrong answer. Explicit removal is exact and fragile: every path that changes the data has to remember, including the overnight job, the admin screen, the import and the support script somebody runs by hand. The path that forgets is always the one you did not know about.

A third option avoids removal entirely: put a version in the key and change the version. Old entries are never read again and age out on their own. That works well when you cannot enumerate keys, and it costs memory you are paying for until the entries expire.

Whichever you choose, be honest with yourself about what you have built. There are now two places that answer the same question, and they can disagree. When a report shows one total and the detail screen shows another, this is usually the reason, and the investigation is slow because both numbers are internally consistent.

builder.Services.AddHybridCache(options =>
{
    options.DefaultEntryOptions = new HybridCacheEntryOptions
    {
        Expiration = TimeSpan.FromMinutes(5),             // shared copy
        LocalCacheExpiration = TimeSpan.FromSeconds(20),  // in-process copy
    };
});

public sealed class InvoiceSummaries(
    HybridCache cache,
    IInvoiceSummaryQuery inner,
    ITenantContext tenant)
{
    public ValueTask<InvoiceSummary> GetAsync(int customerId, CancellationToken token) =>
        cache.GetOrCreateAsync(
            key: "invoice-summary:v2:t" + tenant.TenantId + ":c" + customerId,
            state: (inner, customerId),
            factory: static async (s, ct) => await s.inner.GetAsync(s.customerId, ct),
            tags: ["customer:" + customerId],
            cancellationToken: token);

    // Called by the write side, after the transaction has committed.
    public ValueTask InvalidateAsync(int customerId, CancellationToken token) =>
        cache.RemoveByTagAsync("customer:" + customerId, token);
}

• GetOrCreateAsync runs the factory for one caller per key and makes the rest wait for that result. The two hundred concurrent misses from the previous example become one query, without a lock you had to write.
• Two layers, two lifetimes. The in-process copy removes a network hop for twenty seconds; the shared copy lasts five minutes. The consequence is worth stating: for up to twenty seconds, different instances can return different answers, so the maximum staleness a user can see is the longer of the two windows.
• Tags let the write side invalidate everything about one customer without knowing which keys exist. Keeping a list of key patterns in the write code is the alternative, and it drifts out of date within a release or two.
• Invalidate after the commit, never before. Remove the entry first and a concurrent read can repopulate it from the uncommitted state, leaving the cache wrong until it expires — a defect that appears under load and cannot be reproduced by hand.
• The static factory with a state tuple avoids capturing a closure on every call. A small thing, and free once you know the shape.
• What this does not do is make the cache safe to depend on. If the shared cache is unreachable, the call should behave like a miss and serve a slower correct answer. An application that fails when its cache fails has swapped one dependency for two.