All posts

3 min read

Invalidate by version, not by hunting keys

Cache an expensive derived answer under a key that carries a version number, then invalidate everything it touches with one write instead of a key hunt.

Caching the answer is the easy half. The hard half is the day the answer changes and you have to find every cached copy of it. This post is about a way to skip the finding entirely.

The example is Fieldnote, a made-up SaaS where organizations have teams and teams own projects.

The expensive answer

Every page in Fieldnote needs a team's effective settings: the organization's defaults, overridden by the team's own choices, capped by what the organization's plan allows. Working that out takes three queries and a merge, on every request, for something that changes a few times a month.

That is the textbook case for a cache: expensive to compute, read constantly, rarely written.

TypeScript
async function effectiveSettings(teamId: string): Promise<Settings> {
  const key = `settings:team:${teamId}`;
  const hit = await cache.get(key);
  if (hit) return JSON.parse(hit);

  const settings = await computeSettings(teamId);
  await cache.set(key, JSON.stringify(settings), { ttlSeconds: 3600 });
  return settings;
}

The hunt

Now an admin changes an organization default. Every team in that organization has a stale entry. Which keys are those?

The cache does not know: a key is just a string. So you either query for every team in the organization and delete each key, or scan the cache for a prefix, or keep a separate index of which keys belong to which organization and hope it never drifts. All three are slow, and all three have the same failure mode: miss one key, and one team quietly keeps the old setting until its entry expires.

Put a version in the key

Instead of deleting the old entries, make them unreachable. Keep a small counter per scope, and include it in every key:

SQL
create table cache_versions (
  scope   text primary key,
  version bigint not null default 1
);
TypeScript
async function versionOf(scope: string): Promise<number> {
  const row = await db.one(
    "insert into cache_versions (scope) values ($1) on conflict (scope) do update set scope = excluded.scope returning version",
    [scope],
  );
  return Number(row.version);
}

async function effectiveSettings(team: {
  id: string;
  orgId: string;
}): Promise<Settings> {
  const v = await versionOf(`org:${team.orgId}`);
  const key = `settings:org:${team.orgId}:v${v}:team:${team.id}`;
  const hit = await cache.get(key);
  if (hit) return JSON.parse(hit);

  const settings = await computeSettings(team.id);
  await cache.set(key, JSON.stringify(settings), { ttlSeconds: 3600 });
  return settings;
}

Invalidating the whole organization is now one write:

SQL
update cache_versions set version = version + 1 where scope = 'org:' || $1;

The next read builds a key with the new version, misses, and recomputes. Every old entry is still in the cache, but nothing will ever ask for it again, and its TTL clears it out. The TTL stops being your freshness guarantee and becomes garbage collection.

Pick the scope to match the change

The version belongs to whatever a change can affect. An organization default touches every team in it, so that scope is the organization. A team override touches one team, so give teams their own counter and put both in the key:

TypeScript
const key = `settings:org:${orgId}:v${orgVersion}:team:${teamId}:v${teamVersion}`;

Now a team change invalidates one team, and an organization change invalidates all of them, and neither needs to know which keys exist. Too coarse a scope throws away entries that were still right; too fine a scope means a change has to bump many counters, which brings the hunt back.

Bump in the same transaction as the write

The one ordering rule that matters: the version bump commits with the change it describes.

TypeScript
await db.transaction(async (tx) => {
  await tx.query("update org_settings set defaults = $2 where org_id = $1", [
    orgId,
    defaults,
  ]);
  await tx.query(
    "update cache_versions set version = version + 1 where scope = $1",
    [`org:${orgId}`],
  );
});

If the bump ran as its own step after the write, a crash between the two would leave every cached copy stale for its whole TTL. If it ran before the write committed, a reader in that gap could pick up the new version, compute the old settings, and cache them under the new key, where they would be wrong for the whole TTL. Committing both together closes both gaps.

What it costs

Each read now looks up a version before it looks up the cache. That is a primary-key read on a tiny table, cheap enough that it rarely matters, and if it does, the versions themselves can be cached for a few seconds at the price of a few seconds of staleness after a change.

The cache also holds dead entries until they expire. Size the TTL so that is affordable.

In exchange, invalidation can no longer miss a key, because it never touches one. A change is one write, and it is correct for every key whether you know about it or not.

Want the work behind the notes?

The projects show the same habits in code you can open and run.